HSCI 241 · Lesson 11

Environmental Scans

Finding & Synthesizing Health Evidence

Learning objectives for this lesson:

  • Define an environmental scan, describe its origins in organizational planning, and explain how health services and public health organizations use scans.
  • Distinguish passive from active scanning, and distinguish an environmental scan from a literature review, a needs assessment, an asset map and a program evaluation.
  • Plan a document and website review and build a program inventory that records every candidate program and the decision made about it.
  • Design a short survey of organizations and a key informant interview guide, and plan contact, consent and data governance for both.
  • Define the unit of a scan, write eligibility criteria for it, build a sampling frame from several sources and judge the frame's completeness.
  • Choose between a census and a purposive sample for each scan method, and plan the sampling of jurisdictions for a comparison across provinces.
  • Design and pilot a charting template with a data dictionary, a source for each entry and a date of verification.
  • Combine a scan with a literature review through shared categories and a joint display, and report a scan with a flow of programs, stated denominators, limitations and a date of currency.
  • Build a sampling frame, inventory and charting table for a scan of five to ten organizations, and draft a joint display that links a scan with a review.

This course was developed by Dr. Kiffer G. Card, Faculty of Health Sciences, Simon Fraser University, drawing on the Cochrane Handbook for Systematic Reviews of Interventions and the JBI Manual for Evidence Synthesis.

Lesson 11 · HSCI 241

Environmental Scans

A short guided walkthrough before you work through the lesson at your own pace.

Finding and Synthesizing Health Evidence
Why this lesson matters

Two questions behind one decision

A review says what research has found; a scan says what exists in a place now.

Review

What has been evaluated, and with what outcomes?

Scan

Which programs operate here now, who runs them, and how are they funded and evaluated?

The road map

Four sections

1 · What a scan is

Definitions, origins, uses in health, passive and active scanning.

2 · Scan methods

Documents, inventories, surveys and key informant interviews.

3 · Sampling and charting

Units, sampling frames, census or sample, charting templates.

4 · Integration and reporting

Joint displays with the review, denominators, limitations and currency.

Running case

The Cedar Valley scan (fictional)

18programs identified in British Columbia
11programs answered the survey
7programs took part in interviews

The scan ran in weeks 3 to 10 of the twelve-week project, alongside the rapid scoping review.

How to work through it

Read, watch, check and apply

  • Each section has a reading, a narrated deck, a quiz and a reflection.
  • The final page holds a fifteen-item assessment and a final reflection.
  • The worked examples follow the Cedar Valley scan from sampling frame to report.
Section 1 of 5

What an Environmental Scan Is: Origins and Uses in Health

⏱ Estimated reading time: 35 minutes
Section 1 of 5

What an Environmental Scan Is

Definitions, origins, uses in health, passive and active scanning, and related products.

Definition

What published definitions share

Discovery

A scan finds out what is there.

Synthesis

A scan combines many sources into one account.

Forward-looking

A scan notes trends and coming changes.

Decisions

A scan is done to inform a planning decision.

Charlton and colleagues (2021) found no standard definition across 96 studies.

Origins

Aguilar’s four modes of scanning (1967)

Undirected viewing

General exposure with no specific purpose.

Conditioned viewing

Directed exposure to selected sources.

Informal search

A limited, unstructured effort to find specific information.

Formal search

A deliberate, planned effort that follows a procedure.

Uses in health

Published examples

Rowel et al. (2005)

A scan for a cancer screening project in Baltimore, with focus groups and interviews.

Wilburn et al. (2016)

A scan for HPV vaccination in Kentucky and a seven-step process.

CADTH (2015)

Scans of practice inside and outside Canada, giving a snapshot without recommendations.

Modes and sources

Passive and active scanning

Passive

Literature, websites, documents and directories; codified knowledge.

Active

Organizational surveys and key informant interviews; tacit knowledge.

In Charlton and colleagues’ review, 53 percent of scans combined the two approaches.

Scope

The Cedar Valley scan question

Which organizations currently deliver community-based programs for socially isolated older adults, and how are those programs designed, governed, funded and evaluated, in British Columbia and elsewhere in Canada?

Sub-questions cover location and operators, activities and referral, staffing and funding, evaluation, barriers and advice, and models in other provinces.

Carry forward

From definition to method

  • A scan describes what exists in a defined setting at a stated time.
  • Most scans combine passive and active methods.
  • Section 2 works through the four methods the Cedar Valley team used.

Learning Objectives for this section

  • Define an environmental scan and describe the features that published definitions of scans share.
  • Trace the origins of environmental scanning in organizational planning, including Aguilar's four modes of scanning.
  • Describe how environmental scans are used in health services and public health, with published examples.
  • Distinguish passive from active scanning, internal from external sources, and codified from tacit knowledge.
  • Distinguish an environmental scan from a literature review, a needs assessment, an asset map and a program evaluation.
  • Write a scan question with sub-questions, using the Cedar Valley scan as a model.

Introduction

Lesson 1 introduced the environmental scan as a close relative of the review that studies current programs and services through documents, websites, surveys and interviews. In Lesson 2 the Cedar Valley team wrote a scan question beside its review question, and Lessons 5 and 7 noted that many grey-literature documents describe community programs in detail with little or no evaluation, which makes them useful to a scan even when a review cannot include them as studies. This lesson teaches scan methods in full.

Section 1 defines the scan and describes its origins and uses. Section 2 covers four methods: document and website review, program inventories, surveys of organizations and key informant interviews. Section 3 covers the sampling of organizations and jurisdictions and the charting template, and Section 4 shows how to combine a scan with a literature review and how to report it.

Case: The Cedar Valley evidence review (fictional)

The fictional Cedar Valley Health Authority in British Columbia serves about 210,000 residents, of whom about 46,000 are aged 65 and older, across Cedar City, several smaller communities and 24 primary care clinics. Before launching a community connector (social prescribing) program for older adults, its planning team asked a small evidence team, made up of a health authority evidence officer, a university librarian and a student intern, for a rapid scoping review and an environmental scan within twelve weeks.

The review, which asks which community-based interventions have been evaluated for reducing loneliness or social isolation among adults aged 65 and older, includes 42 studies and 26 grey-literature documents. The scan asks which such programs currently operate in British Columbia and elsewhere in Canada, who runs them, and how they are funded and evaluated. It ran in weeks 3 to 10 of the project, alongside the review, and identified 18 programs operating in British Columbia, of which 11 responded to a short survey and 7 took part in key informant interviews. Other numbers in this lesson are illustrative and consistent within it.

1.1 What an Environmental Scan Is

An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, so that an organization can plan. In this phrase the environment is the organizational setting around the people who are planning. For a health authority, that setting includes community organizations, municipal and provincial governments, funders, professional groups and the residents it serves.

Published definitions vary. Following a protocol published in BMJ Open (Charlton et al., 2019), Charlton and colleagues (2021) reviewed 96 studies that used environmental scans in health services delivery research and reported that the field lacks a standard definition and uses its terms inconsistently. Their thematic analysis of how these studies defined or described scans produced several themes, including the four shown in the cards below, and the authors proposed a working definition for health services delivery. In a later survey of 47 people, mostly in Canada, who had conducted or used scans, Charlton and colleagues (2024) found that respondents often described a scan as a snapshot of a defined setting and found scans hard to distinguish from needs assessments.

Select each card to read what the theme means and how it applies to the Cedar Valley scan.

Instrument of discoveryClick to explore
Knowledge synthesisClick to explore
Forward-lookingClick to explore
Decision makingClick to explore

Taken together, these descriptions point to five features of a well-designed scan: it covers a defined setting at a stated time, it serves a stated decision, it draws on several kinds of sources, it follows a documented procedure that someone else could repeat, and it produces a structured account of what exists.

1.2 Origins in Organizational Planning

Environmental scanning began in business. In his 1967 book Scanning the Business Environment, Francis Aguilar described scanning as the acquisition of information about events and relationships outside a company that would help its senior managers plan the company's future course of action. Aguilar distinguished four modes of scanning, which differ in how purposeful and structured the search for information is. Chun Wei Choo (2001) later linked these modes to research on information seeking and organizational learning, and argued that organizations use all four, moving from broad viewing to formal search as their questions become specific.

Mode (Aguilar, 1967)What it involvesExample in a health authority
Undirected viewingGeneral exposure to information with no specific purpose in mindA manager reads a seniors' newsletter and notices a story about a new friendly-visiting program.
Conditioned viewingDirected exposure to selected sources or topics, while staying alert to what they showPlanners subscribe to updates from the provincial ministry and from a social prescribing network.
Informal searchA limited, unstructured effort to find specific informationA planner phones two colleagues to ask which local groups run programs for isolated older adults.
Formal searchA deliberate, planned effort that follows a stated procedureThe evidence team searches defined directories, surveys every eligible program and interviews selected leads.

An environmental scan as this course teaches it is a formal search in Aguilar's sense. The other three modes still matter, because they often prompt a scan and supply leads that the formal search then checks.

Scanning became part of strategic planning, where its results feed tools such as a SWOT analysis, which sets an organization's internal strengths and weaknesses beside the external opportunities and threats it faces. Many scans also group external information under political, economic, social and technological headings (PEST). Graham, Evitts and Thomas-MacLean (2008) note that scanning spread from business to the non-profit sector, where the United Way of America used comprehensive scanning in the early 1980s, and that health researchers have since used scans in areas such as mental health, nutrition, women's health, Indigenous health, housing and knowledge transfer. The timeline below shows the publications this lesson draws on.

1967Aguilar describesbusiness scanning Early 1980sScanning used innon-profit planning 2005Rowel et al.: a scanin public health 2008Graham et al.: scansin primary care 2016Wilburn et al.:seven-step process 2019 to 2024Charlton et al.:review and survey Teal: origins in organizational planning. Red: health applications and methods research.
Figure 11.1. Environmental scanning moved from business planning to non-profit planning and then to health services and public health, where methods research on scans has grown since the late 2010s.

1.3 Uses in Health Services and Public Health

Charlton and colleagues (2021) found that health services researchers conducted scans for many purposes, of which the most common was to examine the current state of programs, services or policies. Scans informed recommendations in 20 percent of the 96 studies they reviewed. Health organizations use scans to plan new programs, compare policies across provinces or countries, map the services available to a population, find gaps and partners, and prepare for a needs assessment or an evaluation.

Case: A scan for a cancer screening program in Baltimore (Rowel et al., 2005)

Rowel and colleagues (2005) described an environmental scan conducted in 2003 as part of the needs assessment for a project to increase cancer screening among African Americans in Baltimore, Maryland. The scan combined a review of community organizations and interest holders, local cancer incidence data, and community assets and liabilities with focus groups and key informant interviews with service providers. The authors reported that the scan gave the project team quick access to information about local events, trends and relationships, and they described the scan as flexible, responsive and cost-effective, with the added benefit of mobilizing the community. They also observed that the environmental scan was poorly defined, rarely evaluated and seldom used in public health, and that little guidance existed on how to design, conduct or evaluate one.

Wilburn, Vanderpool and Knight (2016) described a scan for a statewide project on human papillomavirus (HPV) vaccination in Kentucky. Their scan combined interviews, a survey of health care providers, reviews of the literature and policy, a scan of television coverage and registry data. From that experience they proposed a seven-step process, which the table below sets beside the Cedar Valley scan.

Step (after Wilburn, Vanderpool and Knight, 2016)How the Cedar Valley team applied it
1. Determine leadership and capacity for the projectThe evidence officer led the scan; the intern built the inventory and charted programs; the librarian advised on directories and websites.
2. Establish the focal area and purpose of the scanThe scan question from Lesson 2, tied to the design of a community connector program.
3. Create and keep to a timeline with incremental goalsWeeks 3 to 10 of the twelve-week project, with the sampling frame closed by the end of week 7.
4. Determine the information to be collectedA charting template whose fields answer each sub-question (Section 3).
5. Identify and engage interest holdersThe planning team, program leads and the health authority's Indigenous health team (Lesson 2).
6. Analyze and synthesize the results into a concise summary reportCounts with stated denominators, a framework summary of interview notes and a joint display with the review (Section 4).
7. Disseminate the results to interest holdersA scan report for the planning team, a summary for participating programs and a section of the evidence brief (Lesson 12).

In Canada, CADTH, now Canada's Drug Agency, has published environmental scans as one of its report types. Its 2015 process document describes scans of health care practices, processes and protocols inside and outside Canada, based on a limited literature search, a survey of Canadian health care decision-makers, or both. These scans give a snapshot of current practice and do not make recommendations for or against a technology (CADTH, 2015).

1.4 Passive and Active Scanning, and the Sources a Scan Uses

Graham, Evitts and Thomas-MacLean (2008) distinguish two modes of scanning. Passive scanning collects knowledge that already exists, such as published literature, reports and information from established contacts. Active scanning creates new knowledge with data collection tools such as surveys and interviews, and it includes a willingness to revise what the team thought it knew. The same authors distinguish internal sources, such as an organization's own memos, meeting minutes and reports, from external sources about the political, social, economic and technological context. They also distinguish codified knowledge, which is written down in reports, statistics and policy documents, from tacit knowledge, which lives in people's experience and comes out in interviews, focus groups and conversations.

Most scans in health use both modes. In the review by Charlton and colleagues (2021), the most common methods were literature reviews (71 percent of studies), key informant or semi-structured interviews (46 percent) and surveys (35 percent), and 53 percent of studies combined passive and active approaches.

Passive scanning Looks at information that exists Active scanning Generates new information Publishedliterature Websites anddocuments Directoriesand lists Organizationsurvey Key informantinterviews Focus groups,site visits UsedUsedUsedUsedUsed Not used The Cedar Valley scan combined three passive and two active methods. Its literature component is the rapid scoping review itself (Section 4).
Figure 11.2. Scan methods arranged from passive to active. Passive methods are quicker and cheaper; active methods reach tacit knowledge and information that no document records.

1.5 How a Scan Differs from Related Products

A scan overlaps with several other products that health organizations commission, and the differences lie mainly in the question asked and the sources used. The table compares them.

ProductMain questionMain sourcesTypical output
Literature review (Lessons 1 to 10)What has research found?Published and grey research reportsA synthesis of studies
Environmental scanWhat exists now in a defined setting, and what is changing?Websites, documents, directories, organizations and peopleAn inventory and structured description of programs, policies and trends
Needs assessmentWhat does a population need, and how do needs compare with services?Population data, resident surveys and consultationsA ranked list of needs and service gaps
Asset mappingWhat strengths and resources does a community have?Residents, community groups and local knowledgeA map or list of community assets
Program evaluationDoes this program work, and how?Program records, participants and staffA judgement about one program
Jurisdictional scanHow do other provinces or countries handle this issue?Legislation, policies, reports and officialsA comparison table across jurisdictions

The boundaries are porous. A scan often supplies the service side of a needs assessment, and a jurisdictional scan is a scan whose unit is a province or country (Section 3).

1.6 The Cedar Valley Scan Question

In Lesson 2 the team wrote its scan question with the same framework as its review question, treating organizations as the population: which organizations currently deliver community-based programs for socially isolated older adults, how those programs are designed, governed, funded and evaluated, in British Columbia and elsewhere in Canada. The team then wrote sub-questions, each of which points to the sources most likely to answer it.

Sub-questionMain sources
1. Which programs operate in British Columbia, where, and who runs them?Directories, funder lists, websites and referrals
2. What do the programs offer, to whom, and how do participants reach them, including referral from primary care?Websites, documents and the survey
3. How are the programs staffed and funded, and how secure is their funding?The survey, annual reports and funder documents
4. Are the programs evaluated, and with which measures?The survey, evaluation reports and interviews
5. What do program leads see as barriers and facilitators, and what would they advise a health authority starting a connector program?Key informant interviews
6. Which models in other provinces might inform the Cedar Valley program?Public documents and websites from selected provinces

The scan's methods were set out in a short appendix to the review protocol (Lesson 2). Section 2 describes each method and the ethical questions it raises.

Try it: Scan, review or something else?

For each request a health authority might receive, decide whether an environmental scan, a literature review, a needs assessment or a program evaluation is the best primary product. (1) "Which organizations in our region offer transportation for older adults to medical appointments, and what do they charge?" (2) "Do exercise programs reduce falls among adults over 75?" (3) "Did our two-year pilot of a walking group reduce loneliness among participants?" (4) "How do the other provinces fund community paramedicine, and who governs it?" (5) "What are the unmet health and social needs of older adults living alone in our rural communities?"

Check your answersv

(1) An environmental scan, because the question asks what exists now in a defined region and who provides it. (2) A literature review, because the question asks what research has found about an intervention's effect. (3) A program evaluation, because the question concerns the effect of one local program. (4) A jurisdictional scan, which is a scan whose units are provinces; its sources are policies, reports and officials. (5) A needs assessment, because the question asks about the needs of a population, although a scan of available services would usually form one part of it.

Reflection

A public health unit in a mid-sized Canadian city plans to launch a peer-support program for the mental health of international post-secondary students, and asks you for an environmental scan. An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, to inform planning. Passive scanning gathers information that already exists, such as documents, websites and directories; active scanning generates new information, for example through surveys of organizations and key informant interviews. (a) Write a scan question and three sub-questions. (b) Name two passive and two active sources you would use and say what each would add. (c) In two or three sentences, explain how your scan differs from a literature review (which asks what research has found) and from a needs assessment (which asks what a population needs and how its needs compare with services).

Model answer

(a) Scan question: which organizations in the city currently offer mental health supports for international post-secondary students, what do those supports involve, and how are they staffed, funded and evaluated? Sub-questions: which supports exist on and off campus, and who runs them; how do students find and reach them, including in languages other than English; and which supports use peers, and how are peer supporters trained and supervised?

(b) Passive sources: the websites and annual reports of the city's colleges and universities, which list campus counselling and international student services; and the regional community services directory, which lists settlement agencies and community mental health services. Active sources: a short survey of service managers, which adds facts that websites omit, such as staffing, hours, languages offered and whether any outcomes are measured; and key informant interviews with three or four managers and a peer-support coordinator, which explain why students do or do not use services and what a new program should avoid duplicating.

(c) A literature review would tell the unit what studies of peer support for international students have found, wherever they were done, while the scan describes what exists in this city now. A needs assessment would ask students what they need; the scan describes the services side, and its results could form one part of a later needs assessment.

Minimum 20 characters required.

✓ Reflection saved
Knowledge Check: this section

Question 1: Which finding did Charlton and colleagues (2021) report from their scoping review of 96 studies that used environmental scans in health services delivery research?

Charlton and colleagues (2021) identified the absence of a standard definition and inconsistent terminology as research gaps, and they proposed a working definition. Literature reviews were the most common method in the scans they reviewed (71 percent), and the most common purpose was to examine the current state of programs, services or policies, which rules out the third and fourth options.

Question 2: A health authority planner phones two colleagues to ask which local groups run programs for isolated older adults. Which of Aguilar's four modes of scanning does this illustrate?

The planner seeks specific information, which rules out the two viewing modes, but does so through a limited and unstructured effort with no stated procedure. A formal search, such as the Cedar Valley scan, follows a planned and documented procedure.

Question 3: Graham, Evitts and Thomas-MacLean (2008) distinguish passive from active scanning. Which activity is an example of active scanning?

Active scanning generates new information through data collection tools such as interviews and surveys. The other three options gather information that already exists in documents, directories or publications, which is passive scanning.

Question 4: Which request is best answered by an environmental scan as the primary product?

A scan asks what exists now in a defined setting and who provides it. The first request calls for a literature review, the second for a program evaluation, and the fourth for a needs assessment, although a scan of services might form part of it.
Section 2 of 5

Scan Methods: Documents, Program Inventories, Surveys and Interviews

⏱ Estimated reading time: 40 minutes
Section 2 of 5

Scan Methods

Documents and websites, program inventories, organizational surveys, key informant interviews and ethics.

Method 1

Document and website review

  • Each search is logged with its source, date, terms and searcher.
  • A saved copy shows what each page said when the scan used it.
  • A field coded “not reported” describes what the source says about the program.
Method 2

The program inventory

Listing

One mention of a program in one source.

Program

The service itself, counted once.

Ordered reasons: not operating; facility-based; no social connection component; not mainly for adults 65 and older; one-time event.

Method 3

Surveying organizations

11 of 18programs responded (61 percent)
5 of 7non-respondents were volunteer-run, against 3 of 11 respondents

Short items, one idea each, a stated reference period and a “don’t know” option.

Method 4

Key informant interviews

  • The guide covers history, referral, partnerships, funding, evaluation and advice.
  • Ten programs were invited and seven took part, one informant each.
  • Each informant checked the summary of their interview.
Ethics

Review, consent and data governance

TCPS 2

Article 2.5 places quality improvement and program evaluation outside board review.

Consent

Participation was voluntary and quotations were attributed by role.

OCAP

Indigenous-led organizations decided what information about their programs was reported.

Carry forward

From methods to sampling

  • Each method fills gaps left by the one before it.
  • Every search, decision and contact is logged.
  • Section 3 covers units, sampling frames and the charting template.

Learning Objectives for this section

  • Plan a document and website review for a scan, with a log that records each source, the date it was searched and a saved copy of what was found.
  • Build a program inventory that lists every candidate program once and records the decision made about it.
  • Design a short survey of organizations with clear items, suitable response options and a contact plan, and report its response rate.
  • Write a key informant interview guide with main questions and probes, and plan how interview notes will be kept and checked.
  • Identify the ethical questions a scan raises, including research ethics review, consent, confidentiality and Indigenous data governance.

Introduction

Section 1 showed that most scans combine passive methods, which gather information that already exists, with active methods, which generate new information from organizations and people. This section works through the four methods the Cedar Valley team used, in the order in which it used them. Each method filled gaps left by the one before it, as Figure 11.3 shows.

Documents andwebsites Weeks 3 to 6 Adds: what programssay about themselves Programinventory Weeks 3 to 7 Adds: one list witha decision for each Organizationsurvey Weeks 5 to 7 Adds: funding, reachand evaluation facts Key informantinterviews Weeks 6 to 8 Adds: how and why,barriers and advice Teal: passive methods. Red: active methods.
Figure 11.3. The four methods of the Cedar Valley scan, with the weeks of the twelve-week project in which each ran and the information each added.

2.1 Document and Website Review

A document and website review searches the public record for programs and reads what it says about them. For a scan of community programs, the most useful sources are organization websites, annual reports, reports to funders, program brochures, municipal and health authority lists of community resources, provincial directories and the reports of funding programs. The Cedar Valley team began with the sources it had already found for the review. Lesson 5 described the targeted website searches, and Lesson 7 showed that the review excluded 12 documents from other sources because they described a program without evaluating it. Those 12 documents, together with many of the 26 grey-literature documents that the review included, became the first documents of the scan.

The team then searched sources chosen for the scan. These included the provincial 211 information and referral directory (bc211), which lists community and social services; the list of local programs funded through Better at Home, a provincially funded program that supports older adults to live independently; the community resource lists of British Columbia's regional health authorities; and the websites of municipalities and seniors' centres. Section 3 explains how these sources were combined into a sampling frame.

The documentation rules from Lesson 5 apply here. For each search the intern recorded the source, the date, the search terms or browsing path, the number of listings found and the person who searched. For each document kept, the intern saved a PDF or archived copy of the web page, because program websites change and disappear, so that a reader can later see exactly what the source said when the scan used it.

Reading program documents critically

Program websites and brochures are written to attract participants and funders, rarely report funding, reach or evaluation, and may describe a program that has since closed or changed. The Cedar Valley team treated documents as the starting point for each program's record, confirmed key facts through the survey or interviews where it could, and coded information that a document did not mention as "not reported". A field coded "not reported" describes the source and says nothing about whether the program has the feature.

2.2 The Program Inventory

A program inventory is a running list of every candidate program the scan identifies, with the source or sources where each was found and the decision made about it. It plays the part that the reference library and screening record play in a review (Lesson 7), and as in a review the same item turns up in several sources. The team therefore distinguished a listing, meaning one mention of a program in one source, from a program, meaning the service itself. De-duplicating listings into programs comes first, and screening programs against the eligibility criteria (Section 3) comes second.

The inventory template below shows the fields the intern used, with an example entry for each of three kinds of decision. Candidate programs carry inventory codes, eligible programs later receive charting codes P01 to P18, and the descriptions are fictional.

FieldEntry for INV-07Entry for INV-12Entry for INV-41
Inventory IDINV-07INV-12INV-41
Program and operatorTelephone befriending service run by a volunteer seniors' societyHome support service (housekeeping and yard work) run by a non-profitWeekly lunch club run by a church volunteer group
Listings and sources3: provincial directory, municipal website, grey-literature document G-142: provincial directory, funder list1: referral from a survey respondent
Web address or contactSaved copy W-102Saved copy W-117Coordinator's public email address
Date first foundWeek 3Week 3Week 6
LocationElsewhere in British Columbia, ruralCedar Valley regionCedar Valley region
DecisionInclude (charted as P05)ExcludeUnclear, then include after a phone call confirmed it was active (charted as P16)
Reason (first that applies)Not applicable3. No social connection componentNot applicable
Decided by and checked byIntern; checked by evidence officerIntern; checked by evidence officerIntern and evidence officer together

Exclusion reasons were recorded from an ordered list, so that each excluded program has one reason and the counts add up, as in full-text screening (Lesson 7). The order was: (1) not currently operating; (2) based in a facility such as a long-term care home or hospital; (3) no component that addresses social connection; (4) not mainly for adults aged 65 and older; and (5) a one-time event. Because the scan was rapid, the intern made each decision and the evidence officer checked every exclusion and a sample of inclusions, and the two decided every unclear case together. Section 4 reports the counts.

2.3 Surveying Organizations

Documents rarely say how a program is funded, how many people it reaches or whether it is evaluated, and those were among the questions the planning team cared about most. A short survey of the organizations that run eligible programs can fill these gaps. An organizational survey asks one informed person, usually the program coordinator or manager, to report facts about the program on its behalf.

Several design principles apply. The survey should be short; the Cedar Valley survey took about ten minutes. It should ask only what documents cannot answer, and it can show what the team has already charted and ask the respondent to confirm or correct it. Closed items should have response options that cover every possibility, including "don't know", and questions about money can offer ranges and a "prefer not to say" option. Each item should ask about one thing and state its reference period, such as "in the past 12 months". The survey should be piloted with one or two people similar to the respondents, which the team did with the coordinators of two programs that fell outside the scan's eligibility criteria.

Contact procedures also affect response. Dillman, Smyth and Christian (2014) recommend a planned sequence of personalized contacts, each with a different purpose. Surveys of organizations tend to have lower response rates than surveys of individuals (Baruch and Holtom, 2008), so a scan should plan its follow-up from the start. The Cedar Valley team sent an advance email from the health authority's planning lead explaining the purpose, an invitation from the evidence officer with a link to the survey, two reminders a week apart and a final telephone call, and kept the survey open for three weeks (weeks 5 to 7).

ItemCedar Valley survey wording (abridged)Response options
1We have recorded the program's name, operator and contact as shown. Please correct anything that is wrong.Pre-filled fields with space for corrections
2Is the program currently operating?Yes; paused (please give the date it paused); ended (please give the date)
3Which one of the following best describes the program's main activity?Group activity; one-to-one befriending or telephone contact; community connector or social prescribing; intergenerational; technology-based; other (please describe)
4How do participants usually come to the program? Select all that apply.Self-referral; family or friends; a primary care clinician; a community connector or another agency; other
5How is the program mainly delivered?In person; by telephone; online; a mix
6In the past 12 months, about how many older adults took part?Fewer than 25; 25 to 49; 50 to 99; 100 to 249; 250 or more; don't know
7Who delivers the program?Paid staff only; volunteers only; paid staff and volunteers
8Which sources fund the program this year? Select all that apply, then mark the largest.Provincial government; health authority; municipality; charitable foundation or United Way; donations or fees; other; don't know
9When does the program's current funding end?Within 12 months; in one to three years; ongoing with no end date; don't know
10In the past three years, which of these has the program done? Select all that apply.Commissioned an external evaluation; measured participant outcomes itself; counted activities or participants only; none of these; don't know
11If the program measures participant outcomes, which questionnaires or measures does it use?Open text
12Could you share any evaluation report from the past three years?Yes (upload or email); no; none exists
13Are there other programs in British Columbia that we should include?Open text
14Would you be willing to take part in a 30- to 45-minute interview?Yes; no

Item 3 uses the intervention categories from the review's charting form (Lesson 8), so that the scan and the review can later be compared category by category (Section 4). Item 13 is a form of snowball sampling, discussed in Section 3. Of the 18 eligible programs, 11 completed the survey, a response rate of 61 percent (11 divided by 18). The team reported both numbers and compared respondents with non-respondents on what the documents showed, and found that 5 of the 7 non-responding programs were run entirely by volunteers, compared with 3 of the 11 that responded. Survey results therefore probably describe paid-staff programs better than volunteer-run ones, a likely case of non-response bias, and the report says so.

Try it: Repair three survey items

A colleague drafted these items for a survey of organizations that run programs for isolated older adults. Identify the problem with each and rewrite it. (a) "How many participants does your program serve and how many volunteers does it have?" (b) "Given the strong evidence that befriending reduces loneliness, does your program offer befriending?" (c) "How many people have taken part in your program?"

Check your answersv

(a) The item is double-barrelled: it asks two questions with one answer space. Split it into two items, each with ranges and a "don't know" option. (b) The item is leading, because its preamble signals the answer the team hopes for and makes a claim that the review has not yet established. Ask simply, "Does your program offer one-to-one befriending, either in person or by telephone?" (c) The item has no reference period, so a program that started last month and one that started in 1998 will answer different questions. Ask, "In the past 12 months, about how many older adults took part in the program?" and offer ranges.

2.4 Key Informant Interviews

A key informant is a person who knows about a topic because of the position they hold, such as a program coordinator, a funder or a health authority manager. A key informant interview asks such a person about the program or system they know, more than about their own experience as an individual. Key informant interviews suit the how and why questions that a survey handles poorly, such as why referrals from clinicians are rare and what a newcomer should avoid. HSCI 207 Lesson 10 teaches how to write an interview guide, how to probe and how to pilot a guide; this section applies those skills to a scan.

The Cedar Valley interviews lasted 30 to 45 minutes and took place by video call or telephone in weeks 6 to 8. The evidence officer asked the questions and the intern took notes. With consent, calls were recorded so that notes could be checked, and within a day the intern completed a summary organized by the guide's topics. Each informant received the summary of their interview and was invited to correct it, a form of member checking. The main questions and probes of the guide appear below.

Opening and the program's historyv

Main question: "Could you tell me about the program and how it started?" Probes: "Who started it, and why?" "How has it changed since then?" This opening question is easy to answer and lets the informant set the scene.

Reaching participantsv

Main question: "How do older adults usually find their way to the program?" Probes: "Do clinicians or other agencies refer people to you? How does that work in practice?" "Which older adults do you think the program does not reach?"

Partnershipsv

Main question: "Which organizations does the program work with most closely?" Probes: "What does the program's relationship with primary care look like?" "Have you worked with a community connector or social prescribing service? What happened?"

Funding and sustainabilityv

Main question: "How secure is the program's funding over the next few years?" Probes: "What happens to the program when a grant ends?" "What would help the program plan further ahead?"

Evaluationv

Main question: "How does the program know whether it is helping people?" Probes: "What, if anything, do you measure?" "What would you measure if you had the time or support?"

Advice for a new programv

Main question: "Our health authority is planning a community connector program for older adults. What advice would you give us?" Probes: "What should we avoid?" "How could a new program work with yours without competing for the same participants or volunteers?"

The team invited 10 programs to interview and 7 took part, one informant from each. Section 3 explains how the 10 were chosen, and Section 4 how the notes were analyzed.

2.5 Ethics, Consent and Data Governance

Surveys and interviews collect information from people, so a scan must settle early whether it needs research ethics review. In Canada, the Tri-Council Policy Statement (TCPS 2, 2022) sets out which activities count as research. Its Article 2.5 states that quality assurance and quality improvement studies, program evaluation activities and similar work, when used exclusively for assessment, management or improvement purposes, do not constitute research under the policy and do not fall within the scope of research ethics board review. Its Article 2.2 exempts research that relies exclusively on information that is publicly available, under stated conditions. A scan done by a health authority to plan its own services often falls under Article 2.5, but the judgement belongs to the organization's research ethics or quality improvement office, and plans to publish the scan as research can change the answer. HSCI 207 Lesson 5 covers research ethics review in Canada.

The Cedar Valley team asked the health authority's research ethics office, which judged the scan to be a planning activity outside the scope of research ethics board review. The team still followed ethical practice. Each invitation explained the purpose of the scan and how information would be reported, and taking part was voluntary. Programs were named in the report only with their permission, quotations were attributed by role, recordings were deleted once the summaries had been checked, and sensitive information about funding and staffing was reported in aggregate. As Lesson 6 described, no scan data were entered into any artificial intelligence tool.

Programs led by Indigenous organizations require particular care. Chapter 9 of TCPS 2 sets out expectations for research involving First Nations, Inuit and Métis Peoples, and the First Nations principles of OCAP (ownership, control, access and possession), developed by the First Nations Information Governance Centre, assert First Nations' control over data about their communities. As decided in Lesson 2, the Cedar Valley team planned its outreach to Indigenous-led programs with the health authority's Indigenous health team and agreed with each organization what information about its program would appear in the report.

Reflection

You are planning the active methods of a scan of 22 community programs that offer free or low-cost exercise classes for older adults in one British Columbia health region. Public documents already give each program's name, operator, location and activities. The planning team also wants to know how many people each program reaches, how it is funded, whether it is evaluated, and why some programs struggle to attract men. Under the Tri-Council Policy Statement (TCPS 2), Article 2.5, quality improvement and program evaluation activities used exclusively for assessment, management or improvement do not fall within the scope of research ethics board review. (a) Write four survey items, with response options and a reference period where relevant, covering reach, funding, evaluation and one other topic. (b) Write two main interview questions, each with one or two probes. (c) Describe your contact plan for the survey. (d) Say how you would decide whether the scan needs research ethics review, and name two ethical practices you would follow whatever the decision.

Model answer

(a) Reach: "In the past 12 months, about how many adults aged 65 and older took part?" (fewer than 25; 25 to 49; 50 to 99; 100 or more; don't know). Funding: "Which sources fund the classes this year? Select all that apply, then mark the largest" (provincial government; health authority; municipality; foundation; fees or donations; other; don't know). Evaluation: "In the past three years, which of these has the program done?" (external evaluation; measured participant outcomes itself; counted participants only; none; don't know). Other: "About what proportion of participants in the past 12 months were men?" (under one quarter; one quarter to one half; over one half; don't know).

(b) "How do older adults usually find out about the classes?" Probes: "Which groups do you think you are not reaching?" "What have you tried?" Second: "Why do you think fewer men take part?" Probe: "Has anything worked to change that?"

(c) An advance email from the health authority explaining the purpose, an invitation with the link, two reminders a week apart and a final phone call, with the survey open for three weeks.

(d) I would ask the health authority's research ethics or quality improvement office whether Article 2.5 applies, noting that plans to publish could change the answer. Whatever it decides, I would make participation voluntary with a clear information letter, and name programs or quote informants only with permission.

Minimum 20 characters required.

✓ Reflection saved
Knowledge Check: this section

Question 1: In the Cedar Valley program inventory, how does a listing differ from a program?

The same program can appear in several sources, so the team distinguished listings (mentions) from programs (services) and removed duplicate listings before screening programs against the criteria, much as a review merges duplicate records before screening (Lesson 7).

Question 2: A draft survey item reads: "How many participants does your program serve and how many volunteers does it have?" What is the main problem with this item?

The item asks about participants and volunteers at once, so a single answer cannot be interpreted. It should be split into two items, each with ranges, a "don't know" option and a reference period such as the past 12 months. Nothing in the wording signals a preferred answer, so it is not leading.

Question 3: Of the 18 eligible Cedar Valley programs, 11 answered the survey. Five of the 7 non-responding programs were run entirely by volunteers, compared with 3 of the 11 that responded. What should the team conclude?

Volunteer-run programs were over-represented among non-respondents, so the survey results probably describe paid-staff programs better, which is a likely case of non-response bias that the report should state. The results can still be reported with their denominator, and non-responding programs remain in the scan through their documents.

Question 4: What does Article 2.5 of the Tri-Council Policy Statement (TCPS 2) state?

Article 2.5 states that quality assurance and quality improvement studies and program evaluation activities used exclusively for assessment, management or improvement purposes do not fall within the scope of research ethics board review. Article 2.2 exempts certain research that relies only on publicly available information, and consent practices still apply to interviews whatever the review decision.
Section 3 of 5

Sampling Organizations and Jurisdictions, and Building a Charting Template

⏱ Estimated reading time: 40 minutes
Section 3 of 5

Sampling and Charting

Units, eligibility criteria, sampling frames, census or sample, jurisdictions and charting templates.

Unit and criteria

What the scan counts

Population

Mainly adults aged 65 and older living in the community.

Concept

Recurring programs that address loneliness, isolation or social connection.

Context

Community-based and operating in British Columbia in the past 12 months.

Sampling frame

Building the frame from six sources

64listings from six sources
43candidate programs after 21 duplicates
18eligible programs

Thirteen eligible programs appeared in two or more sources; five appeared in only one.

Census or sample

Matching the strategy to the method

Census

Documents and the survey covered all 18 programs.

Maximum variation

Ten programs were invited to interview and seven took part.

Twelve of 18 programs provided information directly; six were charted from documents only.

Jurisdictions

Programs elsewhere in Canada

  • A national census of all 13 provinces and territories is often feasible.
  • Cedar Valley reviewed documents for 9 programs in 4 other provinces.
  • Those 9 programs are reported separately from the 18 in British Columbia.
Charting template

Recording the same fields for every program

  • Each field has a definition and allowed values in a data dictionary.
  • Each entry carries its source and the date it was last verified.
  • A pilot on three programs reached 45 of 54 entries in agreement (83 percent).
Carry forward

From charting to integration

  • The frame and the template determine what the scan can report.
  • Shared categories with the review make integration possible.
  • Section 4 covers joint displays and reporting.

Learning Objectives for this section

  • Define the unit of a scan (a program, an organization or a jurisdiction) and write eligibility criteria for it.
  • Build a sampling frame from several sources and judge its completeness from the overlap among those sources.
  • Choose between a census and a purposive sample for each method, and select key informants for maximum variation.
  • Plan the sampling of jurisdictions for a comparison across provinces or countries.
  • Design and pilot a charting template with a data dictionary, a source for each entry and a date of verification.

Introduction

A review defines its population of studies with eligibility criteria and finds them with a documented search. A scan faces the same two tasks for programs, organizations or jurisdictions, with one added difficulty: no database indexes community programs, so the scan must assemble its own list before it can decide which entries belong.

3.1 Choosing the Unit of the Scan

The unit is the thing the scan counts and describes. Three units are common. A program is an organized service with its own activities and participants, such as a weekly walking group. An organization is the body that runs one or more programs, such as a seniors' society or a municipality. A jurisdiction is a government territory, such as a province, that sets policy and funding. The choice follows from the question. A question about what services exist for older adults calls for programs as units; a question about which organizations could partner with a health authority calls for organizations; a question about how provinces fund social prescribing calls for jurisdictions.

The Cedar Valley scan used the program as its unit, because the planning team wanted to know what kinds of programs existed and how each was funded and evaluated. The template also recorded the operating organization and any partners, because one organization may run several programs and one program may be delivered by several partners. The 18 eligible programs were run by 16 organizations: two organizations each ran two eligible programs. The Cedar Valley report states the unit at the top of every table, because counts of organizations and of programs differ.

3.2 Eligibility Criteria for Programs

Eligibility criteria for a scan work like those for a review (Lesson 2): each criterion should be clear enough that two people applying it to the same listing reach the same decision. The Cedar Valley criteria parallel the review's Population, Concept and Context, so that the scan and the review describe the same kinds of programs.

ElementIncludeExcludeReason
PopulationPrograms whose participants are mainly adults aged 65 and older living in the community, or that run a distinct stream for themPrograms for all ages with no older-adult streamMatches the review's population
ConceptOrganized, recurring programs with an aim or component that addresses loneliness, social isolation or social connectionPractical services with no social component, such as housekeeping or transportation alone; one-time eventsMatches the review's intervention categories
ContextCommunity-based programs operating in British Columbia at any time in the 12 months before the scanPrograms in long-term care homes or hospitals; programs that had closed more than 12 months earlierThe planning team needs current community programs

The 12-month rule for "currently operating" allows for programs that pause over the summer or between grants.

3.3 Building a Sampling Frame

A sampling frame is the list from which a study identifies or selects its units. An ideal frame contains every eligible unit exactly once and nothing else. A survey of physicians can often use a licensing register as its frame, but no single list of community programs for older adults exists in British Columbia, so the Cedar Valley team built its frame from six kinds of source. The table shows how many listings each source contributed.

SourceListingsNotes
Provincial 211 information and referral directory19Searched with the categories for seniors and for social and recreational programs
List of local Better at Home programs14Several offered only practical help and were later excluded under reason 3
Regional health authority community resource lists11Lists varied in format and date of last update
Municipal and seniors' centre websites9Searched for the larger municipalities in each region
Grey-literature documents from the review (Lesson 5)7Program descriptions and reports that named a British Columbia program
Referrals from survey respondents and interviewees4Added until the frame closed at the end of week 7
Total64Listings, before duplicates were removed

Removing 21 duplicate listings left 43 unique candidate programs, and screening against the criteria left 18 eligible programs (Section 4 reports the reasons for exclusion). The frame closed at the end of week 7 so that the team had time to chart and verify every program.

Judging the completeness of the frame

A team can never be certain that a frame is complete, but the overlap among sources gives a useful signal. If most eligible programs appear in several independent sources, the sources are probably covering the same population well. If many appear in only one source, other programs probably appear in none. Figure 11.4 shows the overlap for the 18 eligible Cedar Valley programs. Thirteen appeared in two or more sources and 5 in only one, and 2 of those 5 were found only through referrals. Both were small volunteer-run programs, which suggests that the frame under-represents programs of that kind. The report states this limitation. Formal capture-recapture methods, which estimate a population's size from overlapping lists, assume independent lists, which directories that copy from one another rarely are, so the team used the overlap descriptively.

Eligible programs by number of sources in which they appeared (n = 18) 5One source 7Two sources 4Three sources 2Four or more Found only by referral (2) Found in one other source (3) Thirteen of the 18 programs appeared in two or more sources.
Figure 11.4. Overlap among the sources of the Cedar Valley sampling frame. Programs that appear in only one source, especially those found only by referral, signal that similar programs may be missing.

3.4 A Census or a Sample?

When the number of eligible units is small, a scan can include all of them, which is called a census. When it is large, or when a method is too costly to apply to every unit, the scan selects a sample. The choice can differ by method within the same scan. The Cedar Valley team charted all 18 programs from documents and sent the survey to all 18, because both steps were cheap. Interviews were expensive in time, so it selected programs for interview with purposive (judgement) sampling, which chooses units deliberately for what they can contribute. The cards describe four strategies that scans use.

Every eligible unit is included. A census suits small populations and cheap methods, although it is only as complete as the frame. Cedar Valley used a census for documents and the survey.

Units are chosen to differ as much as possible on characteristics that matter to the question, so that findings cover the range of situations. Patton (2015) describes it as one of several purposive sampling strategies. Cedar Valley used it for interviews.

Participants are asked to suggest other units, which helps to find programs that directories miss. It tends to find units connected to those already found, so it adds to a frame without guaranteeing completeness.

Units are included because they are easy to reach, such as programs whose staff the team already knows. It tends to over-represent larger, better-connected organizations and should be reported as a limitation when used.

For the interviews, the team listed the characteristics on which programs varied: main activity, location in the Cedar Valley region or elsewhere in British Columbia, urban or rural setting, type of operating organization, whether the program was Indigenous-led and whether it evaluated outcomes. It also decided to include programs that had not answered the survey, so that the interviews could hear from them. It invited 10 programs, 8 that had responded to the survey and 2 that had not, and 7 took part: 6 survey respondents and 1 non-respondent. Counting both methods, 12 of the 18 programs (11 by survey and 1 by interview only) provided information directly, and the other 6 were charted from documents alone.

ProgramMain activityLocationSettingOperatorSurvey
P02Group activityCedar Valley regionUrbanNon-profit seniors' centreResponded
P05One-to-one befriending or telephoneElsewhere in British ColumbiaRuralVolunteer seniors' societyResponded
P08Community connectorElsewhere in British ColumbiaUrbanNon-profit in partnership with a health authorityResponded
P11Community connectorCedar Valley regionRuralNon-profit community services societyResponded
P13IntergenerationalElsewhere in British ColumbiaUrbanMunicipalityResponded
P15Group activityElsewhere in British ColumbiaRuralIndigenous-led organizationDid not respond
P17Technology-basedElsewhere in British ColumbiaUrbanNon-profitResponded

The seven interviewed programs cover five of the six activity categories, both locations, both settings and four kinds of operator. No interviewed program was a one-to-one program in the Cedar Valley region, which the report notes as a gap in the interview sample.

3.5 Sampling Jurisdictions

A jurisdictional scan compares how provinces, territories or countries handle an issue, for example how each funds or governs a type of program. Canada has 10 provinces and 3 territories, so a national jurisdictional scan can often take a census of all 13; CADTH's process for environmental scans, for example, includes surveys of health care decision-makers across Canada (CADTH, 2015). When time is short, a scan selects jurisdictions purposively, for example those with a similar health system or a well-known model, and the report states which were included and why.

The Cedar Valley scan question also asked about programs elsewhere in Canada. With twelve weeks, the team chose a lighter method for this part. It selected four other provinces, chosen to vary by region and population size and to include provinces with publicly documented programs, and reviewed public documents and websites only, with no survey or interviews. It charted 9 programs from those provinces with a shortened version of the template. The report keeps these 9 separate from the 18 British Columbia programs, because they were found and described with different methods and cannot be counted together.

3.6 Building the Charting Template

A charting template is the form on which the scan records the same set of fields for every unit. As with the review's charting form (Lesson 8), each field answers part of the question, each has a definition and allowed values in a data dictionary, and the form is piloted before full use. Two features are particular to scans. Because information comes from several sources that may disagree, the template records the source of each entry. And because programs change, it records the date on which each program's information was last verified.

The Cedar Valley template appears below in abridged form. Its activity categories are those of the review's charting form (Lesson 8), which made the integration in Section 4 possible.

FieldDefinition and allowed valuesPreferred source
Program ID and nameP01 to P18; name as the operator uses itInventory
Operator and partnersOperator type: non-profit; municipality; health authority; Indigenous-led organization; other. Partners namedWebsite, survey
Location and settingCedar Valley region or elsewhere in British Columbia; urban, or rural and small communityWebsite
Main activityGroup activity; one-to-one befriending or telephone; community connector or social prescribing; intergenerational; technology-based; other. Main activity is the one with the most contact hours; record a secondary activity if presentSurvey, website
Delivery mode and frequencyIn person, telephone, online or mixed; weekly, every two weeks, monthly or otherSurvey, website
Eligibility and referral routesAge and other criteria; self-referral, family, primary care clinician, connector or agency, otherSurvey, interview
StaffingPaid staff only; volunteers only; bothSurvey
ReachOlder adults taking part in the past 12 months, in the survey's rangesSurvey, annual report
FundingAll sources this year and the largest; end date of the largest sourceSurvey, annual report
Evaluation statusExternal evaluation; internal outcome measurement; activity or satisfaction monitoring only; none reportedSurvey, evaluation report
Outcome measures usedNames of questionnaires or measures as reportedSurvey, evaluation report
Links with primary careFormal referral pathway; informal contact; none reportedInterview, survey
Barriers and facilitatorsBrief summary in the informant's termsInterview
Sources and verificationSaved copy codes, survey and interview dates; date the entry was last verified; conflicts between sources and how each was resolvedAll

Three coding rules prevent common errors. First, the template distinguishes "not reported" (no source mentions it), "don't know" (the respondent did not know) and "not applicable" (the field does not apply), because the three mean different things when the team later counts programs. Second, when sources disagree, the team records the most recent direct report, usually the survey or the interview, and notes the conflict. Third, every entry carries its source.

Before charting all 18 programs, the intern and the evidence officer charted three programs independently and compared their entries. They agreed on 45 of 54 entries (3 programs with 18 entries each), or 83 percent. Of the 9 disagreements, 4 concerned the main activity of a seniors' centre that ran both drop-in groups and outreach visits, 3 concerned whether participant satisfaction surveys counted as outcome measurement, and 2 concerned which end date to record when a program had several funding sources. They added three rules to the data dictionary: the main activity is the one with the most contact hours, which is the rule the review used in Lesson 8; satisfaction surveys count as monitoring and not as outcome measurement; and the funding end date is that of the largest source.

Case: The charted entry for program P08 (fictional)

Operator and partners: a non-profit, with a partnership agreement with a regional health authority. Location and setting: elsewhere in British Columbia; urban. Main activity: community connector or social prescribing; secondary activity, group activity. Delivery: in person and by telephone, about every two weeks. Eligibility and referral: adults aged 65 and older living in the community; referrals mainly from primary care clinicians, with some self-referral. Staffing: paid staff and volunteers. Reach: 100 to 249 older adults in the past 12 months (survey). Funding: health authority (largest) and a charitable foundation; largest source ends in one to three years (survey). Evaluation status: external evaluation; report shared with the team (document D-03). Outcome measures: a three-item loneliness scale and a well-being scale, as named in the evaluation report. Links with primary care: formal referral pathway (interview). Barriers and facilitators: connectors attend clinic team meetings, which helped referrals grow; winter transport limits group activities (interview). Sources: saved web page W-088 (week 4), survey (week 6), interview (week 7), document D-03. Conflict: the website described weekly meetings, and the survey reported meetings every two weeks since a staffing change; the survey value was recorded. Last verified: week 9, when the program confirmed its interview summary.

Try it: Chart program P12

Program P12's website, last updated about two years before the scan, says: "Our Tuesday Tea Club welcomes all seniors aged 60 and over for conversation, games and a light lunch. Generously funded by the City." The program's survey response reports that its main activity is a group activity; that 25 to 49 older adults took part in the past 12 months, most of them over 70; that it is run by volunteers only; that its funding comes from the municipality (largest) and donations, with the municipal grant ending within 12 months; that it counted participants only; and, for outcome measures, "don't know". Chart the main activity, eligibility, staffing, funding, evaluation status and outcome measures, and note any conflict between sources.

Check your answerv

Main activity: group activity (survey and website agree). Eligibility: adults aged 60 and older, with participants mainly over 70; the program meets the criterion of serving mainly adults aged 65 and older, and the age difference should be noted in case the planning team asks. Staffing: volunteers only (survey). Funding: municipality (largest) and donations; largest source ends within 12 months (survey). The survey adds donations to the website's account and is recorded as the more recent direct report, with the difference noted. Evaluation status: activity or satisfaction monitoring only (survey). Outcome measures: "don't know" (survey), which differs from "not reported" and from "not applicable" and should be coded as such. The entry's sources are the saved web page and the survey, with the survey date as the date of verification.

Reflection

A scan of youth mental health walk-in services in British Columbia uses the service as its unit. Its sources produced these listings: a provincial directory, 30; health authority resource lists, 18; school district websites, 12; and referrals from service managers, 5. The team removed 23 duplicate listings and screened the remaining candidates against its criteria, excluding 16. Of the eligible services, 15 appeared in two or more sources and 11 in only one, including 6 found only by referral. The budget allows 8 interviews; the survey is cheap to send. (a) Calculate the number of listings, candidate services and eligible services, and check that the numbers reconcile. (b) Interpret the overlap among sources for the completeness of the frame. (c) Decide whether to use a census or a sample for the survey and for the interviews, and name three characteristics you would vary when selecting services for interview. (d) List five fields for the charting template, each with allowed values.

Model answer

(a) Listings: 30 + 18 + 12 + 5 = 65. Candidates: 65 − 23 = 42. Eligible: 42 − 16 = 26. The overlap figures reconcile, because 15 + 11 = 26.

(b) Eleven of 26 services (42 percent) appeared in only one source, and 6 were found only by referral, which suggests that the directories miss many services and that other services were probably found by no source at all. I would add sources, for example community health centre and youth agency websites, keep asking for referrals until the frame closes, and report the likely under-coverage as a limitation.

(c) A census for the survey, since there are only 26 services and the survey is cheap. For interviews, a purposive maximum variation sample of 8, varying by health authority region, urban or rural setting and operator type (health authority, non-profit or school district), and including one or two services that did not answer the survey.

(d) Operator type (health authority; non-profit; school district; Indigenous-led organization; other). Setting (urban; rural or small community). Hours (weekdays only; includes evenings; includes weekends). Ages served (under 19; 12 to 24; other). Evaluation status (external evaluation; internal outcome measurement; activity monitoring only; none reported). Each entry would also record its source and the date it was verified.

Minimum 20 characters required.

✓ Reflection saved
Knowledge Check: this section

Question 1: In the Cedar Valley sampling frame, 13 of the 18 eligible programs appeared in two or more sources and 5 in only one, including 2 found only through referrals. What does this pattern most reasonably suggest?

Programs that appear in only one source, especially by referral, signal that similar programs may appear in no source, so the frame probably under-represents small volunteer-run programs. Overlap cannot prove completeness, and directories that copy from one another are not independent, so formal capture-recapture assumptions do not hold.

Question 2: Why did the Cedar Valley team use a census for the survey but a purposive maximum variation sample for the interviews?

Sending the survey to all 18 programs was cheap, while interviews were costly in time, so the team chose 10 programs that varied by activity, location, setting, operator, Indigenous leadership and evaluation status. Purposive samples aim to cover a range of situations; they are not statistically representative and do not include every program.

Question 3: Which distinction does the Cedar Valley charting template make when information is missing?

"Not reported" means no source mentions the item, "don't know" means the respondent did not know, and "not applicable" means the field does not apply. Keeping them apart prevents errors when programs are counted. When sources disagree, the team records the most recent direct report, usually the survey, and notes the conflict.

Question 4: The Cedar Valley team reviewed public documents for 9 programs in four other provinces, without a survey or interviews. How should the scan report treat these programs?

The 9 programs were selected purposively and described from documents alone, so combining them with the 18 British Columbia programs found through a frame and surveyed would produce counts with no clear meaning. They answer a separate sub-question about models elsewhere in Canada and are reported on their own.
Section 4 of 5

Combining a Scan with a Literature Review, and Reporting a Scan

⏱ Estimated reading time: 35 minutes
Section 4 of 5

Combining and Reporting a Scan

Integration with the review, joint displays, analysis, reporting, limitations and currency.

Integration

Three levels of integration

Design

The scan ran in parallel with the review, weeks 3 to 10.

Methods

Shared categories, and documents passed in both directions.

Interpretation

A joint display sets the findings side by side.

Joint display

What the review and the scan show together

Group activity

8 trials, low certainty; 6 local programs, 2 measuring outcomes.

Community connector

5 non-randomized studies, very low certainty; 4 local programs, 2 measuring outcomes.

All results belong to the fictional case and are illustrative.

Analysis

Denominators, sources and themes

  • Seven of 18 programs measured at least one participant outcome.
  • Of 12 programs with direct information, 8 had funding ending within three years.
  • A framework matrix organized the seven interview summaries by topic.
Reporting

Borrowing from existing guidelines

PRISMA-S

Searches of websites and directories.

PRISMA-ScR

Charting and summarizing.

CHERRIES

Online surveys.

COREQ

Interviews.

Flow of programs: 64 listings, 43 candidates, 18 eligible, 11 surveyed, 7 interviewed.

Limitations

Stating limits and currency

  • Information is current to the end of week 10 of the project.
  • Small volunteer-run programs are probably under-represented.
  • The inventory and template are kept so that the scan can be updated.
Next steps

Reflection and assessment

  • The reflection below asks you to build a joint display.
  • The final assessment and final reflection follow on the last page.
  • A scan ends with a joint display that sets it beside the review.

Learning Objectives for this section

  • Explain why a scan and a literature review answer different parts of a planning question, and how each can inform the other.
  • Describe the levels at which a scan and a review can be integrated: design, methods, and interpretation and reporting.
  • Build a joint display that sets review findings beside scan findings using shared categories.
  • Analyze scan data with counts that state their denominators and sources, and summarize interview notes with a framework approach.
  • Report a scan with a flow of programs from identification to participation, a description of methods, a statement of limitations and a date of currency.

Introduction

The Cedar Valley planning team asked two questions: which community-based interventions have been evaluated for reducing loneliness or social isolation among older adults, and which programs of this kind operate in British Columbia now. The rapid scoping review answers the first and the environmental scan answers the second. A decision about what kind of community connector program to launch, and with which partners, needs both answers read together. This section shows how the team combined the two products, how it analyzed the scan data, and how it reported the scan so that readers could judge it and update it.

4.1 Why Combine a Scan with a Review

A review and a scan look at the same field from two sides. The review describes what research has found about programs, wherever they ran; most of the 42 studies in the Cedar Valley review were conducted outside Canada. The scan describes what is available locally, whether or not anyone has studied it; most of the 18 British Columbia programs have never been evaluated in a published study. Read together, the two products show where local practice and research evidence coincide, where local programs operate with little evidence behind them, and where the evidence describes models that no local program offers.

Combining the two is common practice: in the review by Charlton and colleagues (2021), a literature review was the most frequent method used within scans. In the Cedar Valley project the literature component is a full rapid scoping review, so the integration needs to be planned explicitly.

4.2 Levels of Integration

Combining a scan and a review is a form of mixed-methods work, and the mixed-methods literature offers a useful vocabulary. Fetters, Curry and Creswell (2013) describe integration at three levels. At the design level, the components may run in sequence, with one informing the next, or in parallel, with their results brought together at the end. At the methods level, one component can build on another, for example by using its categories, or connect to it, for example by drawing its sample from the other's results. At the level of interpretation and reporting, the findings can be woven together in narrative or set side by side in a joint display. HSCI 207 Lesson 6, Section 2.4, and HSCI 207 Lesson 12, Section 3.6, treat integration levels and joint displays more fully; the definitions here are enough to follow this lesson without that course. The tabs show where each level appeared in the Cedar Valley project.

The scan ran in parallel with the review (weeks 3 to 10), and the two were brought together in the synthesis that began in week 9 (Lesson 1), which suited the twelve-week deadline.

The scan's main-activity field and survey item 3 used the intervention categories of the review's charting form (Lesson 8). The 12 program descriptions that the review excluded because they reported no evaluation went to the scan as documents, and 7 grey-literature documents supplied listings for the sampling frame.

Programs found by the scan were checked for evaluation reports. The 3 external evaluation reports that programs shared during the scan were passed to the review team and are among the 7 documents received from organizations in the review's flow diagram (Lesson 7).

The team built a joint display that sets the review's findings for each intervention category beside the scan's findings for the same category (Section 4.3). The evidence brief in Lesson 12 draws on both.

4.3 A Joint Display for Cedar Valley

A joint display is a table or figure that places the results of two components side by side so that readers can see how they relate and what new interpretation their combination supports. Guetterman, Fetters and Creswell (2015) described joint displays as a way of integrating quantitative and qualitative results in health sciences research, and the same idea works for a review and a scan. Shared categories make a joint display possible. Because the Cedar Valley scan charted programs with the review's intervention categories, each row of the display compares like with like.

Main activityReview: studies in the certainty ratings (Lesson 8)Review: certainty for lonelinessScan: British Columbia programs (n = 18)Scan: programs measuring outcomesIntegrated interpretation
Group activity8 randomized trialsLow; may reduce loneliness slightly62The most common local model has the most trial evidence, and few local programs measure outcomes.
One-to-one befriending or telephone6 randomized trialsLow; may reduce loneliness42Evidence and local supply both exist, and telephone programs reach rural participants.
Community connector or social prescribing5 non-randomized controlled studiesVery low; very uncertain42The model Cedar Valley plans has the least certain evidence; the two local programs that measure outcomes could share their tools.
Intergenerational and technology-based5 non-randomized controlled studies across the two categoriesNot rated; too few studies in each category2 (1 in each)1Little evidence and few local programs.
OtherNot part of the ratingsNot rated20Multi-service seniors' centres that do not fit one category.

The display supports three interpretations that neither product supports alone. First, the local supply of programs is concentrated in group activities, where trial evidence is most plentiful, although its certainty is low. Second, the community connector model that Cedar Valley plans to launch rests on very uncertain evidence and has only four counterparts in the province, two of which measure outcomes; this strengthens the case, made in Lesson 8, for launching the program with an evaluation built in, and suggests that the team could adopt a short outcome measure already used by those programs. Third, a connector program depends on having programs to refer people to, and the scan shows which kinds of programs exist and where. All results in this section belong to the fictional case and are illustrative.

4.4 Analyzing Scan Data

Counts with denominators and sources

Most scan results are counts and proportions, and the most frequent reporting error is a proportion whose denominator is unclear. Because the survey and interviews reached only some programs, each count must say which programs it describes and where the information came from. "Seven of the 18 programs measured at least one participant outcome (3 through an external evaluation), according to surveys, evaluation reports and program documents" is a complete statement. "Of the 12 programs that provided information directly, 8 reported that their largest source of funding ends within three years, and 3 reported a formal referral pathway from primary care" is another. The table shows the largest funding source for all 18 programs, drawn from the survey where available and from annual reports and websites otherwise.

Largest funding sourcePrograms (n = 18)
Provincial government5
Health authority4
Charitable foundation or United Way4
Municipality2
Donations or fees1
Not reported2

Summarizing interview notes

The interview summaries were analyzed with framework analysis, also called the framework method (Gale et al., 2013), which suits structured questions and short timelines. The team built a matrix with one row for each of the 7 programs and one column for each topic of the guide, wrote a short summary in each cell with the informant's key phrases, and then read down the columns to find patterns and differences. HSCI 207 Lesson 11 teaches framework analysis in more depth. In the illustrative Cedar Valley matrix, most informants said that participants arrived by word of mouth and that links with clinics depended on individual clinicians. Informants with short grants described a cycle of hiring, training and losing staff, several said they would measure outcomes if given a short shared tool, rural informants named transport as the main barrier, and most advised a new connector program to refer people into existing programs and bring resources such as volunteer training.

The team then compared what different sources said about the same topic, a step called triangulation. The survey showed that 3 of 12 programs had a formal referral pathway from primary care; the interviews explained why the number was low and what had made the pathway work in program P08. Where sources disagreed, the report says so.

4.5 Reporting a Scan

Charlton and colleagues (2024) noted that the EQUATOR Network, which catalogues reporting guidelines for health research, listed no reporting guideline for environmental scans, and that many published scans describe their methods incompletely. Until one exists, a scan can borrow from the guidelines for each of its components: PRISMA-S for the searches of websites and directories (Lessons 4 and 5), PRISMA-ScR for charting and summarizing (Lesson 10), the Checklist for Reporting Results of Internet E-Surveys (CHERRIES; Eysenbach, 2004) for an online survey, and the Consolidated Criteria for Reporting Qualitative Research (COREQ; Tong, Sainsbury and Craig, 2007) for interviews. The checklist below gathers the items the Cedar Valley team reported.

ItemWhat to report
Question and purposeThe scan question, sub-questions and the decision the scan serves
Unit and eligibilityThe unit (program, organization or jurisdiction) and the criteria with reasons
Sampling frameEvery source, the date searched, the search terms or path, and the date the frame closed
SelectionA flow of units from listings to eligible programs, with ordered exclusion reasons
SurveyThe instrument, mode, contact sequence, dates, pilot, response rate with numerator and denominator, and a comparison of respondents with non-respondents
InterviewsHow informants were selected, the guide, who interviewed, recording and notes, analysis and member checking
ChartingThe template and data dictionary, the pilot and its agreement, coding rules and the handling of conflicting sources
Ethics and governanceThe decision about research ethics review, consent procedures, confidentiality and agreements with Indigenous organizations
ResultsCounts with denominators and sources, and summaries of interview findings
Limitations and currencyKnown gaps in the frame and samples, and the date to which the information is current

Lesson 7 noted that the scan is reported separately from the review's PRISMA flow diagram, because its counts describe programs and people. Figure 11.5 shows the flow of programs from identification to participation. Its structure follows the PRISMA diagram, with boxes that must reconcile: 64 listings minus 21 duplicates gives 43 candidate programs, and 43 minus 25 exclusions gives 18 eligible programs.

Listings identified (n = 64) Directories and lists 44, websites 9, grey literature 7, referrals 4 Duplicate listings removed (n = 21) Candidate programs screened (n = 43) Excluded (n = 25) Not currently operating (6) Based in a facility (3) No social connection component (9) Not mainly for adults 65+ (5) One-time event (2) Eligible programs charted from documents (n = 18) Survey, weeks 5 to 7 Invited 18, completed 11 Interviews, weeks 6 to 8 Invited 10, completed 7 Programs with direct information (n = 12) Charted from documents only (n = 6) Other provinces (separate) 4 provinces, 9 programs, documents and websites only Interviews: 6 survey respondents and 1 non-respondent
Figure 11.5. Flow of programs in the Cedar Valley environmental scan, from listings identified to programs that provided information directly. Every box reconciles with the boxes above it.

Limitations and currency

Every scan describes a moment in time, so its report should state the date to which its information is current, and a reader in two years should treat it as a historical record until it is updated. The Cedar Valley report states that program information was current at the end of week 10 and lists the scan's known limitations: the frame probably under-represents small volunteer-run programs (Section 3.3); volunteer-run programs were also less likely to answer the survey (Section 2.3); survey and interview information is self-reported and may present programs favourably; the interview sample included no one-to-one program in the Cedar Valley region; and programs in other provinces were described from documents only. The team kept the inventory, saved copies and template so that the scan could be updated.

4.6 Worked Example: The Cedar Valley Scan Report

The Cedar Valley scan report runs to about twelve pages plus appendices. Open each part to see what it contains.

Summaryv

One page giving the question, the methods, the main findings with denominators, the joint display's interpretations and the date of currency.

Purpose, question and methodsv

The scan question and six sub-questions; the unit and eligibility criteria; the six sources of the sampling frame with dates; the survey, its contact sequence and its response (11 of 18 programs, 61 percent); the selection of 10 programs for interview and the 7 interviews completed; the charting template and its pilot (83 percent agreement before the rules were revised); and the ethics decision and agreements with Indigenous organizations.

Resultsv

Figure 11.5, the flow of programs; a characteristics table of the 18 programs by main activity, location, setting, operator, reach, funding and evaluation status; and a narrative of interview findings by topic. A sample sentence reads: "Of the 18 programs, 6 offered mainly group activities, 4 one-to-one befriending or telephone contact, 4 community connector services, 1 an intergenerational program, 1 a technology-based program and 2 a mix of services. Seven measured at least one participant outcome, 3 of them through an external evaluation."

Integration with the reviewv

The joint display (Section 4.3), with a paragraph on each interpretation and a cross-reference to the review report.

Limitations, currency and implicationsv

The limitations and date of currency above, and implications for the connector program, such as referring participants into existing programs and adopting a shared outcome measure.

Appendicesv

The inventory, the template and data dictionary, the survey, the interview guide, the search log and the 9 programs from other provinces.

Try it: Repair a results paragraph

A colleague’s draft scan report, on a different question, contains this paragraph: "Most programs (64%) are evaluated. Funding is a major concern for programs. Clinicians rarely refer patients to programs." Using what you know about reporting scans, identify at least three problems and rewrite the paragraph for the colleague’s scan, in which 9 of 14 survey respondents reported measuring participant outcomes, 8 of 14 reported that their largest funding source ends within three years, and 5 of 8 interviewed program leads described referrals from clinicians as rare.

Check your answerv

The paragraph gives no denominators, so readers cannot tell that 64 percent means 9 of 14 survey respondents. It does not say which programs the figures describe, so readers may assume they cover all programs in the scan, although non-respondents may differ. "Evaluated" is undefined: measuring outcomes, counting participants and commissioning an external evaluation are different things. "A major concern" and "rarely" are not tied to a source or a count. A revision: "Of the 14 programs that answered the survey, 9 reported measuring at least one participant outcome, and 8 reported that their largest funding source ends within three years. Five of the 8 program leads interviewed described referrals from clinicians as rare." The report should also compare survey respondents with non-respondents, so that readers can judge whether the figures describe the programs that did not answer.

The Cedar Valley team now has a charted review of 42 studies and 26 grey-literature documents, a scan of 18 British Columbia programs and a joint display that links them. Lesson 12 turns these into a report written to PRISMA 2020 and its extensions and an evidence brief for the planning team.

Reflection

A rapid review of school-based physical activity interventions rated certainty for daily physical activity as follows: classroom activity breaks, 12 randomized trials, low certainty (may increase activity); after-school programs, 9 randomized trials, moderate certainty (probably increase activity); and active travel programs, 4 non-randomized studies, very low certainty (very uncertain). A parallel environmental scan of 20 British Columbia school districts, charted with the same categories, found activity breaks in 14 districts (3 of which measure outcomes), after-school programs in 4 districts (2 of which measure outcomes) and active travel programs in 9 districts (1 of which measures outcomes). A district can offer more than one kind of program. A joint display is a table that sets the results of two components side by side so that readers can draw interpretations that neither supports alone. (a) Describe the rows and columns of a joint display for these results. (b) Write three integrated interpretations. (c) Write one results sentence for the scan report that states its denominator and source.

Model answer

(a) One row for each category (activity breaks, after-school programs, active travel). Columns: review studies and designs; review certainty and statement; districts offering the program (of 20); districts measuring outcomes; and an integrated interpretation.

(b) First, after-school programs have the most certain evidence (moderate, 9 trials) yet operate in only 4 of 20 districts, so they are the clearest candidate for expansion if cost and feasibility allow. Second, activity breaks are the most widespread local program (14 districts) and have low-certainty evidence, and only 3 of the 14 districts measure outcomes, so districts could adopt a shared measure to learn whether local breaks increase activity. Third, active travel programs operate in 9 districts on very uncertain evidence, and only 1 measures outcomes, which makes them the strongest case for building evaluation into any expansion.

(c) "Of the 20 school districts in the scan, 14 offered classroom activity breaks, and 3 of those 14 reported measuring students' physical activity, according to the district survey and program documents."

An alternative strong answer might add a column for gaps, noting any category with evidence but no local programs, which in these data does not occur.

Minimum 20 characters required.

✓ Reflection saved
Knowledge Check: this section

Question 1: What is a joint display?

A joint display places the results of two components, here a review and a scan, side by side so that readers can draw interpretations that neither supports alone (Guetterman, Fetters and Creswell, 2015). A summary of findings table and a flow diagram each report a single component.

Question 2: What made the Cedar Valley joint display possible?

Because the scan's main-activity field and survey item 3 used the categories from the review's charting form (Lesson 8), each row of the display compares like with like. The review did not pool studies, and the certainty ratings came from the review team's GRADE assessment.

Question 3: Which sentence reports a scan result in the way Section 4 recommends?

The third sentence states its denominator, defines what was counted and names its sources. The others give no denominator, use undefined terms such as "evaluated" or "rarely", or make claims that are not tied to a count or source.

Question 4: Which statement about reporting guidance for environmental scans is accurate?

Charlton and colleagues (2024) noted that the EQUATOR Network listed no reporting guideline for environmental scans. A scan can borrow from PRISMA-S for searches, PRISMA-ScR for charting, CHERRIES for online surveys and COREQ for interviews. Lesson 7 explains why the scan's counts are reported separately from the review's PRISMA diagram.
Section 5 of 5

Final Assessment

⏱ Estimated time: 25 minutes

Bringing It All Together

This lesson followed the fictional Cedar Valley evidence team through the environmental scan that it ran beside its rapid scoping review. An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, to inform planning. The method began in business planning, where Aguilar described four modes of scanning in 1967, and health services and public health organizations now use it to plan programs, compare jurisdictions and find gaps and partners. Published definitions still vary, and Charlton and colleagues found that most scans combine passive methods, which gather existing information, with active methods, which generate new information from organizations and people.

The Cedar Valley team combined four methods. It searched directories, websites and documents and saved a copy of each source; it built a program inventory that turned 64 listings into 43 candidate programs and 18 eligible ones, each with a recorded decision; it surveyed all 18 programs, of which 11 responded; and it interviewed leads from 7 programs chosen for maximum variation. It judged the completeness of its sampling frame from the overlap among sources, kept 9 programs from other provinces separate because they were found with different methods, and charted every program with a piloted template that recorded the source and date of each entry.

Finally, the team integrated the scan with the review through shared intervention categories and a joint display, which showed that the connector model it plans rests on very uncertain evidence and has few local counterparts. It reported the scan with a flow of programs, counts that state their denominators and sources, a statement of limitations and a date of currency.

Key Takeaways from this lesson

  • An environmental scan describes what currently exists in a defined setting, such as programs, services, policies and organizations, so that an organization can plan.
  • Published definitions of scans vary, and recurring themes describe the scan as an instrument of discovery, a form of knowledge synthesis, forward-looking and a support for decisions.
  • Scanning began in business planning, and Aguilar's modes range from undirected viewing to formal search; a planned environmental scan is a formal search.
  • Most scans combine passive methods, such as documents, websites and directories, with active methods, such as surveys of organizations and key informant interviews.
  • A program inventory lists every candidate program once, separates listings from programs, and records a decision with one ordered reason for every exclusion.
  • Because no database indexes community programs, a scan builds its sampling frame from several sources and uses the overlap among them as a signal of completeness.
  • A census suits cheap methods and small populations, while costly methods such as interviews use purposive samples chosen for maximum variation.
  • A charting template with a data dictionary, a source for every entry and a date of verification makes scan data comparable, checkable and updatable.
  • Shared categories let a scan and a review be integrated in a joint display that shows where local practice and research evidence coincide or diverge.
  • A scan report states every denominator and source, shows a flow of programs from identification to participation, names its limitations and gives a date of currency.

Core Concepts Reviewed

Section 1: definitions of the environmental scan, Aguilar's modes of scanning, uses in health services and public health, passive and active scanning, codified and tacit knowledge, and how scans differ from reviews, needs assessments, asset maps and evaluations.

Section 2: document and website review with saved copies, the program inventory with listings and ordered exclusion reasons, organizational surveys with contact plans and response rates, key informant interview guides, and ethics, consent and Indigenous data governance.

Section 3: the unit of a scan, eligibility criteria for programs, sampling frames built from several sources, census and purposive sampling, jurisdictional scans, and charting templates with data dictionaries and piloting.

Section 4: integration of a scan with a review at the design, methods and interpretation levels, joint displays, counts with denominators, framework analysis of interview notes, triangulation, and reporting a scan with a flow of programs and a date of currency.

The final reflection asks you to design a complete environmental scan for a new planning question, drawing on all four sections of the lesson.

Reflection

A health authority plans a community connector program for adults living with chronic pain and asks for a rapid scoping review and an environmental scan within ten weeks. The review asks which community-based self-management and social programs have been evaluated for adults with chronic pain, and with what outcomes. Design the scan. Address: (a) the scan question and the unit of the scan; (b) two eligibility criteria that parallel the review's criteria; (c) at least three sources for the sampling frame and how you would judge its completeness; (d) which methods you would use, and whether each uses a census or a sample; (e) five charting fields, including how you would record the source and date of each entry; (f) how you would integrate the scan with the review; and (g) three items your report must include, with one likely limitation.

Model answer

(a) Which organizations currently deliver community-based self-management, peer-support or social programs for adults living with chronic pain in the province, and how are those programs designed, funded and evaluated? The unit is the program, with the operating organization recorded.

(b) Population: programs whose participants are mainly adults living with chronic pain. Concept: organized, recurring programs with a self-management, peer-support or social component; clinical treatment alone is excluded, matching the review.

(c) The provincial community services directory, chronic pain society and peer-support network lists, and health authority resource lists, with referrals from survey respondents. I would count how many eligible programs appear in two or more sources; many single-source programs would signal missing ones.

(d) Documents and websites for every eligible program (census); a short survey of every program (census); and six to eight key informant interviews chosen for maximum variation by program type, region and operator.

(e) Main activity, using the review's categories; referral routes; funding and its end date; evaluation status; and outcome measures. Each entry carries its source code and the date it was last verified, with conflicts noted.

(f) Shared categories allow a joint display setting the review's evidence beside the number of local programs and their evaluation status.

(g) A flow of programs from listings to participants, a response rate with numerator and denominator, and a date of currency. A likely limitation is that small volunteer-run peer groups are under-listed and less likely to respond.

Minimum 30 characters required.

✓ Reflection saved

Final Knowledge Assessment

Final Assessment, this lesson: Environmental Scans (15 Questions)

Question 1: Which feature best distinguishes an environmental scan from a literature review?

A review asks what research has found, and its sources are research reports. A scan asks what currently exists in a defined setting, and its sources include websites, documents, organizations and people. Scans describe programs and do not estimate effects.

Question 2: Which description fits the environmental scan reported by Rowel and colleagues (2005)?

Rowel and colleagues described a 2003 scan for a project to increase cancer screening among African Americans in Baltimore, which combined a review of community organizations, cancer incidence data and community assets with focus groups and key informant interviews. The other options describe CADTH scans, Charlton and colleagues (2021) and Wilburn and colleagues (2016).

Question 3: According to Graham, Evitts and Thomas-MacLean (2008), tacit knowledge lives in people's experience. Which source is most likely to yield tacit knowledge about a program?

Tacit knowledge comes out in conversation, interviews and meetings. The annual report, the directory entry and the grant list are codified knowledge, written down in documents.

Question 4: A scan finds a home support service that offers only housekeeping and yard work to older adults in the community, and it is currently operating. Using the Cedar Valley ordered exclusion reasons, which reason applies?

The service is operating, community-based and for older adults, so reasons 1, 2 and 4 do not apply, and it is ongoing, which rules out reason 5. It offers practical help only, so the first reason that applies is reason 3.

Question 5: A program's website describes weekly meetings, and its more recent survey response reports meetings every two weeks. Under the Cedar Valley coding rules, what does the team record?

When sources disagree, the team records the most recent direct report, usually the survey or interview, and notes the conflict in the template. This is what happened for program P08.

Question 6: A scan invited 25 organizations to complete a survey, and 15 did so. Which sentence reports the response in the way this lesson recommends?

The response rate is 15 divided by 25, or 60 percent. The report should give the numerator and denominator and compare respondents with non-respondents on what documents show, because non-response may be related to program characteristics.

Question 7: In the terms used by Fetters, Curry and Creswell (2013), using the review's intervention categories to write a scan survey item is integration at which level?

Building one component's data collection on another's categories is integration at the methods level. Running the components in parallel is a design choice, and joint displays and narrative weaving integrate at the interpretation and reporting level.

Question 8: In the Cedar Valley joint display, the community connector model had very low certainty evidence for loneliness and four counterparts in British Columbia, two of which measured outcomes. Which implication does the display support?

Very low certainty means the true effect is very uncertain in either direction, which does not show that the model fails. It strengthens the case for evaluation built into the launch, and the two local programs that measure outcomes may offer a shared measure. Pooling different interventions would not address the reasons for the rating.

Question 9: A health authority plans a scan with surveys and interviews to design one of its own programs. Which approach to research ethics is appropriate?

Whether a planning scan falls under Article 2.5 is a judgement for the organization's ethics or quality improvement office, and plans to publish may change it. Surveys and interviews still collect information from people, so voluntary participation, clear information and confidentiality apply whatever the decision.

Question 10: What do the First Nations principles of OCAP assert?

OCAP, developed by the First Nations Information Governance Centre, stands for ownership, control, access and possession, and asserts First Nations' control over data about their communities. The Cedar Valley team agreed with each Indigenous-led organization what information about its program would be reported.

Question 11: Which description best fits a sampling frame?

A sampling frame is the list from which a study identifies or selects its units. The first option describes a purposive interview sample, the second a charting template and the third an ordered list of exclusion reasons.

Question 12: A scan completed in 2026 states that its information is current to March of that year. How should a planner reading it in 2028 treat its findings?

A scan describes a moment in time, so its date of currency tells readers how old its information is. Programs open, close and change, so a later reader should treat the scan as a historical record until it is updated; no fixed expiry date applies.

Question 13: In the review by Charlton and colleagues (2021), what did just over half (53 percent) of the included studies do?

Fifty-three percent of studies combined passive approaches, which look at existing information, with active approaches, which seek new information through surveys or interviews. Literature reviews were the most common single method (71 percent), but most scans did not rely on them alone.

Question 14: A scan asks how each Canadian province and territory funds social prescribing. What is the most suitable unit for this scan?

The question concerns provincial and territorial funding and policy, so the unit is the jurisdiction, and with 13 provinces and territories a census is feasible. A program or organization would be the unit for a question about the services available locally.

Question 15: Before charting all programs, two team members chart three programs independently, each with 18 entries, and agree on 45 entries. What is their percent agreement?

Three programs with 18 entries each give 54 entries, and 45 divided by 54 is 0.833, or 83 percent. Dividing 54 by 45 instead gives the 120 percent distractor. The pattern of the 9 disagreements, which concerned main activity, evaluation status and funding dates, led to three new coding rules.
✦ Complete the final reflection above before submitting

Congratulations!

You have successfully completed this lesson: Environmental Scans.

You can now define an environmental scan and place it among related products, plan document reviews, program inventories, organizational surveys and key informant interviews, build and judge a sampling frame, choose a census or a purposive sample for each method, design and pilot a charting template, integrate a scan with a review in a joint display, and report a scan with stated denominators, limitations and a date of currency.

Lesson 12, Reporting and Translating Review Findings, brings the Cedar Valley review and scan together in a report written to PRISMA 2020 and its extensions, with characteristics tables, summary of findings tables and evidence maps, a plain-language summary and an evidence brief for the planning team.

Reference

Glossary: Key Terms, People & Frameworks

📚 Reference page, available throughout the lesson

Definitions of the terms, tools and people introduced in this lesson on environmental scans.

Core Concepts
Environmental scan A structured collection and interpretation of information about current programs, services, policies, organizations and trends in a defined setting, carried out to inform planning.
Passive scanning Scanning that gathers information that already exists, such as literature, documents, websites and directories.
Active scanning Scanning that generates new information with data collection tools, such as surveys of organizations and key informant interviews.
Codified knowledge Knowledge written down in reports, statistics, policy documents and other records.
Tacit knowledge Knowledge held in people's experience, which emerges in interviews, focus groups, meetings and conversation.
Jurisdictional scan A scan whose units are jurisdictions, such as provinces, territories or countries, usually comparing their policies, programs or funding.
Program inventory A running list of every candidate program a scan identifies, with the sources where each was found and the decision made about it.
Listing One mention of a program in one source; several listings may refer to the same program and are de-duplicated before screening.
Sampling frame The list from which a study identifies or selects its units; an ideal frame contains every eligible unit exactly once.
Census The inclusion of every eligible unit in a frame, used when the population is small or a method is cheap.
Purposive (judgement) sampling Deliberate selection of units for what they can contribute to the question, as opposed to random selection. Purposive sampling and judgement sampling are two names for the same method.
Maximum variation sampling A purposive strategy that selects units differing as much as possible on characteristics that matter to the question.
Snowball sampling Asking participants to suggest further units, which helps to find programs that directories miss.
Key informant A person who knows about a topic because of the position they hold, such as a program coordinator, funder or manager.
Organizational survey A survey in which one informed person reports facts about a program or organization on its behalf.
Response rate The number of units that completed a survey divided by the number invited, reported with both numbers.
Non-response bias Bias that arises when units that do not respond differ systematically from those that do, so that results describe respondents better than the whole population.
Charting template The form on which a scan records the same fields for every unit, with a source and date of verification for each entry.
Data dictionary A document giving the definition, allowed values and coding rules for each field of a charting or extraction form.
Member checking Returning a summary of an interview or a charted entry to the informant or organization so that they can correct it.
Triangulation Comparing what different sources or methods say about the same topic, to check findings and explain differences.
Joint display A table or figure that places the results of two components side by side so that readers can draw integrated interpretations.
Frameworks & Tools
Aguilar's modes of scanning Four modes described by Aguilar in 1967: undirected viewing, conditioned viewing, informal search and formal search, which differ in how purposeful and structured the search is.
SWOT analysis A planning tool that sets an organization's internal strengths and weaknesses beside the external opportunities and threats it faces, often informed by a scan.
Seven-step scanning process The process proposed by Wilburn, Vanderpool and Knight (2016), from determining leadership and purpose to analyzing results and disseminating them to interest holders.
TCPS 2 Article 2.5 The provision of Canada's Tri-Council Policy Statement stating that quality improvement and program evaluation used exclusively for assessment, management or improvement fall outside research ethics board review.
OCAP principles The First Nations principles of ownership, control, access and possession of data about First Nations communities, developed by the First Nations Information Governance Centre.
Tailored design method An approach to survey design and administration developed by Dillman and colleagues that uses a planned sequence of personalized contacts to encourage response.
Framework analysis (framework method) A qualitative analysis approach, also called the framework method, that summarizes data in a matrix of cases by topics, suited to structured questions and short timelines (Gale et al., 2013).
CHERRIES The Checklist for Reporting Results of Internet E-Surveys (Eysenbach, 2004), which a scan can borrow for reporting an online survey.
COREQ The Consolidated Criteria for Reporting Qualitative Research (Tong, Sainsbury and Craig, 2007), which a scan can borrow for reporting interviews.
Key People
Francis J. Aguilar A business scholar whose 1967 book Scanning the Business Environment described how managers acquire information about the external environment and set out four modes of scanning.
Chun Wei Choo An information scientist who linked Aguilar's modes of scanning to research on information seeking and organizational learning.
Patricia Charlton and colleagues Health services researchers who published a scoping review protocol (2019), a scoping review of 96 studies (2021) and a survey of people who conduct and use scans (2024) on environmental scans in health services delivery research.
P. Graham, T. Evitts and R. Thomas-MacLean Authors of a 2008 article in Canadian Family Physician on the usefulness of environmental scans for primary care research, which distinguished passive and active scanning.
Rowel and colleagues Authors of a 2005 article in the Journal of the National Medical Association on the utility of the environmental scan for public health practice, drawn from a cancer screening project in Baltimore.
Wilburn, Vanderpool and Knight Authors of a 2016 article in Preventing Chronic Disease that described a scan for Kentucky's human papillomavirus vaccination project and proposed a seven-step process.
Don A. Dillman A survey methodologist known for the tailored design method for mail, telephone, internet and mixed-mode surveys.
Michael D. Fetters and John W. Creswell Mixed-methods researchers who described integration at the design, methods, and interpretation and reporting levels, and the use of joint displays.
No matching entries. Try a different search term.