# Lesson 11: Environmental Scans

*Companion-podcast transcript, Sarah and Kiffer*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. Today we are on Lesson Eleven, which is about environmental scans.

**Sarah:** I'll admit that the phrase "environmental scan" makes me picture someone measuring air quality.

**Kiffer:** That is a common first reaction. In this lesson the environment means the organizational setting around the people who are planning something. For a health authority, that includes community organizations, governments, funders and the residents it serves. A scan describes what is going on in that setting right now. We have mentioned scans since Lesson One, and this is the lesson where we teach the methods properly.

**Sarah:** Let's set up the running case before we start.

**Kiffer:** The Cedar Valley Health Authority is fictional. It serves about two hundred and ten thousand people in British Columbia, about forty-six thousand of them aged sixty-five and older. Its planning team wants to launch a community connector program, which is a form of social prescribing, for older adults. Before it does, it asked a small evidence team, made up of an evidence officer, a university librarian and a student intern, for a rapid scoping review and an environmental scan within twelve weeks.

**Sarah:** And the scan asks what, exactly?

**Kiffer:** It asks which programs for socially isolated older adults currently operate in British Columbia and elsewhere in Canada, who runs them, and how they are funded and evaluated. The scan ran from week three to week ten, alongside the review. It identified eighteen programs in British Columbia. Eleven answered a short survey, and leaders from seven took part in interviews. All of these numbers are illustrative, but they are consistent across the lesson.

**Sarah:** Good. Section One starts with a definition. What is an environmental scan?

**Kiffer:** A scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, so that an organization can plan. The key words are current and defined. A scan describes what exists now in a particular place.

**Sarah:** Is that a definition everyone agrees on?

**Kiffer:** No, and the lesson is honest about that. Patricia Charlton and her colleagues published a scoping review in twenty twenty-one of ninety-six studies that used environmental scans in health services delivery research. They found no standard definition and a good deal of inconsistent terminology.

**Sarah:** So what did they find that the definitions had in common?

**Kiffer:** Their thematic analysis produced several themes. A scan is described as an instrument of discovery, meaning a way of finding out what is there. It is a form of knowledge synthesis, because it combines many sources into one account. It is forward-looking, because it notices trends. And it supports decision making, because someone needs it to choose what to do.

**Sarah:** The lesson says the idea came from business. Tell me about that.

**Kiffer:** In nineteen sixty-seven, Francis Aguilar published a book called Scanning the Business Environment. He described scanning as the way managers acquire information about events and relationships outside their company that helps them plan its future. He also described four modes of scanning. The first is undirected viewing, which is general exposure to information with no purpose in mind, like noticing a story in a newsletter. Conditioned viewing, which is paying attention to selected sources, like subscribing to updates from a ministry. Informal search, which is a limited, unstructured effort to find something specific, like phoning two colleagues. And formal search, which is a planned effort that follows a procedure.

**Sarah:** And a scan in this course is the last one.

**Kiffer:** Yes, an environmental scan as we teach it is a formal search. The other modes still matter, because they often prompt a scan and supply leads that the formal search then checks. Chun Wei Choo later connected Aguilar's modes to research on how organizations seek information and learn.

**Sarah:** How did it get from business into health?

**Kiffer:** Scanning became part of strategic planning, where it feeds tools such as an analysis of strengths, weaknesses, opportunities and threats. Graham, Evitts and Thomas-MacLean, writing in Canadian Family Physician in two thousand eight, note that it spread to the non-profit sector, and that the United Way of America used comprehensive scanning in the early nineteen eighties. Health researchers then picked it up for topics such as mental health, nutrition, women's health, Indigenous health and housing.

**Sarah:** Give me a concrete health example.

**Kiffer:** Rowel and colleagues, in two thousand five, described a scan done in two thousand three for a project to increase cancer screening among African Americans in Baltimore. It combined a review of community organizations, local cancer incidence data, and community assets and liabilities with focus groups and key informant interviews with service providers. The authors found it flexible, responsive and cost-effective, and they said it helped mobilize the community. They also observed that the scan was poorly defined, rarely evaluated and seldom used in public health, and that there was little guidance on how to design or evaluate one. That observation still holds in part, which is one reason this lesson spends so much time on documentation.

**Sarah:** The reading also mentions a scan in Kentucky.

**Kiffer:** Wilburn, Vanderpool and Knight described a scan for a statewide human papillomavirus vaccination project in Kentucky, published in two thousand sixteen. It combined interviews, a survey of health care providers, reviews of the literature and policy, a scan of television coverage and registry data. From that experience they proposed seven steps, running from deciding who leads the work and what it is for, through choosing what information to collect and engaging interest holders, to analyzing the results and sharing them.

**Sarah:** And there is a Canadian example too.

**Kiffer:** The Canadian Agency for Drugs and Technologies in Health, which is now Canada's Drug Agency, has published environmental scans as one of its report types. Its process document describes scans of health care practices inside and outside Canada, based on a limited literature search, a survey of health care decision-makers, or both. Those scans give a snapshot of current practice and do not make recommendations.

**Sarah:** Section One also distinguishes passive and active scanning.

**Kiffer:** That distinction comes from Graham and colleagues. Passive scanning gathers what already exists, such as literature, reports and directories. Active scanning generates new information with tools such as surveys and interviews. They also distinguish codified knowledge, which is written down, from tacit knowledge, which lives in people's experience and comes out when you talk to them.

**Sarah:** Do most scans use both?

**Kiffer:** Most do. In the Charlton review, literature reviews appeared in seventy-one percent of scans, key informant or semi-structured interviews in forty-six percent, and surveys in thirty-five percent. Fifty-three percent combined passive and active approaches. Cedar Valley used three passive methods and two active ones.

**Sarah:** How is a scan different from a needs assessment? They sound similar.

**Kiffer:** They overlap, and people who conduct scans say so. A needs assessment asks what a population needs and how its needs compare with the services available. A scan asks what services exist. A scan often supplies the services side of a needs assessment. A program evaluation is different again, because it judges whether one program works. And a review asks what research has found, wherever that research was done.

**Sarah:** And the Cedar Valley question itself?

**Kiffer:** In Lesson Two, the team asked which organizations currently deliver community-based programs for socially isolated older adults, and how those programs are designed, governed, funded and evaluated, in British Columbia and elsewhere in Canada. It added six sub-questions. They cover where programs are and who runs them, what they offer and how people reach them, staffing and funding, evaluation, the barriers and advice that program leaders describe, and models in other provinces.

**Sarah:** Section Two is about methods. Where does the work start?

**Kiffer:** With documents and websites. The team began with what the review had already found. Lesson Seven showed that the review excluded twelve documents because they described a program without evaluating it. Those twelve, along with many of the twenty-six grey-literature documents the review included, became the first documents of the scan. Then the intern searched sources chosen for the scan, such as the provincial two-one-one directory of community services, the list of local Better at Home programs, the community resource lists of the regional health authorities, and municipal and seniors' centre websites. The rules from Lesson Five apply. For each search, the intern recorded the source, the date, the search terms or browsing path, the number of listings and who searched. For every document kept, the intern saved a copy, because program websites change and disappear. A saved copy shows exactly what a page said when the scan used it.

**Sarah:** Can you trust what program websites say?

**Kiffer:** With care. Websites are written to attract participants and funders. They rarely mention funding, reach or evaluation, and some describe programs that have since changed or closed. The team coded missing information as not reported, and it is important to read that code correctly. It tells you about the source. It tells you nothing about whether the program has the feature.

**Sarah:** Next comes the program inventory.

**Kiffer:** An inventory is a running list of every candidate program, with the sources where each was found and the decision made about it. It does the job that the reference library and screening record do in a review. As in a review, the same thing turns up in several places, so the team separated a listing, which is one mention in one source, from a program, which is the service itself.

**Sarah:** So duplicate listings come out first.

**Kiffer:** Yes, and then each remaining program is screened against the criteria. Each exclusion gets one reason from an ordered list: not currently operating, based in a facility such as a long-term care home, no component that addresses social connection, not mainly for adults aged sixty-five and older, or a one-time event. Using the first reason that applies means the counts add up.

**Sarah:** What about the survey?

**Kiffer:** Documents rarely say how a program is funded, how many people it reaches, or whether it is evaluated, and those were among the planning team's main questions. So the team surveyed every eligible program. This is an organizational survey, in which one informed person, usually the coordinator, reports facts about the program on its behalf.

**Sarah:** What makes a good survey of this kind?

**Kiffer:** It should be short. The Cedar Valley survey took about ten minutes. It should ask only what documents cannot answer, and it can show what the team has already recorded and ask the respondent to confirm it. Each item should ask about one thing and state its time frame, such as the past twelve months, and the options should include don't know. The team also piloted it with two coordinators whose programs fell outside the scan.

**Sarah:** How did they get people to respond?

**Kiffer:** By planning the contacts. Dillman, Smyth and Christian recommend a sequence of personalized contacts, each with a different purpose. Cedar Valley sent an advance email from the planning lead, an invitation from the evidence officer with the survey link, two reminders a week apart and a final phone call. The survey stayed open for three weeks.

**Sarah:** And eleven of the eighteen programs answered.

**Kiffer:** Eleven of eighteen, which is sixty-one percent. The team reported both numbers and then compared respondents with non-respondents using what the documents showed. Five of the seven programs that did not respond were run entirely by volunteers, compared with three of the eleven that did respond.

**Sarah:** So the survey tells you more about programs with paid staff.

**Kiffer:** Probably, yes. That is a likely case of non-response bias, and the report says so plainly. It does not make the survey useless. It tells readers which programs the survey describes well.

**Sarah:** Then come the interviews.

**Kiffer:** A key informant is someone who knows about a topic because of the position they hold, such as a program coordinator or a funder. Interviews suit how and why questions. Why are referrals from clinicians rare? What happens when a grant ends? What should a newcomer avoid? The research methods course in this series, in its Lesson Ten, teaches how to write a guide and probe, and this lesson applies those skills.

**Sarah:** What was in the Cedar Valley guide?

**Kiffer:** It opened with the program's history, then asked how older adults find their way to the program, which organizations it works with, especially primary care, how secure its funding is, how it knows whether it is helping, and what advice the informant would give a health authority starting a connector program. Each main question had one or two probes.

**Sarah:** How did the interviews run in practice?

**Kiffer:** The interviews took thirty to forty-five minutes by video call or phone. The evidence officer asked the questions and the intern took notes. With consent, calls were recorded so that notes could be checked, and the intern wrote a summary within a day. Each informant then received their summary to correct. That step is called member checking. The team invited ten programs and seven took part, one informant each. We will come back to how the ten were chosen in Section Three.

**Sarah:** Section Two ends with ethics. Does a scan need ethics approval?

**Kiffer:** It depends, and the team has to settle it early. In Canada, the Tri-Council Policy Statement sets out what counts as research. Its Article Two point Five says that quality improvement and program evaluation activities used only for assessment, management or improvement do not fall within the scope of research ethics board review. Article Two point Two covers research that relies only on publicly available information.

**Sarah:** So who decides?

**Kiffer:** The organization's research ethics or quality improvement office. At Cedar Valley, that office judged the scan to be a planning activity. Plans to publish could change the answer. And even without board review, the team followed ethical practice. Participation was voluntary, programs were named only with permission, quotations were attributed by role, and sensitive information about funding and staffing was reported in aggregate.

**Sarah:** And programs led by Indigenous organizations?

**Kiffer:** Those need particular care. The Tri-Council Policy Statement has a chapter on research involving First Nations, Inuit and Métis Peoples, and the First Nations principles of ownership, control, access and possession, developed by the First Nations Information Governance Centre, assert First Nations' control over data about their communities. Cedar Valley planned its outreach with the health authority's Indigenous health team and agreed with each organization what would be reported about its program. And, as Lesson Six said, no scan data were entered into any artificial intelligence tool.

**Sarah:** Section Three is about sampling. Where does that start?

**Kiffer:** With the unit, meaning the thing the scan counts. It could be a program, an organization or a jurisdiction such as a province. Cedar Valley counted programs, because the planning team wanted to know what kinds of programs exist. It also recorded the organization running each one, because the eighteen programs were run by sixteen organizations.

**Sarah:** And then eligibility criteria.

**Kiffer:** Criteria that parallel the review's population, concept and context. Programs had to serve mainly adults aged sixty-five and older living in the community, they had to be organized and recurring with some aim or component that addresses social connection, and they had to have operated in British Columbia within the past twelve months, which allows for programs that pause over the summer or between grants.

**Sarah:** Now the sampling frame.

**Kiffer:** A sampling frame is the list from which you identify your units. Ideally it has every eligible unit exactly once. A survey of physicians can use a licensing register, but there is no single list of community programs for older adults in British Columbia. So the team built one from six kinds of source.

**Sarah:** Walk me through the numbers.

**Kiffer:** The provincial directory gave nineteen listings, the Better at Home list fourteen, the health authority lists eleven, municipal and seniors' centre websites nine, grey-literature documents seven, and referrals from survey respondents and interviewees four. That is sixty-four listings. Removing twenty-one duplicates left forty-three candidate programs, and screening left eighteen eligible ones.

**Sarah:** How do you know you didn't miss programs?

**Kiffer:** You can never be certain, but the overlap among sources gives a signal. Thirteen of the eighteen programs appeared in two or more sources, and five appeared in only one. Two of those five were found only through referrals, and both were small volunteer-run programs. If programs of that kind appear in only one source, similar programs probably appear in none. The report says the frame probably under-represents small volunteer-run programs. Epidemiologists have formal capture-recapture methods for estimating a population from overlapping lists, but those methods assume that the lists are independent, and directories often copy from one another, so the team used the overlap descriptively.

**Sarah:** Did they survey and interview everyone?

**Kiffer:** They surveyed everyone. When the number of eligible units is small and a method is cheap, you can take a census and include them all. Documents and the survey covered all eighteen. Interviews cost much more time, so the team used purposive sampling, specifically maximum variation sampling.

**Sarah:** Remind me what maximum variation means.

**Kiffer:** You deliberately choose units that differ as much as possible on the characteristics that matter. The team varied programs by main activity, by location in the Cedar Valley region or elsewhere in the province, by urban or rural setting, by type of operator, by whether the program was Indigenous-led, and by whether it measured outcomes. It also included programs that had not answered the survey, so that the interviews could hear from them. In the end it invited ten programs, eight that had answered the survey and two that had not. Seven took part, six survey respondents and one non-respondent. Putting the methods together, twelve of the eighteen programs gave information directly, eleven through the survey and one through an interview only, and the other six were charted from documents alone.

**Sarah:** The scan question also mentions the rest of Canada. How did they handle that?

**Kiffer:** That part is a small jurisdictional scan. Canada has ten provinces and three territories, so a national scan can often include all thirteen. With twelve weeks, Cedar Valley chose four other provinces, varied by region and population size, and reviewed public documents and websites only. It charted nine programs from those provinces.

**Sarah:** Do those nine get added to the eighteen?

**Kiffer:** No. They were found and described with different methods, so adding them would produce counts with no clear meaning. The report keeps them separate.

**Sarah:** Then the charting template.

**Kiffer:** The charting template is the form on which the scan records the same fields for every program. Like the review's charting form in Lesson Eight, each field has a definition and allowed values in a data dictionary. The template uses the review's intervention categories for main activity, which matters a great deal in Section Four. And it does two things that are particular to scans. It records the source of each entry, and it records the date each program's information was last verified.

**Sarah:** The reading describes three coding rules.

**Kiffer:** First, the template separates not reported, don't know and not applicable, because they mean different things when you count programs later. Second, when sources disagree, the team records the most recent direct report, usually the survey or the interview, and notes the conflict. Third, every entry carries its source.

**Sarah:** And they piloted it.

**Kiffer:** The intern and the evidence officer charted three programs independently, eighteen entries each, so fifty-four entries in all. They agreed on forty-five, which is eighty-three percent. Four disagreements were about the main activity of a seniors' centre that ran both drop-in groups and outreach visits. Three were about whether a satisfaction survey counts as measuring outcomes. Two were about which funding end date to record. So they added three rules. The main activity is the one with the most contact hours, which matches the rule the review used. Satisfaction surveys count as monitoring and not as outcome measurement. And the funding end date is that of the largest source.

**Sarah:** Section Four brings the review and the scan together. Why do they need each other?

**Kiffer:** Because each sees the field from a different side. Most of the forty-two studies in the review were done outside Canada. Most of the eighteen British Columbia programs have never been evaluated in a published study. Read together, they show where local practice and research evidence coincide, where local programs run with little evidence behind them, and where the evidence describes models no local program offers.

**Sarah:** The lesson borrows some vocabulary from mixed methods here.

**Kiffer:** It does. Fetters, Curry and Creswell describe integration at three levels. At the design level, Cedar Valley ran the scan in parallel with the review and brought them together in the synthesis from week nine. At the methods level, the scan built on the review's categories, the review passed program descriptions to the scan, and the scan passed three external evaluation reports back to the review team. At the level of interpretation and reporting, the team built a joint display.

**Sarah:** What exactly is a joint display?

**Kiffer:** A table or figure that sets the results of two components side by side, so that readers can draw conclusions that neither supports alone. Guetterman, Fetters and Creswell described joint displays for health research. In the Cedar Valley display, each row is an intervention category, and the columns show the review's studies and certainty ratings beside the scan's count of local programs and how many measure outcomes.

**Sarah:** What does it show?

**Kiffer:** Three things. First, group activities are the most common local model, six of the eighteen programs, and they also have the most trial evidence, eight randomized trials, although its certainty is low. Second, the community connector model that Cedar Valley plans rests on very uncertain evidence from five non-randomized studies and has only four counterparts in the province, two of which measure outcomes.

**Sarah:** Which supports building an evaluation into the new program.

**Kiffer:** Yes, as Lesson Eight argued, and it suggests the team could adopt a short outcome measure that those two programs already use. The third point is practical. A connector program depends on having programs to refer people to, and the scan shows which kinds exist and where.

**Sarah:** Section Four also talks about analyzing scan data. What's the main risk there?

**Kiffer:** Unclear denominators. Because the survey and interviews reached only some programs, every count has to say which programs it describes and where the information came from. Seven of the eighteen programs measured at least one participant outcome, three of them through an external evaluation. Of the twelve programs that gave information directly, eight said their largest funding source ends within three years, and three had a formal referral pathway from primary care.

**Sarah:** And the interviews?

**Kiffer:** The team used the framework method. It built a matrix with a row for each of the seven programs and a column for each topic in the guide, summarized each cell, and read down the columns for patterns. Most informants said participants arrived by word of mouth and that links with clinics depended on individual clinicians. Several said they would measure outcomes if given a short shared tool. Rural informants named transport as the main barrier.

**Sarah:** And comparing sources is triangulation.

**Kiffer:** Right. The survey showed that three of twelve programs had a formal referral pathway. The interviews explained why that number was low and what had made the pathway work in one program, where connectors attend clinic team meetings.

**Sarah:** How should a scan be reported? Is there a PRISMA for scans?

**Kiffer:** Not at present. Charlton and colleagues noted in twenty twenty-four that the EQUATOR Network, which catalogues reporting guidelines, listed none for environmental scans. So a scan borrows from the guidelines for each component. It can use the search extension of PRISMA for its website and directory searches, the scoping review extension for charting, the Checklist for Reporting Results of Internet E-Surveys for an online survey, and the Consolidated Criteria for Reporting Qualitative Research for interviews.

**Sarah:** And the flow diagram that Lesson Seven promised?

**Kiffer:** The scan has its own flow of programs, built like a PRISMA diagram. Sixty-four listings, minus twenty-one duplicates, gives forty-three candidates. Forty-three, minus twenty-five exclusions, gives eighteen eligible programs. Then the survey and the interviews, and finally twelve programs with direct information and six from documents only. Every box must reconcile with the boxes above it.

**Sarah:** And limitations?

**Kiffer:** The report states that the frame probably misses small volunteer-run programs, which were also less likely to answer the survey, that survey and interview information is self-reported, that no one-to-one program in the Cedar Valley region was interviewed, and that programs in other provinces were described from documents alone. It also states a date of currency.

**Sarah:** Why does the date matter so much?

**Kiffer:** Because a scan describes a moment. Programs open, close and change. Someone reading the Cedar Valley scan in two years should treat it as a record of what existed at the end of week ten, and the team kept the inventory, the saved copies and the template so that the health authority can update it.

**Sarah:** Let's finish by putting it together. How would a small team run a scan of five to ten organizations on its own question?

**Kiffer:** First, it restates the scan question and writes eligibility criteria that parallel the review's criteria. Second, it builds a sampling frame from at least three sources and keeps an inventory with a decision and an ordered reason for every candidate.

**Sarah:** And then?

**Kiffer:** Third, it charts the organizations or programs from public documents and websites, with a template, a data dictionary, and a source and date for every entry, using categories that match the review's charting table from Lesson Eight. Any contact with staff follows the ethics decision the team settled early. Fourth, it draws the flow of units and drafts a joint display that sets the review beside the scan.

**Sarah:** Any last advice?

**Kiffer:** Record the source and the date of everything. Most of what makes a scan trustworthy comes down to whether a reader can see where each piece of information came from and when.

**Sarah:** Next time, Lesson Twelve, reporting and translating review findings.

**Kiffer:** We will bring the review and the scan together into a report and an evidence brief that the Cedar Valley planning team can actually use.

**Sarah:** Thanks, Kiffer. And thanks to everyone for listening to Office Hours.