# Lesson 12: Reporting and Translating Review Findings

*Companion-podcast transcript, Sarah and Kiffer*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This is the episode for Lesson twelve of Health Sciences two forty-one, and it is the last lesson of the course. Once a review team has finished its analysis, it still has to write the work up, present it to the people who asked for it, and decide how to keep it current. That is what this lesson covers.

**Sarah:** So the Cedar Valley team has reached the end.

**Kiffer:** It has. The Cedar Valley evidence review is our fictional running case. A health authority in British Columbia is planning a community connector program, a form of social prescribing, for adults aged sixty-five and older, and it asked a small evidence team for a rapid scoping review and an environmental scan within twelve weeks. The team is an evidence officer, a university librarian and a student intern.

**Sarah:** Remind us what they have to report.

**Kiffer:** They included forty-two studies, fourteen of them randomized trials, and twenty-six grey-literature documents. They rated the certainty of evidence on loneliness as low for group-based programs and for one-to-one befriending or telephone programs, and very low for community connector programs. And the scan found eighteen programs in British Columbia, eleven of which answered a survey, while seven people took part in key informant interviews.

**Sarah:** That is a lot of material. Where do you start, and why do reports need so much attention? If the review was done well, surely the report follows.

**Kiffer:** The team writes a full report for the record, posted with its protocol on the Open Science Framework, and a short evidence brief for the planning team. Section one is about the report, and a reader can only judge what a report shows. If I cannot see which databases were searched, I cannot tell whether the search was complete. Chalmers and Glasziou, in two thousand nine, counted incomplete and unusable reporting among the main sources of avoidable waste in health research. Studies of published systematic reviews have kept finding missing search strategies and missing descriptions of how risk of bias was assessed.

**Sarah:** And the response is the reporting guideline.

**Kiffer:** Yes. A reporting guideline is a checklist of the minimum information a report should contain, developed by consensus among methodologists, editors and users, and usually published with an explanation and elaboration paper that gives the reasons for each item and examples. The EQUATOR Network, which stands for Enhancing the Quality and Transparency of health Research, keeps a library of them.

**Sarah:** Does following a checklist make a review good?

**Kiffer:** No. A reporting guideline tells you what to report, and methods guidance such as the Cochrane Handbook tells you how to conduct the review. A well-reported weak review is still weak, but the weakness is visible, so readers can discount the conclusions accordingly.

**Sarah:** Let's talk about PRISMA, since students have met it before.

**Kiffer:** They met the flow diagram in Lesson seven. PRISMA stands for Preferred Reporting Items for Systematic reviews and Meta-Analyses. The twenty twenty version, led by Matthew Page and colleagues, has twenty-seven items grouped into seven parts: the title, the abstract, the introduction, the methods, the results, the discussion and a final part called other information. There is also a separate twelve-item checklist for abstracts.

**Sarah:** Is there a logic to the order, or is it just a list?

**Kiffer:** There is a logic. The methods and results mirror each other. If item eleven tells you how risk of bias was assessed, item eighteen gives you the judgements. And the discussion item splits limitations into two kinds, the limitations of the evidence and the limitations of the review process. We will come back to that, because many reports blur it.

**Sarah:** The Cedar Valley review is a scoping review, though. Does PRISMA still apply?

**Kiffer:** That is where the extensions come in. The PRISMA extension for scoping reviews, usually written PRISMA S C R, was led by Andrea Tricco and published in twenty eighteen. It has twenty essential items and two optional ones. It talks about sources of evidence instead of studies, because a scoping review can include documents that are not research, and it makes critical appraisal optional.

**Sarah:** And there are others.

**Kiffer:** Several. The extension for literature searches, PRISMA S, has sixteen items, and students used it in Lesson four. The protocol guideline, PRISMA P, has seventeen items. The Synthesis Without Meta-analysis guideline, known as SWiM, has nine items for reviews that synthesize effects without pooling, which Lesson nine teaches.

**Sarah:** How does a team choose among them?

**Kiffer:** It chooses one main checklist that matches its review type, then adds others for particular parts of the report. For Cedar Valley, the main checklist is the scoping review extension. The search is reported with the search extension, and the synthesis of effects on loneliness follows SWiM.

**Sarah:** What about the certainty ratings? You said scoping reviews seldom rate certainty.

**Kiffer:** Right, and the scoping review extension has no item for it. So the team borrows items fifteen and twenty-two from PRISMA twenty twenty, which cover how certainty was assessed and what the ratings were, and it explains in the methods why a scoping review rated certainty for one outcome. The short answer is that the planning team asked directly how confident it could be.

**Sarah:** And the environmental scan?

**Kiffer:** There is no single agreed reporting guideline for environmental scans, so the team reports the elements set out in Lesson eleven, such as how programs were identified, what was asked and how responses were charted.

**Sarah:** Let's go back to the two kinds of limitation. Why does the distinction matter?

**Kiffer:** Because it tells the reader what to do next. Limitations of the evidence belong to the included studies. In Cedar Valley, no randomized trial was at low risk of bias for loneliness, because participants knew which program they were in and reported their own loneliness. And only six of the forty-two studies were Canadian. Those problems would remain even if the review were done perfectly. Limitations of the review process come from the team's own decisions, such as charting only English and French reports, limiting the search to reports from two thousand ten onward, or having one person chart the descriptive fields with a second person checking.

**Sarah:** What about the final part of the report, the other information? It sounds like housekeeping.

**Kiffer:** It is easy to treat it that way, and it is a mistake. Each item tells the reader something they need. Item twenty-four asks for the registration and for any amendments to the protocol. The Cedar Valley team made six, and two of them concern how results were chosen. In week seven it added a rule for choosing one loneliness scale when a study reported two, taking the first available scale from a ranked list that starts with the UCLA scale. In week eight it added a rule for time points, using the end-of-program result in the synthesis and charting the longest follow-up separately. All six are recorded as dated notes on the registration.

**Sarah:** Why would a reader care about rules like that?

**Kiffer:** Because without a rule, a team can choose whichever result looks best, even without meaning to. Reporting the rule and when it was added lets a reader judge whether it could have shaped the findings.

**Sarah:** And funding?

**Kiffer:** This one is interesting for Cedar Valley. The review was funded by the health authority that will make the decision, and the evidence officer is employed by that authority. That is a competing interest that readers deserve to know about. So the report says who set the question, that the planning team commented on the draft brief, and that the evidence team made all decisions about methods and wording.

**Sarah:** Students spent a whole lesson on artificial intelligence tools. Does that show up in the report too?

**Kiffer:** It does. Lesson six taught students to log every use of an artificial intelligence tool and to verify its output. In the report, each use is described in the methods: the tool and its version, the date, the task, and how the output was checked. In week five, after the database searches, the Cedar Valley team used a conversational assistant and an artificial intelligence research assistant to suggest sources and check the sensitivity of the search, verified every citation they suggested, and put the log in an appendix. Emerging guidance such as the RAISE recommendations, which stands for Responsible AI in Evidence Synthesis, expects that kind of disclosure.

**Sarah:** And the last item is data availability.

**Kiffer:** The team shares its search strategies, screening decisions, charting form and completed charting table on the Open Science Framework. That lets others check the review and makes an update much easier. The scan is the exception. The survey responses and interview notes were collected on the understanding that they would be reported in summary, so only the template and a summary table are shared.

**Sarah:** That brings us to section two, tables and figures. Why give them a whole section?

**Kiffer:** Because many readers go to the tables first. Rosenbaum and colleagues tested this in a randomized comparison and found that adding a summary of findings table to Cochrane reviews improved readers' understanding of the results and how quickly they found key information.

**Sarah:** Are there general rules for a good table?

**Kiffer:** A few. Each table answers one question, and its title says what that question is. Rows are ordered in a way that means something to the reader, such as by intervention category and then by design. The terms match the text and the charting form. Participant numbers go beside any percentage. Abbreviations and longer explanations go in footnotes. And a reader should be able to understand the table without reading the text around it.

**Sarah:** Start with the characteristics table.

**Kiffer:** The characteristics table has one row for each included study, drawn straight from the charting form in Lesson eight. It gives the study and country, the design, the participants, the program and comparator, and the outcome measures with their time points. In a review with forty-two studies, the full table goes in an appendix, and the report carries a short excerpt and a summary.

**Sarah:** What does the Cedar Valley summary show?

**Kiffer:** It groups the forty-two studies by intervention category and design. Fourteen evaluated group-based programs, eight of them in randomized trials. Eight evaluated one-to-one programs, six of them in trials. Twelve evaluated community connector programs, and none of those were trials. The rest evaluated intergenerational or technology-based programs. You can see at once that the model the health authority wants to launch has no randomized evidence behind it.

**Sarah:** Next is the summary of findings table. Students built an evidence profile in Lesson eight. How is this different?

**Kiffer:** They are two views of the same judgements. The evidence profile records the judgement for each certainty domain: risk of bias, inconsistency, indirectness, imprecision and publication bias. It is a working document for the review team and for guideline panels. The summary of findings table is for readers. For each important outcome, it gives the number of studies and participants, what they found and the certainty, with footnotes that explain each rating.

**Sarah:** And when there is a meta-analysis?

**Kiffer:** Then the table includes the anticipated absolute effects and the relative effect with its confidence interval. Those calculations belong to Health Sciences two thirty, Lesson two, so I will not teach them here. Guidance from the Grading of Recommendations Assessment, Development and Evaluation working group, known as GRADE, also suggests no more than seven outcomes, and including important outcomes even when no study measured them, so the gap is visible.

**Sarah:** Cedar Valley did not pool its results. What goes in the table instead?

**Kiffer:** Following Murad and colleagues, the team replaced the effect columns with a description in words of the direction, size and consistency of the results. For group-based programs, eight trials with one thousand two hundred and thirty-six participants, five trials reported small reductions in loneliness, two reported little or no difference and one reported a larger reduction. The certainty is low, so the statement is that group-based programs may reduce loneliness slightly.

**Sarah:** And the connector programs?

**Kiffer:** Five non-randomized studies with one thousand four hundred and eighty participants. Most reported lower loneliness among participants than among comparison groups, and two found larger reductions among people with more connector contacts. But the certainty is very low, so the statement is that the evidence is very uncertain about the effect of community connector programs on loneliness.

**Sarah:** Those exact words come from somewhere, don't they?

**Kiffer:** They come from Santesso and colleagues, writing in twenty twenty, who proposed standard wording matched to each level of certainty. High certainty gets a plain verb, reduces. Moderate gets probably reduces. Low gets may reduce. Very low gets the evidence is very uncertain. Using the same words everywhere helps readers learn what they mean.

**Sarah:** You also included rows for things the team did not rate.

**Kiffer:** Yes, for intergenerational and technology-based programs, which had too few studies, and for outcomes like social isolation and health service use, which the protocol said would be charted but not rated. Reporting them prevents a reader from mistaking silence for an absence of effect.

**Sarah:** Then there is the evidence map. What is that?

**Kiffer:** An evidence map shows how much evidence exists for each combination of two dimensions, usually interventions and outcomes. The Cedar Valley map has the five program categories as rows and six outcome domains as columns, with a circle in each cell sized by the number of studies.

**Sarah:** What does it show?

**Kiffer:** Loneliness was measured as a quantitative outcome in thirty-three studies, mostly of group-based and community connector programs. Health service use, which matters a great deal to a health authority, was measured in only seven studies, five of them about connector programs. And intergenerational and technology-based programs have few studies in any column.

**Sarah:** I can imagine a board member looking at the big circle for group programs and loneliness and saying those programs work best.

**Kiffer:** That is exactly the misreading to guard against. A big circle means many studies measured that outcome for that category. It says nothing by itself about whether the program works or how certain the evidence is. For that, you go back to the summary of findings table, which rates the group-based evidence as low certainty.

**Sarah:** Any advice on figures generally?

**Kiffer:** Each figure answers one question. The flow diagram shows how studies were found, risk-of-bias plots show where bias clusters, and effect direction plots show which way results point. And every figure should be accessible. That means a caption that states the message, alternative text for screen-reader users, and never relying on colour alone, which is why the Cedar Valley map prints a number in every circle.

**Sarah:** Section three is where the findings leave the research world.

**Kiffer:** Yes. Turning the review into something the planning team can use is part of knowledge translation. The Canadian Institutes of Health Research describe knowledge translation as a dynamic and iterative process that includes synthesis, dissemination, exchange and the ethically sound application of knowledge.

**Sarah:** Is there a simple way to plan it?

**Kiffer:** John Lavis and colleagues proposed five questions in two thousand three: what should be transferred, to whom, by whom, how, and with what effect. For Cedar Valley, the message is the findings with their certainty plus the scan results, the audiences include the planning team and the older adults who will be affected, and the hoped-for effect is a launch designed to answer the questions the evidence leaves open.

**Sarah:** So one review produces several products.

**Kiffer:** Each one complete for its reader. The scientific abstract serves researchers. The plain-language summary serves patients, families and the public. The evidence brief serves decision-makers. The full report serves analysts and future update teams. And they must all say the same thing.

**Sarah:** Why stress that?

**Kiffer:** Because it is easy to drift. A plain-language summary that sounds more confident than the review misrepresents the evidence, even if every sentence is technically accurate. The Cedar Valley team avoided that by building every product from its summary of findings table.

**Sarah:** Let's talk about the plain-language summary. What makes a good one?

**Kiffer:** The main answer comes first. Sentences are short and in the active voice, so we searched instead of searches were conducted. Everyday words replace technical ones. The word significant is avoided, because most readers hear it as important. And certainty is expressed with the same verbs as the summary of findings table, may, probably, and we are very uncertain.

**Sarah:** Cochrane publishes these with every review.

**Kiffer:** It does, and its template uses headings phrased as questions, such as what did we find, what are the limitations of the evidence, and how up to date is this evidence. I think the last two matter as much as the findings, because readers otherwise assume that published evidence is certain and current.

**Sarah:** What does the Cedar Valley summary say?

**Kiffer:** It is titled with the question, do community programs help older adults feel less lonely. Its key messages say that group programs may reduce loneliness a little, that weekly visits or calls from a volunteer may also reduce loneliness, and that we are very uncertain whether community connector programs reduce loneliness. It explains what a connector does in one sentence. It ends by saying that the search ran to the tenth of February, twenty twenty-six.

**Sarah:** Did anyone outside the team read it?

**Kiffer:** Yes. The team had members of the health authority's patient and family advisory council read a draft, and they asked for the word intervention to be replaced.

**Sarah:** And the evidence brief?

**Kiffer:** An evidence brief presents review findings to people who must make a decision, together with what the findings mean in their setting. Lavis and colleagues described policy briefs that set out a problem, the options and the considerations for putting an option into practice. The McMaster Health Forum prepares briefs of this kind.

**Sarah:** I've heard of something called the one, three, twenty-five format.

**Kiffer:** That came from the Canadian Health Services Research Foundation, whose work now continues in Healthcare Excellence Canada. It means one page of main messages, a three-page executive summary and a report of no more than twenty-five pages. The idea is graded entry. Each layer is complete for its reader, and a reader who wants more can go to the next layer. The Cedar Valley team adapted it into a two-page brief with key messages on the first page, a full report and open appendices.

**Sarah:** Walk me through the brief itself.

**Kiffer:** The title states the conclusion: community connector programs for older adults, launch with an evaluation built in. Then come five key messages: group-based and one-to-one programs may reduce loneliness slightly, the evidence on connector programs is very uncertain, few studies measured service use or costs, most British Columbia programs depend on short-term grants, and the team recommends launching in stages across the twenty-four clinics.

**Sarah:** And the rest of the first page?

**Kiffer:** The question and why it matters, a short methods box, and the research findings. The second page has a compact summary of findings table, the scan results, what this means for Cedar Valley, the limitations, and who prepared and funded the brief.

**Sarah:** Let me push on the recommendation. Isn't it the planning team's job to decide?

**Kiffer:** Usually, yes. Evidence teams often describe implications and leave the decision to the people who have to weigh costs, values and feasibility. In this case, though, the planning team asked directly how to launch the program. So the brief makes a recommendation, labels it clearly as the evidence team's, and bases it on the gap in the evidence instead of on a claim that the program works.

**Sarah:** What exactly does it recommend?

**Kiffer:** Introducing the program in four groups of six clinics, three months apart, so that clinics that have not yet started can serve as comparisons for those that have. That is called a stepped-wedge design. It also recommends measuring loneliness and health service use from the start, building referral agreements with the programs found in the scan, and planning for people in smaller communities.

**Sarah:** And applicability?

**Kiffer:** Applicability is whether findings from the included studies are likely to hold in Cedar Valley. With only six Canadian studies, that is a real question, and the scan supplies what the published studies cannot say about who runs programs in British Columbia and how they are funded.

**Sarah:** Section four. The brief ends with a date. Why does that matter so much?

**Kiffer:** Because every review describes the evidence as it stood on its search date, and the world moves on. Kaveh Shojania and colleagues followed a sample of one hundred reviews and found that the median time before new evidence signalled a need to update was about five and a half years, and almost a quarter needed updating within two years.

**Sarah:** Is it always new studies that make a review outdated?

**Kiffer:** Not always. Methods change too, as when RoB two replaced the original Cochrane risk-of-bias tool. The decision context can change, and new kinds of intervention can appear, as with technology-based programs for older adults.

**Sarah:** How does a team decide whether to update?

**Kiffer:** Paul Garner and colleagues proposed three questions in twenty sixteen. First, is the review still current, meaning it addresses a question that matters to decisions and used acceptable methods? Second, are there new methods or new studies, data or other information? Third, could they change the findings, the conclusions or the credibility of the review? You update when all three answers are yes.

**Sarah:** The third one sounds like a judgement call.

**Kiffer:** It is. A new small trial that agrees with eight earlier trials of group programs is unlikely to change anything. A first randomized trial of a community connector program probably would, because the existing evidence comes only from non-randomized studies. Teams can make the judgement more consistent by writing down update triggers in advance, which also protects against updating only when the new results are welcome.

**Sarah:** How would a team even know a new trial had appeared?

**Kiffer:** Through surveillance. Saved database searches can send email alerts when new records match the review's strategy. Citation alerts, from Lesson five, report new papers that cite key studies. Trial registries show studies that are under way. And each check is dated and logged.

**Sarah:** What does the Cedar Valley plan look like?

**Kiffer:** The librarian runs monthly alerts, and the evidence officer watches citation alerts on the twelve connector studies and checks trial registries every six months. The environmental scan is refreshed every twelve months, because programs change much faster than research. The whole team makes a formal update decision in February, twenty twenty-seven. Every task goes to permanent staff, because the intern's placement is ending, which is the kind of detail that decides whether a plan actually happens.

**Sarah:** What about living reviews? Lesson one introduced them.

**Kiffer:** A living systematic review is continually updated as new evidence appears. The team searches at a set frequency, often monthly, screens new records as they arrive, and updates the conclusions when pre-specified criteria are met. It is published in dated versions. Julian Elliott and colleagues introduced the idea, and living reviews were used widely during the pandemic, for example in a living network meta-analysis of drug treatments for COVID nineteen that informed the World Health Organization's living guideline.

**Sarah:** When is it worth doing?

**Kiffer:** Elliott and colleagues suggested three conditions. The question is a priority for decision-making. The certainty of the existing evidence is low or very low, so new studies could change the conclusions. And new evidence is likely to appear soon. A living review also needs a standing team, stable funding, a way to publish versions, and a plan for retiring the review when those conditions no longer hold.

**Sarah:** So should Cedar Valley go living? It sounds like it meets all three.

**Kiffer:** It does. What it lacks is capacity: three people working part time, an intern who is leaving and no ongoing funding. So the team chose surveillance with a planned update. If a randomized trial or a large controlled study of a connector program appears, it will revise that row of the summary of findings table and reissue the brief within three months. Otherwise it decides on a full update in February, twenty twenty-seven.

**Sarah:** And the health authority's own launch becomes part of the evidence.

**Kiffer:** That is my favourite part of the case. The staggered launch will produce local evidence on exactly the question the review could not answer, and the update should include it.

**Sarah:** How is an update reported?

**Kiffer:** The title says it is an update. The abstract gives the previous and new search dates and says whether the conclusions changed. A short section lists what is new, including new studies, changed methods, changed certainty and changed conclusions. The flow diagram uses the PRISMA twenty twenty template for updated reviews from Lesson seven. And a reissued brief says which version it replaces.

**Sarah:** Before we finish, this is the last episode. What should students take from the whole course?

**Kiffer:** The Cedar Valley team has now passed through every stage of an evidence synthesis, from choosing a review type to writing a brief. The same habits ran through all of it. They planned before acting, wrote down their decisions and reasons, had a second person check the steps most open to error, and reported what they did in enough detail for others to repeat or update it. Those habits are what make a review trustworthy at any scale.

**Sarah:** And when a team assembles the final report of a rapid scoping review and scan, what goes into it?

**Kiffer:** The team outlines its report under the scoping review extension's headings, puts its full search strategies and its search log in an appendix, and completes the checklist with the section where each item appears. It builds a characteristics table and either an evidence map or a summary of findings table, separates the two kinds of limitation, and reports amendments, any use of artificial intelligence tools, funding and competing interests.

**Sarah:** And the brief?

**Kiffer:** A two-page evidence brief for a named British Columbia decision-maker, with a title that states the message, key messages that carry certainty, a methods box, findings from the review and from the scan, implications, limitations and the date to which the evidence is current. Then one paragraph on how the review should be kept current.

**Sarah:** Thank you, Kiffer, for this episode and for the whole series.

**Kiffer:** Thank you, Sarah, and thank you to everyone who has listened along. Good luck with your own reviews.