Introduction

Lesson 8 showed how researchers collect quantitative data directly and introduced the main kinds of administrative health records: physician billing claims, hospital discharge abstracts, prescription records and vital statistics. This lesson asks where existing data are held in Canada, how a researcher obtains permission to use them, how records from different sources are joined, and what safeguards protect the people whose information is used.

Primary data are data that a research team collects for its own study, such as the answers to a survey it designed. Secondary data were collected for another purpose and are reused for research. A hospital records a diagnosis so that the patient can be treated and the hospital funded; a researcher who later counts hospital stays for heart failure is making secondary use of that record. Secondary data offer large numbers of people, long follow-up and information about people who never answer surveys, with no extra burden on the people described. Their limitations follow from their origin: they contain only what the original purpose required, their coding reflects administrative rules, and access takes time.

1.1 Who Holds Health Data in Canada

Provinces and territories deliver most health care, so most records of physician visits, hospital stays, prescriptions and insurance registrations are created by provincial and territorial systems and held by ministries of health, health authorities and agencies. National organizations hold two other kinds of data. Statistics Canada, the national statistical agency, conducts the census and a large program of surveys. The Canadian Institute for Health Information (CIHI) receives records from the provinces and territories and organizes them to common national standards so that they can be compared.

Several roles recur in every access process, although organizations name them differently. A data custodian is the organization legally responsible for holding and protecting a data set, such as a ministry of health. A data steward is the person or body that decides, on the custodian’s behalf, whether a request to use the data should be approved. A data platform or data centre prepares data for research, links records from different sources and provides a secure place to analyze them. The researcher requests access, analyzes the data and is accountable for using them as approved. In British Columbia, the Ministry of Health is the custodian of physician billing records, data stewards review requests to use them, and Population Data BC has long been one of the platforms through which researchers request and analyze them.

1.2 Statistics Canada Surveys and Research Data Centres

Statistics Canada collects information under the Statistics Act, which requires it to keep information about individual people confidential. Its surveys use probability samples (Lesson 7) and weights to describe provinces and, for larger surveys, health regions.

Three levels of access

A public use microdata file describes real respondents, but identifying details have been removed or coarsened so that no one can be recognized. Students and researchers can usually analyze a PUMF obtained through a university library, although a file that reports only the province cannot answer a question about one health region.

Research Data Centres and the CRDCN

A research data centre (RDC) is a secure facility where approved researchers analyze confidential Statistics Canada files. The Canadian Research Data Centre Network (CRDCN), a partnership between Statistics Canada and Canadian universities, operates RDCs on more than thirty campuses, including campuses in British Columbia. RDCs hold survey master files, some administrative files such as tax and hospitalization records, and files from the agency’s Social Data Linkage Environment, which links survey, census and administrative records. Statistics Canada has, for example, linked responses from some health surveys to hospital and mortality records for respondents who agreed to linkage.

The researcher submits a proposal explaining why the confidential data are needed. Approved researchers undergo security screening and are sworn in as deemed employees of Statistics Canada, which places them under the confidentiality obligations of the Statistics Act. They work on computers that cannot send files out, and every result they wish to take away is reviewed by Statistics Canada staff to confirm that it cannot reveal information about an individual. The network has introduced a virtual RDC for remote work under similar controls, and fees depend on the institution and type of project, so students should ask local RDC staff about current arrangements.

1.3 The Canadian Institute for Health Information

CIHI is an independent, not-for-profit organization, created in 1994, that collects health system data from the provinces, territories and health care organizations and publishes information about Canada’s health systems. Its value to researchers lies in standardization. Hospitals code diagnoses with the Canadian version of the International Classification of Diseases (ICD-10-CA) and procedures with the Canadian Classification of Health Interventions (CCI), so a hospital stay for pneumonia is recorded the same way in Nova Scotia and British Columbia.

CIHI’s public reports and interactive tools give aggregate results, such as hospitalization rates by province, without any application. Researchers who need custom tables or record-level files submit a data request, which CIHI reviews for privacy and which can involve cost-recovery fees. Record-level CIHI files are de-identified and are not usually linked to a researcher’s own survey. Studies that need that kind of linkage more often use the provincial version of the same records through a provincial data centre, because the province holds the health insurance numbers that make linkage possible.

1.4 Population Data BC and Its Counterparts in Other Provinces

Population Data BC (PopData) is a data and education resource hosted at the University of British Columbia. It has long helped researchers request, link and analyze British Columbia administrative data. These include the Ministry of Health, whose data sets cover physician services billed to the Medical Services Plan, hospital discharges, prescriptions recorded in PharmaNet and registration files, along with sources such as vital statistics, the cancer registry and education records. Researchers submit a data access request through PopData’s Data Access Unit, data stewards review it, and approved researchers analyze the data remotely in a secure environment. PopData can also link data that researchers collected themselves, such as survey responses, to administrative records, provided that participants explicitly agreed to the linkage on a consent form that meets the data providers’ requirements and has been approved by a research ethics board.

Arrangements in British Columbia have been changing. In 2024 the Ministry of Health began accepting academic data requests through the Health Data Platform BC, and since August 2025 new research projects that request Ministry of Health data have been handled under that program through a shared PopData and Health Data Platform BC request process. Analysis is moving from PopData’s Secure Research Environment to a cloud-based Trusted Analysis Environment. Students should check the current process before planning a project; the principles in this lesson apply in either arrangement.

Other provinces and territories have their own arrangements, which also change over time.

Research across provinces

Provincial privacy laws and data agreements usually require record-level health data to stay in the province, so a study comparing provinces cannot simply pool their files. One solution is distributed analysis: the team writes a common protocol and analysis code, analysts in each province run the code on their own data, and only summary results are combined. The Canadian Network for Observational Drug Effect Studies (CNODES) has used this approach to study the safety of medications across several provinces. Health Data Research Network Canada (HDRN Canada) is a non-profit network of provincial, territorial and pan-Canadian data organizations that works to make research across regions easier. Its Data Access Support Hub offers researchers a single entry point and a common request form for data from more than one region, an inventory of available data sets, and an inventory of algorithms (tested definitions of conditions written in terms of administrative codes).

1.5 Cohort Studies, Registries and Indigenous-Governed Data

Some sources were created for research from the start. The Canadian Longitudinal Study on Aging (CLSA) follows more than 50,000 adults who were aged 45 to 85 when recruited and collects information on health, function and social participation over many years; researchers apply through its own access process. Disease registries, such as provincial cancer registries, record every diagnosed case of a condition in a defined population.

Data about First Nations, Inuit and Métis people raise questions of governance that go beyond privacy law. The First Nations Information Governance Centre conducts the First Nations Regional Health Survey and holds it under the OCAP® principles of ownership, control, access and possession described in Lesson 4. Provincial records also include First Nations people, and the use of data that identify them can require approval under agreements with First Nations governance bodies. For the Cedar Valley team, the fictional Cedar Valley First Nations Health Centre takes part in decisions about whether and how data about its members are requested, analyzed and reported.

Section 2 turns from where the data are held to how a researcher obtains permission to use them.

Introduction

Administrative health data cannot be downloaded. A researcher who wants to use them must make a formal application, usually called a data access request, which describes the research, the people and records it needs, and the safeguards that will protect them. The request is reviewed by the people responsible for the data, and access is granted only for the approved purpose. This section describes that process in general terms. Each organization has its own forms and rules, which change over time, so the details of any real application come from the organization’s current guidance and its data access staff.

Planning for access begins well before the request is submitted. The Cedar Valley team (a fictional study) knew from the start that it wanted to link survey responses to health records. It therefore wrote the linkage into its consent form (Lesson 5), asked respondents for their personal health number in the survey (Lesson 8), and contacted Population Data BC while the survey was still being designed. Of the 1,600 adults aged 65 and older who completed the survey, 1,312 (82.0 percent) consented to linkage and provided a personal health number. Those 1,312 people are the population for which the team can request linked records.

2.1 The Stages of a Data Access Request

Processes differ between organizations, but most follow a similar sequence. Figure 2.1 shows eight typical stages. Population Data BC, for example, describes its process in stages that run from planning the request through data steward review, contracts and privacy training, data release, analysis and pre-publication review, to project closure.

In the enquiry and intake stage, the researcher contacts the organization’s data access staff, describes the question and learns which data sets might answer it. In the feasibility and cost stage, the researcher confirms that the data exist for the needed years and population, reads the data set documentation (often called metadata or a data dictionary), and asks for a cost estimate, which funders may want to see in a grant application. The ethics stage obtains approval from a research ethics board (REB). The submission stage completes the request form. In data steward review, the people responsible for each data set decide whether to approve the request, often after asking questions. The agreements and training stage covers the signed research agreement, confidentiality undertakings from every team member, privacy training and the creation of accounts. In the linkage and release stage, the platform links and extracts the records and places them in a secure environment. The final stage covers analysis, output review and closure: results are checked before they leave the environment, some organizations review publications before release, and at the end of the project the data are destroyed or retained as the agreement specifies.

2.2 What a Data Access Request Contains

A data access request is a short research proposal written for a particular audience. Data stewards want to know whether the use is appropriate, whether the amount of data requested is the minimum the question needs, and whether the people and settings involved will keep the data safe. The tabs below describe the usual parts of a request, with the Cedar Valley version of each.

Data minimization

The principle that runs through every part of a request is data minimization: a project should receive only the people, years and variables that its question requires, at the least identifying level of detail that will work. Population Data BC’s guidance puts the idea directly: the selection of data fields in an extract must be restrictive. Minimization reduces the harm that would follow a breach, and it makes approval easier, because every extra variable is another item a data steward must be persuaded to release.

2.3 Who Reviews a Request, and on What Grounds

Several groups review a request, and each looks at it from a different angle.

Data stewards decide whether the use is permitted under the law and agreements that govern each data set and whether it is appropriate. In British Columbia, the Freedom of Information and Protection of Privacy Act sets the conditions under which public bodies may disclose personal information for research, and Ontario’s Personal Health Information Protection Act plays a similar role for ICES. Stewards consider the public benefit of the research, whether the data requested are the minimum needed, whether consent was obtained or its absence is justified, and whether the people and settings involved are secure.

Research ethics boards review the study under the Tri-Council Policy Statement (TCPS 2), whose rules on privacy, secondary use of information and consent Lesson 5 described. TCPS 2 contains a specific article on data linkage (Article 5.7). It requires REB approval before linkage takes place, asks researchers to describe the data to be linked and the likelihood that linkage will create identifiable information, and, where it will, asks researchers to show that the linkage is essential to the research and that appropriate security measures will protect the information. Data providers usually require proof of current ethics approval before they release data, and any change to the request later may require an amendment to both approvals.

Indigenous governance bodies have authority over data about their communities under agreements and under the principles described in Lesson 4. In the Cedar Valley study, the fictional Cedar Valley First Nations Health Centre reviewed the request, agreed with the team on how results about First Nations participants would be reported, and asked to review any such results before release.

The researcher’s institution signs the research agreement, because the university, as well as the researcher, takes on legal obligations for the data.

2.4 Timelines and Costs

Access to linked administrative data takes time. Population Data BC describes the request, approval and data provisioning process as one that can take some months, and requests involving several data providers, new linkages or researcher-collected data tend to take longer. Researchers should treat any study that depends on linked administrative data as a long-term undertaking, and a study that must finish within a few months is rarely feasible unless the data are already available to the team.

Access also costs money. Data platforms commonly charge fees to recover the cost of preparing, linking and extracting data and of providing a secure analysis environment, and the size of those fees depends on the complexity of the request. Statistics Canada RDC access can involve fees that depend on the institution and project, and CIHI custom requests can involve cost-recovery charges. A project budget should also include the time of an analyst who can work with large administrative files, the time all team members need for privacy training, and the cost of amendments. A cost estimate obtained during the feasibility stage allows these items to be included in a grant application.

These allowances are the fictional team’s own planning assumptions. Actual times depend on the request and the organization, and the team confirmed its assumptions with data access staff at the intake meeting. The months are counted from the intake meeting, which the team held in month 3 of the project, so access approval falls in project month 11 and the linked data arrive in month 14, as the Gantt chart in Lesson 6 shows. Lesson 6 also showed how to place such allowances in a project timeline so that analysis and reporting are scheduled realistically.

Once access is approved, the platform must find each consenting respondent in the administrative files. Section 3 describes how that record linkage is done and what happens when it goes wrong.

Introduction

After its request was approved, the Cedar Valley team (a fictional study) needed the platform to find each of its 1,312 consenting respondents in the provincial files. That task is record linkage: bringing together records that belong to the same person (or family, address or organization) from different sources, or from within one source. Linkage allows a survey answer about loneliness to sit beside a record of an emergency department visit, and it is also a source of error that is invisible in the final data set.

3.1 What Record Linkage Is

Halbert Dunn (1946), an American public health statistician, used the term “record linkage” to describe assembling the records of a person’s life, from birth to death, into a single “book of life”. Howard Newcombe and colleagues (1959), working in Canada, showed that a computer could link birth and marriage records automatically by weighing how strongly agreement on names and other details suggested that two records belonged together. Ivan Fellegi and Alan Sunter (1969), statisticians at what is now Statistics Canada, gave probabilistic linkage the mathematical form it still has. Fellegi later served as Chief Statistician of Canada.

A linkage key is the identifier or combination of identifiers used to decide whether two records belong together, such as a health number, or a surname combined with a date of birth. A match is a pair of records that truly belong to the same person. A link is the decision, made by a rule or a reviewer, to treat two records as belonging to the same person. Linkage error is the gap between them.

3.2 Identifiers and Why They Disagree

Health records in British Columbia carry a personal health number (PHN), a lifetime identifier assigned to each person registered for provincial health insurance. It is the strongest linkage key available, because each person has one and no two people should share one. Names, birth dates, sex and postal codes can all disagree between two records of the same person.

These errors are unevenly spread. People who move often, whose names are often misspelled or transliterated, or who have changed their names are more likely to have records that disagree. Older adults who have recently moved to a smaller town, the group at the centre of one Cedar Valley qualitative question, are one example.

3.3 Deterministic Linkage

Deterministic linkage links two records when they agree exactly on a specified linkage key or set of identifiers. The simplest rule uses one high-quality identifier: link two records if their personal health numbers are identical. A stepwise (or multi-pass) deterministic linkage applies a sequence of rules, starting with the strictest; records left unlinked by the first rule are tried against a second, such as agreement on surname, given name, date of birth and sex.

Deterministic linkage is transparent and works very well when a unique identifier is recorded accurately in both files. A single error in the linkage key prevents a link, however, because a strict rule cannot tell one transposed digit from a different person.

3.4 Probabilistic Linkage

Probabilistic linkage compares several identifiers and asks how strongly the pattern of agreement and disagreement supports the conclusion that two records belong to the same person. It is used when no unique identifier is available in both files, or when the identifier is incomplete. Following Fellegi and Sunter (1969), each identifier is described by two probabilities.

Software usually estimates the probabilities from the data and applies blocking, comparing only pairs that agree on a simple variable such as year of birth, over several passes.

Moving the thresholds trades one error for another. Raising the upper threshold to 25 would send pairs 2 and 5 to clerical review, where the reviewer would probably reject the twins, at the cost of more manual review; lowering the thresholds would reduce missed matches and increase false matches.

3.5 Linkage Error and How It Is Measured

A missed match (a false negative) occurs when two records of the same person are not linked. A false match (a false positive) occurs when records of two different people are linked. When the truth is known for a sample of pairs, for example from a careful manual check, two measures summarize performance. Sensitivity is the proportion of true matches that were linked. Positive predictive value is the proportion of links that are true matches.

Six pairs are too few to judge a method, but they show the usual pattern: deterministic rules on a strong identifier tend to produce few false matches and some missed matches, while probabilistic methods recover more true matches at the risk of some false ones. Many linkage units combine the two, with a deterministic pass on the health number followed by probabilistic passes for the records that remain.

3.6 The Consequences of Linkage Error

The consequences of linkage error depend on how unlinked records are handled and on whether errors are spread evenly across the groups being compared (Harron et al., 2017; Doidge & Harron, 2019).

In the actual (fictional) Cedar Valley linkage, the platform linked 1,268 of the 1,312 consenting respondents in a deterministic pass on health number and date of birth and 30 more in probabilistic passes; 5 of 8 pairs sent to clerical review were accepted. In total, 1,303 respondents (99.3 percent) were linked. Five of the 9 unlinked respondents were lonely (1.7 percent of 298) and 4 were not (0.4 percent of 1,014), a small error in the same direction as the example, which the team reported.

3.7 What Researchers Can Do

Section 4 describes how a linked file, which reveals more than any of its sources, is protected.

Introduction

A linked file can reveal more about a person than any of its sources. The Cedar Valley survey (a fictional study) records how lonely a respondent feels; the health records show when that person went to an emergency department and why. Joined together, they form a detailed account of a person’s life that the person shared on the understanding that it would be protected. Lesson 5 introduced privacy and confidentiality under TCPS 2, the categories of identifiable information, and data management plans. This section describes the specific safeguards that make the research use of linked administrative data possible: reducing how identifiable the data are, controlling who uses them and where, and checking every result before it is released.

4.1 How People Are Identified

A direct identifier identifies a person on its own: a name, a personal health number, a street address, a telephone number or an email address. An indirect identifier, also called a quasi-identifier, does not identify anyone on its own but can do so in combination with other information. Date of birth, postal code, sex, ethnicity, occupation, a rare diagnosis and the dates of hospital stays are all indirect identifiers.

The power of combinations is easy to underestimate. Latanya Sweeney (2000), a computer scientist, estimated that about 87 percent of the population of the United States could be uniquely identified by three items: five-digit ZIP code, sex and full date of birth. Removing names from a file therefore does little to protect people if the file keeps detailed dates and locations. TCPS 2 reflects this by distinguishing directly identifying, indirectly identifying, coded, anonymized and anonymous information (Lesson 5), and by treating information as identifiable whenever it could reasonably be expected to identify a person, alone or in combination with other available information.

Risk is higher in small populations. In a city of 90,000 people, a 91-year-old woman who visited an emergency department in March is one of many. In a small community such as the fictional Kestrel Lake in the Cedar Valley region, a neighbour who knows those three facts might recognize her. This is why rural communities, small First Nations communities and people with rare conditions need particular care in the release of results, and why the release rules described below often require small groups to be combined.

4.2 De-identification Techniques

De-identification is the set of techniques that reduce the chance that a person can be identified from a data set, while keeping the data useful for the research question. It reduces risk to a low level without removing it entirely, so it is always combined with the other safeguards in this section. Khaled El Emam (2013), a Canadian researcher in this field, describes de-identification as a process of measuring the risk of re-identification and applying techniques until the risk is acceptably low for the setting in which the data will be used.

The separation principle

Linkage needs identifiers, and analysis needs content, but no single person needs both. The separation principle organizes linkage around that fact. Data providers send identifying information (names, health numbers, birth dates) to a linkage unit, which uses them to work out which records belong together and assigns each person a project-specific study number. The content (loneliness scores, visit dates, diagnoses) travels separately, without names or numbers, and is joined using the study numbers inside a secure environment. Linkage staff therefore see identifiers without health content, and researchers see health content without identifiers. Because the study numbers are specific to one project, files released to different projects cannot be joined to each other.

4.3 The Five Safes Framework

De-identification protects the data themselves. Other safeguards control the project, the people, the setting and the results. The Five Safes framework, developed by Felix Ritchie and colleagues from work on secure data access at the United Kingdom’s Office for National Statistics (Desai et al., 2016), organizes all of these safeguards as five questions. Statistical agencies and data centres in several countries use it to design and explain their access arrangements, and it is a practical tool for planning any study that uses sensitive data. Click each card to read the question it asks.

The five dimensions work as a set. When one is strong, others can be lighter, and when one must be weak, others must compensate. A public use microdata file has very safe data, because detail has been removed, so students and researchers at participating institutions can analyze it on an ordinary computer. A confidential master file in a Research Data Centre has much less safe data, because it keeps detail that researchers need, so access requires screened people, a controlled setting and checked outputs. The table compares three arrangements described in Section 1.

The Five Safes is a planning framework. Privacy law, data agreements, TCPS 2 and Indigenous data governance under principles such as OCAP® apply alongside it, and a First Nations partner may set conditions on projects, people and outputs that go further than any of the five questions require.

4.4 Secure Research Environments and Output Checking

A secure research environment (also called a trusted research environment) is a controlled computing system in which approved researchers analyze sensitive data without being able to remove them. Population Data BC’s Secure Research Environment, the cloud-based Trusted Analysis Environment that is replacing it, the analysis environments at ICES and the virtual Research Data Centre all follow this model, although their details differ. Common features include remote access to a virtual desktop rather than a copy of the data, strong authentication when logging in, no internet, email or copy-and-paste out of the environment, a record of what each user does, and a single controlled route by which files enter and results leave.

That single route is where output checking happens. Before a table, figure or model result leaves the environment, it is checked for disclosure risk, either by the researcher against the platform’s rules, by platform staff, or both. The main risks are listed below.

4.5 Worked Example: The Cedar Valley Five Safes Plan

The Cedar Valley team summarized its safeguards in a Five Safes table that it attached to its data management plan and its data access request. A plan of this kind shows reviewers, partners and participants how each risk is handled.

Bringing It All Together

This lesson followed the path that existing data take from the systems that create them to a research result. In Canada, most administrative health records are created by provincial and territorial health systems, while Statistics Canada holds the census and national surveys and CIHI organizes records from the provinces to common national standards. Researchers reach these data through published tables, public use microdata files, Research Data Centres, CIHI data requests and provincial platforms such as Population Data BC and ICES, and networks such as HDRN Canada support research that crosses provincial borders.

Access depends on a data access request that defines the study population precisely, justifies each variable, explains the linkage and consent, names the people involved and describes the secure setting. Data stewards, research ethics boards and Indigenous governance bodies each review the request, and the process commonly takes months and carries costs that belong in the project budget. Once approved, records are joined by deterministic rules, probabilistic weights or both. Both methods make errors, and missed matches and false matches can bias a study’s results when they fall unevenly on the groups being compared.

The linked file is protected by layers of safeguards. De-identification and the separation principle reduce what any one person can see, and the Five Safes framework asks whether the project, the people, the setting, the data and the outputs are each safe enough. Secure research environments and output checking keep record-level data inside a controlled system and ensure that released results cannot identify anyone, which matters most in small communities such as those in the fictional Cedar Valley region.

The final reflection asks you to plan the data sources, access, linkage and safeguards for a new study from start to finish.

Final Knowledge Assessment

These terms, organizations and people appear in this lesson on Canadian data sources, record linkage and privacy safeguards.