Why the population question comes first

Every quantitative study makes a claim about a group of people. A survey reports that a quarter of older adults in a region are lonely, and a cohort study reports that lonely people visit the emergency department more often. Each claim is only as good as the answer to an earlier question: which people does the finding describe, and how did the people who supplied the data come to be in the study? Sections 1 and 2 of this lesson answer that question in practical terms, covering populations, sampling frames, sampling methods and recruitment. Sections 3 and 4 turn to measurement: deciding exactly what will be recorded about each participant and choosing the instruments that record it.

A population is the full group of people (or clinics, households, charts or other units) that a study is about. A sample is the subset of the population from which data are actually collected. Researchers study samples because it is rarely possible or necessary to collect data from everyone, and a well-chosen sample of a few thousand people can describe a population of millions. The unit that is selected is called the sampling unit. It is often a person, although a study might select households, clinics or medical charts instead. When a study selects clinics first and then patients within those clinics, there are two sampling units, one at each stage.

Two populations and a sample: target, source and study sample

Research methods textbooks distinguish several populations that sit inside one another. The terms vary somewhat between disciplines, so this course uses the three definitions below and asks you to state your own definitions explicitly whenever you write a protocol.

The target population

The target population is the group to which the researchers want their findings to apply. It is defined by the research question. For the Cedar Valley survey, the PECO question from Lesson 2 asks about adults aged 65 and older in the region, so the target population is all adults aged 65 and older who live in the Cedar Valley Health Authority region, about 46,000 people. A target population is usually defined by person (age 65 and older), place (living in the region) and time (during the study period, 2026 to 2027 in this illustration).

The source population

The source population is the part of the target population that the study can actually reach and select from. In practice it is the set of people who appear on the list or are covered by the procedure that the team uses to select participants. If the Cedar Valley team selects people from the patient lists of primary care clinics, then older adults who have no regular clinic are part of the target population but are outside the source population. Some textbooks call this the accessible population or the sampled population, and HSCI 230 and HSCI 341 use "study population" as another name for the source population.

The study sample (participants)

The study sample, or the participants, is the group of people who are selected, are found to be eligible, agree to take part and provide data. It is the group that appears in the results tables. In the Cedar Valley survey, 5,250 people are invited and 1,600 complete the survey, and those 1,600 respondents form the study sample for the survey analyses. Section 2 shows how a participant flow diagram accounts for every person between the invitation and the analysis.

Eligibility criteria

Eligibility criteria are the rules that decide who may take part. Inclusion criteria describe the characteristics a person must have to be part of the study, and exclusion criteria describe characteristics that remove an otherwise eligible person. Good criteria follow directly from the research question, are written precisely enough that two research assistants would make the same decision about the same person, and come with a reason. Each criterion should be paired with the screening question or record that will confirm it (Section 2 covers screening).

The Tri-Council Policy Statement (TCPS 2), which you studied in Lesson 5, asks researchers not to exclude people on the basis of characteristics such as age, language, culture or disability unless there is a valid reason related to the research question or to participant safety (Chapter 4). Exclusions made for convenience, such as leaving out everyone who does not read English, can remove exactly the people whose experience the study needs. The Cedar Valley advisory group of six older adults pointed out that many lonely older adults in the region have low vision or limited English, so the team added a paper version of the self-completed survey and a telephone help line that answers questions about it, with interpreters available in the two languages other than English most often spoken by older adults in the region.

Sampling frames

A sampling frame is the list, map or procedure that identifies the members of the source population so that some of them can be selected. Everyone on a list frame has a chance of selection and nobody missing from it has any chance, which is why the frame defines the source population. The choice of frame is usually constrained by what lists exist, who holds them, and whether the holder may share them or contact people on the researchers' behalf.

Common problems with frames

No frame matches the target population perfectly. The Cedar Valley team checked each candidate frame against five common problems. Leslie Kish (1965) described the first four in his textbook on survey sampling, and the fifth follows from the way lists are kept.

Choosing a frame for the Cedar Valley survey

The team considered four candidate frames. The table summarizes the strengths and gaps that the team and its advisory group identified.

Two features of this decision deserve attention. First, the clinics may not give patient names and addresses to researchers without consent, so each clinic mailed the invitations to the people selected from its list, using study identification numbers, and the researchers learned names only when a person chose to respond. Second, the Cedar Valley First Nations Health Centre decided how its own clients would be approached. Under the engagement agreement from Lesson 4 and the principles of TCPS 2 Chapter 9, the Health Centre included its client list in the frame, mailed the invitations itself with a letter from its health director, and kept a say over how results about its community would be reported.

The people who are hardest to reach

Studies of loneliness face a particular difficulty. The most isolated people are the least likely to be on membership lists, to have a regular clinic, or to answer an invitation from a stranger, so a frame or a response process that misses them makes a region look more connected than it is. The Cedar Valley team accepted the clinic frame as the best available option, recorded the gap of about 6,000 older adults as a known limitation, and planned two partial remedies: recruiting some interviewees through community connectors and outreach workers who know isolated residents (Section 2), and comparing respondents with census figures for the region on age, sex and community. Neither remedy closes the gap, and the report in Lesson 12 will need to say so.

What makes a sample a probability sample

Sampling methods fall into two families. In a probability sample, every member of the source population has a known chance of selection greater than zero, and a chance mechanism such as a random number generator decides who is selected. In a non-probability sample, people enter the study through the researcher's judgement, their own decision to volunteer or their availability, so the chance that any particular person is included cannot be stated.

The everyday meaning of the word "random" causes much of the confusion here. In conversation, "random" means haphazard, as in "I asked some random people at the mall". In research, a random sample is a planned procedure in which chance, applied to a defined frame, does the choosing. A useful test is to ask whether, before selection began, you could have written down the probability that a named person on the frame would be chosen. In the Cedar Valley survey, each person on the Cedar City part of the frame had a one in ten chance, and each person elsewhere had a three in twenty chance. Nobody can state the chance that a given resident had of walking past a table at the mall, so a sample recruited there is a convenience sample.

Probability samples allow researchers to estimate how far a sample result, such as the percentage of respondents who are lonely, is likely to be from the value in the source population. Jerzy Neyman (1934) set out the argument for random selection in surveys. The theory, including sampling error, weighting and sample size calculation, belongs to HSCI 341 Lesson 2, and this lesson concentrates on recognizing each method and carrying it out.

Four probability sampling methods

Non-probability samples

Non-probability samples are common and often appropriate, for example in qualitative studies, pilot studies and studies of populations with no list. Their limitation is specific: by itself, a non-probability sample cannot support a statistical estimate of how common something is in a population.

The Cedar Valley qualitative strand uses non-probability sampling deliberately. The 24 interviews with older adults who live alone use purposive sampling (also called judgement sampling) in its maximum variation form. Most interviewees are survey respondents who live alone and agreed to be contacted, chosen to include Cedar City and each smaller community, women and men, recent and earlier movers, and people who do and do not drive. Because the survey frame misses older adults without a regular clinic, community connectors and outreach workers also pass the invitation to isolated residents they know. The four focus groups (two with older adults, one with family caregivers, and one with clinic staff and community connectors) are also purposive.

A survey link on social media is a convenience sample

Dr. Hart asked the assistant to apply the known-chance test. A person reaches the survey through a post only if the person uses that platform, follows or is connected to the pages where the link appears, is shown the post by the platform's algorithm, decides to click and decides to finish. None of these probabilities is known or set by the researchers, and all of them depend on the person's age, health, digital skills, social network and interest in the topic. A survey about loneliness spread by sharing among friends would reach people who have friends to share it with. The result is a convenience sample (more precisely, a volunteer or self-selected sample), and it remains one whether 50 or 50,000 people respond.

Social media still has legitimate uses in research, such as recruiting for a pilot study, reaching a group with no list, or recruiting interviewees for a purposive sample. The requirement is honest labelling: a methods section should name the platforms, call the sample a convenience sample, and present its percentages as descriptions of the respondents. Open links also attract fraudulent responses, which Lesson 8 addresses.

Random sampling and random allocation are different procedures

A second confusion involves two procedures that share the word "random". Random sampling decides who enters the study, and it supports generalization from the sample to the source population. Random allocation (also called randomization or random assignment) uses chance to decide which group each person already in a study joins, such as an intervention group or a control group. It makes the groups comparable at the start, so that differences in outcomes can be attributed to the intervention. Ronald Fisher (1935) set out the case for random allocation in agricultural experiments. A study can use either procedure, both or neither.

Suppose the Cedar Valley team later pilots a telephone befriending service. Clinic staff invite older adults who score 6 or higher on the loneliness scale, and the 80 people who agree are randomly allocated, 40 to weekly calls and 40 to usual care. This is a randomized trial built on a convenience sample. Random allocation makes the two groups similar at the start, so a difference in loneliness after six months can be credited to the calls. Because the 80 volunteers probably differ from lonely older adults in general, the team should be cautious about how far the result applies. In the student survey mentioned above, 35 percent chose "to ensure the sample represents the population" as the purpose of randomization in a trial. That answer describes random sampling. The purpose of random allocation is comparable groups.

Recruitment

Recruitment is the set of activities that contact selected or eligible people, explain the study and invite them to take part, using channels such as mailed letters, telephone calls, email, clinicians and other gatekeepers, posters, community organizations, registries, social media and referrals. Three principles apply. First, a probability sample must recruit the people who were selected, because replacing a non-responder with a neighbour who is easier to reach turns it back into a convenience sample. Second, several contacts by more than one mode raise response. Following the multiple-contact approach of Dillman, Smyth and Christian (2014), the Cedar Valley team sent a pre-notice letter, an invitation with a web link and an access code a week later, a reminder postcard a week after that, a replacement paper questionnaire to non-responders two weeks later, and a final contact two weeks after that (Lesson 8 covers reminders and incentives). Third, every recruitment letter and script is part of the ethics application (Lesson 5) and should avoid undue influence, which is why the Cedar Valley letters stated that a decision not to take part would not affect a person's care.

Eligibility screening

Screening confirms that each respondent meets the eligibility criteria from Section 1. The first page of the Cedar Valley survey asked for age in years, whether the person lives in the region, and whether the person lives in a private home or in a long-term care home. Ineligible respondents saw a thank-you message, and the reason was recorded. Screening should collect only what is needed to decide eligibility, because the person has not yet consented to the main study.

The recruitment log

A recruitment log records each contact with each selected person and its outcome. It supplies the numbers for the participant flow diagram and the response rate. It uses study identification numbers only, and the key linking numbers to names stays with the clinics.

The participant flow diagram

A participant flow diagram shows how many people were at each stage of a study and how many were lost at each step and why. Reporting guidelines recommend one: the CONSORT 2025 statement for randomized trials (Hopewell et al., 2025) and the STROBE statement for observational studies (von Elm et al., 2007), which Lesson 12 introduces. The diagram below is built directly from the Cedar Valley recruitment log.

Each step should balance: 5,250 selected minus 210 not delivered gives 5,040, and 5,040 minus 3,240 non-responders gives 1,800. Numbers that do not add up point to gaps in the recruitment log.

From a concept to a number

Sections 1 and 2 decided who will be in the Cedar Valley study. Sections 3 and 4 decide what will be recorded about them. Measurement is the assignment of numbers or categories to people (or to clinics, charts or other units) according to explicit rules. Every column in a research dataset is the end product of a chain of decisions, and a reader can judge a finding only if each link in the chain is written down. The chain has five links. A concept (or construct) is the idea the study is about, such as loneliness. A conceptual definition states in words what the concept means. An operational definition states the exact procedure that will be used to observe or record it. The procedure produces a variable, which is a characteristic that takes different values for different people, and each person receives a value on that variable.

Conceptual definitions

A conceptual definition says what a concept means, in words, and separates it from neighbouring concepts. It usually comes from the theory and literature behind the study, and it should be cited. For loneliness, many researchers use the definition given by Perlman and Peplau (1981), who described loneliness as the unpleasant experience that arises when a person's network of social relationships falls short, in quantity or in quality, of what the person wants. Three features of this definition matter for measurement. Loneliness is subjective (it is a feeling), it is unpleasant, and it depends on a comparison between the relationships a person has and the relationships the person wants.

That definition immediately separates loneliness from social isolation, which is usually defined as an objective state of having few social relationships, few social roles or infrequent contact with others. The two concepts are related, and they are frequently confused in news stories and even in research papers. The Cedar Valley research question names loneliness as the exposure, so the team needs a measure of loneliness. If it also wants to describe social isolation, it needs a second measure.

Operational definitions

An operational definition turns the conceptual definition into a procedure that two different researchers could follow and obtain the same kind of data. It should be specific enough to program into a survey platform or an abstraction form without further questions. The accordion lists the elements a complete operational definition contains.

The table shows operational definitions for six variables in the Cedar Valley survey and linked data.

Two lessons follow from the table. First, an operational definition always narrows the concept. The three UCLA items capture how often a person feels a lack of companionship, left out and isolated from others, and they do not capture every aspect of the experience that Perlman and Peplau described. Second, different operational definitions of the same concept can disagree. The team could have asked respondents how many times they went to an emergency department in the past year. Self-report depends on memory and on whether the respondent counts an urgent care centre as an emergency department, while records capture only visits to facilities whose data reach the data holder. Lesson 8 compares these data sources in detail. The protocol should explain which definition was chosen and why.

Levels of measurement

Once a variable exists, its values carry a certain amount of information. The psychologist S. S. Stevens (1946) proposed four levels of measurement that describe how much information a variable's numbers carry and, as a result, which summaries make sense. The levels are cumulative: each one has all the properties of the level before it and adds one more.

Nominal and ordinal variables are often grouped together as categorical variables, and interval and ratio variables as numeric (or quantitative) variables. Numeric variables can be discrete, taking only whole-number values such as counts of visits, or continuous, taking any value within a range, such as weight. In practice, the difference between the interval level and the ratio level rarely changes which summary a student would choose in this course. The important line is the one between categorical and numeric variables, which decides whether Lesson 11 summarizes a variable with a frequency table or with a mean and standard deviation.

Collect at the highest sensible level

A variable can always be moved down the levels after data collection. Age in years can be grouped into 65 to 74, 75 to 84 and 85 and older, and the UCLA score can be split at 6. A variable cannot be moved up: if the survey records only age groups, nobody can later calculate a mean age. The general advice is therefore to collect each variable at the highest level that is feasible and ethical, and to create categories during analysis. HSCI 341 Lesson 3 Section 4 ("From Response Options to Analysis Variables") shows how several analysis variables are derived from one item, and HSCI 410 Lesson 7 Section 1 treats published cut-points.

Collapsing has costs as well as uses. In the Cedar Valley survey, 392 of the 1,600 respondents (24.5 percent) scored 6 or higher and were classed as lonely. The binary variable is easy to report and to compare with other studies that use the same cut-off. It also treats a respondent who scored 5 the same as one who scored 3, and treats respondents who scored 5 and 6 as different kinds of people, although their answers differ on a single item. The team therefore stores the item responses and the total score and creates the binary variable from them, so that both forms are available.

Two considerations can justify collecting less detail. The first is privacy. A full date of birth combined with a postal code can identify a person in a small community such as Kestrel Lake, so the Cedar Valley survey asks for age in years and never for a full date of birth, in keeping with the data minimization principle in Lesson 5. The second is respondent willingness. People are often more willing to report household income in ranges than as an exact amount, and a question that many people skip collects less information than a slightly coarser question that most people answer.

What a validated instrument is

An instrument is a fixed set of questions or items, with instructions, response options and scoring rules, that produces a measurement of a concept. Health researchers also call instruments measures, scales, questionnaires or tools. A validated instrument is one whose developers and later users have published evidence about how it performs, including evidence of its reliability (whether it gives consistent results) and its validity (whether it measures the concept it is meant to measure). Section 3 introduced these two words. HSCI 410 Lesson 7 (Sections 2 and 3) teaches how the evidence is gathered and judged, and this section uses the words only to ask whether such evidence exists for the people in your study.

The word "validated" is a convenient shorthand that can mislead. Evidence about an instrument comes from particular groups of people, in particular languages and settings, completing it in particular ways. A loneliness scale tested with university students in the United States has evidence for that group, and the evidence may or may not carry over to adults aged 80 in rural British Columbia completing the scale by telephone. When choosing an instrument, the useful question is whether there is published evidence for its use in a population and setting like yours.

Most studies measure their main concepts with existing instruments, for four practical reasons. Existing instruments allow results to be compared with other studies and with national surveys. They come with evidence that a new set of questions would lack. They save the months of development and testing that a new instrument needs. They also come with scoring rules and, often, published cut-offs. Writing new questionnaire items is a skill taught in HSCI 341 Lesson 3, and a second-year research plan should normally use existing instruments for its main concepts. Some concepts do not need a multi-item instrument at all. Age, living arrangement and community of residence are observed directly with a single well-worded question, and multi-item scales are reserved for concepts that cannot be observed directly, such as loneliness, depression or health-related quality of life.

Where to find instruments

Selecting an instrument

Finding candidates is the easy part. Choosing among them means checking each one against the needs of the study. The table lists the criteria the Cedar Valley team used, written as questions.

The team chose the three-item UCLA scale as its primary measure, mainly because of its brevity, its development for large surveys of older adults, including telephone surveys, and its wide use. The Office for National Statistics in the United Kingdom recommended in 2018 that national surveys measure loneliness with these three UCLA items together with a direct question, and the team added the direct question as a secondary item so that both forms would be available for comparison. Two review steps followed. The advisory group of six older adults read the candidate items aloud to one another and confirmed that the three-item wording was clear and the response options easy to use on paper and on screen. The Cedar Valley First Nations Health Centre reviewed the full survey and asked that the invitation letter explain why the questions were being asked and how the answers would be used. The Health Centre also asked that the interviews explore what connection means to participants in their own terms, a request that the qualitative strand took up.

Using an instrument as published

Evidence about an instrument applies to the instrument as it was tested. Changes that seem small, such as rewording an item, changing the response options, dropping an item or altering the recall period, can change how people answer, and the published evidence may then no longer apply. The accordion covers the main rules.

Describing and citing an instrument

A methods section describes each instrument so that a reader can understand exactly what was measured. A complete description names the instrument and version, cites the development paper, states what it measures, gives the number of items with an example item, gives the response options and scoring, states the possible range and its direction, gives any cut-off with its source, and cites evidence from a population like the study's. When students later learn to assess reliability in their own data (HSCI 410), they add that result. The paragraph below shows the pattern for the Cedar Valley survey.

In APA 7 style, an instrument is cited through the paper that developed or described it, and the reference list gives the full entry. Lesson 12 covers APA 7 formatting in detail. The two entries for the paragraph above are as follows.

Worked example: the Cedar Valley measurement plan

A measurement plan brings Sections 3 and 4 together in one table, with a row for each main concept. It becomes the starting point for the survey build in Lesson 8 and the data dictionary in Lesson 11. The Cedar Valley plan for its exposure, outcome and one further concept is shown below.

The plan connects each measurement decision to the research question and to its source, and it makes gaps visible. A row whose source column is empty, or whose conceptual definition does not match the instrument, signals work still to be done before the survey is built.

Bringing It All Together

This lesson followed the Cedar Valley Social Connection Study through two decisions that every quantitative study must make: who will supply the data, and what exactly will be recorded about them. The first half moved from the target population of about 46,000 older adults, through a sampling frame of 40,000 people on clinic and Health Centre lists, to a stratified random sample of 5,250 and a study sample of 1,600 respondents, with a participant flow diagram accounting for every person along the way. It treated two common confusions directly: a survey link posted on social media produces a convenience sample whatever its size, and random sampling and random allocation are different procedures that contribute different things to a study.

The second half moved from concepts to numbers. Conceptual definitions separate loneliness from social isolation, operational definitions state exactly how each value is produced, and levels of measurement decide which summaries make sense. Choosing existing instruments with published evidence, using them as published and citing them fully makes the measurements comparable with other studies and open to scrutiny. Together these decisions form a study's sampling, recruitment and measurement plan.

The final reflection asks you to plan the sampling and measurement for a new prevalence study, drawing on all four sections.

Final Knowledge Assessment

These terms, tools and people appear in this lesson on sampling, recruitment and measurement.