Appendix 1: Quantitative Methodology – Public Opinion Research Study on Electoral Matters - Wave 4

Printable PDF version

A.1 Quantitative methodology

Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.

As a Canadian Research Insights Council (CRIC) member, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research–Series D–Quantitative Research.

Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix A2.

A.1.1 Sampling Procedure

Computer Aided Web Interviewing (CAWI)

Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,501 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo's online panel.

Léger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled on different sociodemographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.

A.1.2 Data Collection

Fieldwork for the survey was conducted from March 12 to 19, 2024. The participation rate for the survey was 15%. A pretest of 51 interviews was completed on March 12, 2024.

To achieve data reliability in all subgroups, a total sample of 2,501 Canadians who are eligible voters were surveyed, in all regions of the country.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.

Based on data from Statistics Canada's 2021 national census, Léger weighted the results of this survey by age and gender within each region of the country.

The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.

Table A.1 Regional distribution of respondents
Region Number of respondents
Atlantic 182
Quebec 581
Ontario 964
Manitoba/Saskatchewan 157
Alberta 275
British Columbia + Territories 342
Total 2,501

A.1.3 Participation Rate

The overall participation rate for this study is 11.3%.

Below is the calculation of the web survey's participation rate. The overall response rate for this study is 11.3%. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.

Table A.2 Participation rate calculation
Invalid cases 193
Invitations mistakenly sent to people who did not qualify for the study 193
Incomplete or missing email addresses -
Unresolved (U) 19,738
Email invitations bounce back 19
Email invitations unanswered 19,719
In-scope non-responding units (IS) 120
Non-response from eligible respondents -
Respondent refusals -
Language problem -
Selected respondent not available (illness; leave of absence; vacation; other) -
Early breakoffs 120
Responding units (R) 2,540
Surveys disqualified – quota filled 31
Completed surveys disqualified for other reasons 8
Completed interviews 2,501
POTENTIALLY ELIGIBLE (U+IS+R) 22,398
Participation rate 11.3%

The typical participation rates for a web survey are between 20% and 30%. A response rate of 11.3% may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate. The participation rate is similar to that of the last wave of the study that took place in 2023.

A.1.4 Unweighted and Weighted Samples

A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions including the Atlantic region and the Prairies in order to have a sufficient sample in these regions. Therefore, the weighting minimized the weight of these regions that had been voluntarily inflated and slightly increased the weight of Quebec and Ontario.

Table A.3 Unweighted and weighted sample distribution by province
Region Unweighted Weighted
Atlantic 182 168
Quebec 581 576
Ontario 964 965
Manitoba/Saskatchewan 157 160
Alberta 275 277
British Columbia + Territories 342 355
Total 2,501 2,501

The following tables present the demographic distribution of respondents, according to gender and age.

First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.

Table A.4 Unweighted and weighted sample distribution by gender
Gender Unweighted Weighted
Male 1,222 1,216
Female 1,267 1,272
Total 2,501 2,501

* Responses not displayed in the table correspond to "other" and "refusal."

Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.

Table A.5 Unweighted and weighted sample distribution by age group
Age Unweighted Weighted
Between 18 and 34 662 668
Between 35 and 55 797 805
55 and over 1,042 1,028
Total 2,501 2,501

There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study. The relatively small weight factors (see section below) and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.

The following tables present the weighting factors applied to the database according to the different respondent profiles.

Table A.6 Weight factors by profile
Gender Province Age Weight
Men British Columbia + Territories 18–24 0.7036
25–34 1.2050
35–44 1.1420
45–54 1.0586
55–64 1.1747
65+ 1.6115
Alberta 18–24 0.6049
25–34 1.0033
35–44 1.0819
45–54 0.9115
55–64 0.8983
65+ 0.9914
Manitoba/Saskatchewan 18–24 0.3823
25–34 0.5589
35–44 0.5456
45–54 0.4782
55–64 0.5303
65+ 0.6584
Ontario 18–24 2.1192
25–34 3.3074
35–44 2.9929
45–54 2.9760
55–64 3.2772
65+ 4.0270
Quebec 18–24 1.0850
25–34 1.7944
35–44 1.8835
45–54 1.7539
55–64 2.0655
65+ 2.6936
Atlantic region 18–24 0.3232
25–34 0.4643
35–44 0.4638
45–54 0.5166
55–64 0.6304
65+ 0.8496
Women British Columbia + Territories 18–24 0.6627
25–34 1.1949
35–44 1.1755
45–54 1.1361
55–64 1.2613
65+ 1.8419
Alberta 18–24 0.5662
25–34 1.0078
35–44 1.0900
45–54 0.9042
55–64 0.9135
65+ 1.1221
Manitoba/Saskatchewan 18–24 0.3501
25–34 0.5518
35–44 0.5532
45–54 0.4834
55–64 0.5437
65+ 0.7759
Ontario 18–24 1.9659
25–34 3.2647
35–44 3.1814
45–54 3.1905
55–64 3.4634
65+ 4.8329
Quebec 18–24 1.0393
25–34 1.7768
35–44 1.8885
45–54 1.7373
55–64 2.1039
65+ 3.1964
Atlantic region 18–24 0.2995
25–34 0.4650
35–44 0.4964
45–54 0.5484
55–64 0.6678
65+ 0.9837

Table A.7 Weight factors by province and territory
Label Weight
British Columbia13,8783
Alberta11,0950
Saskatchewan2,9165
Manitoba3,4955
Ontario38,5984
Quebec23,0182
New Brunswick2,1532
Nova Scotia2,6997
Prince Edward Island0,4218
Newfoundland1,4340
Northwest Territories0,1044
Yukon + Nunavut0,1850