Appendix 1: Quantitative Methodology – Public Opinion Research Study on Electoral Matters - Wave 3

Printable PDF version

Quantitative research was conducted through online surveys using Computer Aided Web Interviewing (CAWI) technology.

As a CRIC Member, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research?Series D?Quantitative Research.

Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix A2.

Sampling Procedure

Computer Aided Web Interviewing (CAWI)

Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,505 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo's online panel.

Léger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled on different sociodemographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.

Data Collection

Fieldwork for the survey was conducted from March 1 to 7, 2023. The participation rate for the survey was 15%. A pretest of 89 interviews was completed on March 1, 2023.

To achieve data reliability in all subgroups, a total sample of 2,505 Canadians who are eligible voters were surveyed, in all regions of the country.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.

Based on data from Statistics Canada's 2021 national census, Léger weighted the results of this survey by age, gender within each region of the country.

The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.

Participation rate

The overall participation rate for this study is 12.4%.

Below is the calculation of the web survey's participation rate. The overall response rate for this study is 16%. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.

Typical participation rates for web surveys are between 20% and 30%. A response rate of 12.4% may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate. The participation rate is similar to that of the first wave of the study that took place in 2021.

Unweighted and weighted samples

A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions, including the Atlantic region and the Prairies, in order to have a sufficient sample in these regions. Therefore, the weighting minimized the weight of these regions that had been voluntarily inflated and slightly increased the weight of Quebec and Ontario.

The following tables present the demographic distribution of respondents according to gender and age.

First, regarding gender, we can see that weighting has adjusted slightly the proportions of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.

*This total is slightly smaller than the total of 2,505 respondents, with the discrepancy representing responses of "other" and "refusal".

Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.

There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study. The relatively small weight factors (see section below) and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.

The following tables present the weighting factors applied to the database according to the different respondent profiles.

• Previous
• Table of Contents
• Next