2023-24 IRCC Online Tracking Surveys
Methodology report
Prepared for: Immigration, Refugees and Citizenship Canada
Supplier: Leger Marketing Inc.
Contract Number: CW2325246
Contract Value: $49,991.20 (including HST)
Award Date: 2023-08-08
Delivery Date: 2024-07-12
Registration Number: POR 031-23
For more information on this report, please contact IRCC at:
IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca
Ce rapport est aussi disponible en français.
Copyright
This public opinion research report presents the methodology used for the online surveys conducted by Leger Marketing Inc. on behalf of Immigration, Refugees, and Citizenship Canada. The first survey was conducted with 2,279 Canadians between November 13th and November 27th, 2023; the second survey was conducted with 2,253 Canadians between June 10th and June 23rd, 2024.
Cette publication est aussi disponible en français sous le titre : 2023-24 Sondages de suivi en ligne d'IRCC.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Immigration, Refugees and Citizenship Canada. For more information on this report, please contact Immigration, Refugees and Citizenship Canada at IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca or at:
Communications Branch
Immigration, Refugees and Citizenship Canada
Jean Edmonds Tower South
365 Laurier Ave W
Ottawa ON K1A 1L1
Catalogue Number: Ci4-245/2024E-PDF
International Standard Book Number (ISBN): 978-0-660-72814-8
Related publications (registration number: POR 031-23):
Catalogue Number: Ci4-245/2024F-PDF (Final Report, French)
ISBN: 978-0-660-72815-5
Copyrights- His Majesty the King in Right of Canada, as represented by the Minister of Immigration, Refugees and Citizenship Canada, 2024
Executive Summary
Leger is pleased to present Immigration, Refugees and Citizenship Canada with this public opinion research methodology report. This report was prepared by Léger Marketing Inc. who was contracted by Immigration, Refugees and Citizenship Canada (contract number CW2325246, awarded August 8th, 2023, for a total value of $49,991.20 including HST).
Background and Objectives
IRCC conducts an ongoing research program to help the Department develop a better understanding of Canadian attitudes toward the issues surrounding citizenship and immigration.
IRCC has been conducting annual telephone tracking surveys since 1996. In 2018, IRCC started running complementary online tracking surveys, which has allowed IRCC to compare Canadians’ responses to a number of key questions in telephone and online surveys. There are significant response differences by mode, so maintaining ongoing tracking both by telephone and online allows the department to:
- compare and contrast Canadians’ attitudes expressed in telephone and online surveys;
- validate trends seen in each mode against each other;
- compare third-party survey findings collected using either mode against known IRCC benchmarks; and
- leverage lower cost online surveys to conduct a broader range of survey research.
The 2023-24 IRCC Online Tracking Surveys project is comprised of two online panel survey waves with the Canadian general public. It will provide IRCC with comparable tracking data about public attitudes toward immigration, which is of key importance to IRCC’s policies, programs and services.
The research objectives of this study include assessing Canadians’ perceptions of:
- immigration levels;
- the impact of immigration on Canada;
- Canada’s immigration system and priorities; and
- the settlement and integration of newcomers.
Methodology
This public opinion research was conducted via online surveys, using Computer Aided Web Interviewing (CAWI) technology. The questionnaires were provided by IRCC. Leger Marketing Inc. used the questionnaires without making any modifications to its format nor to its content.
Wave 1: Fieldwork for the survey was carried out from November 13th to November 27th, 2023. A pre-test of 46 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. A total of 2,279 Canadians with demographic characteristics reflective of the Canadian population were surveyed. An average of 13 minutes was required for respondents to complete the survey. The sample was drawn randomly from the Leo panel and the overall participation rate for the survey was 10.28%.
Wave 2: Fieldwork for the survey was carried out from June 10th to June 23rd, 2024. A pre-test of 55 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. A total of 2,253 Canadians with demographic characteristics reflective of the Canadian population were surveyed. An average of 14 minutes was required for respondents to complete the survey. The sample was drawn randomly from the Leo panel and the overall participation rate for the survey was 10.87%.
A complete methodological description is provided in the Appendices section of this document.
Notes on Interpretation of the Research Findings
This report was compiled by Leger based on the research conducted specifically for this project. This research is not probabilistic; the results cannot be inferred to the general population of Canada.
Since a sample drawn from an Internet panel is not probabilistic in nature, margins of sampling error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population.
Political Neutrality Statement and Contact Information
Leger certifies that the final deliverables fully comply with the Government of Canada’s political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed:
Christian Bourque, Senior Researcher
Léger
Appendix 1: Quantitative Methodology (Wave 1)
As a Canadian Research Insights Council (CRIC) Member, Leger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research— Series A – Fieldwork and Data Tabulation for Online Surveys.
Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Leger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.
Sampling Procedure
Leger conducted a panel-based Internet survey with a sample of adult Canadians, using Computer Aided Web Interviewing (CAWI) technology. A total of 2,279 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo’s online panel.
Leger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of Web users profiled on different sociodemographic variables. The majority of Leger’s panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.
Data Collection
Fieldwork for the survey was carried out from November 13th to November 27th, 2023. A pre-test of 46 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. A total of 2,279 Canadians with demographic characteristics reflective of the Canadian population were surveyed. An average of 13 minutes was required for respondents to complete the survey. The sample was drawn randomly from the Leo panel and the overall participation rate for the survey was 10.28%.
Since a sample drawn from an Internet panel is not probabilistic in nature, a margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.
During data collection, regional quotas were set, and soft quotas were set in order to ensure a sufficient subsample size for Immigrants and Indigenous Peoples. The tables below detail the distribution of respondents by region, gender, age, place of birth, and Indigenous self-identification.
| Number of respondents (unweighted) | |
|---|---|
| British Columbia | 282 |
| Alberta | 241 |
| Saskatchewan | 121 |
| Manitoba | 130 |
| Ontario | 777 |
| Quebec | 473 |
| New Brunswick | 80 |
| Nova Scotia | 100 |
| Prince Edward Island | 13 |
| Newfoundland | 62 |
| Number of respondents (unweighted) | |
|---|---|
| Woman | 1,163 |
| Man | 1,104 |
| Another gender | 9 |
| Prefer not to say | 3 |
| Number of respondents (unweighted) | |
|---|---|
| 18-24 | 229 |
| 25-34 | 372 |
| 35-44 | 384 |
| 45-54 | 358 |
| 55-64 | 402 |
| 65+ | 534 |
| Number of respondents (unweighted) | |
|---|---|
| Canada | 1,778 |
| Outside Canada | 495 |
| Prefer not to say | 6 |
| Number of respondents (unweighted) | |
|---|---|
| Indigenous | 113 |
| Non-Indigenous | 2,150 |
| Prefer not to say | 16 |
Participation Rate
The overall participation rate for this study is 10.28%.
The participation rate is calculated as the number of in-scope responding units (n=2,383), divided by the sum of unresolved units, in-scope non-responding units, and in-scope responding units (n=23,168). The detailed completion results are as follows:
- Total number of sample units selected to participate: 23,516
- Number of invalid cases: 348
- Invitations mistakenly sent to people who did not qualify for the study: 11
- Incomplete or missing email addresses: 337
- Number of unresolved units: 20,405
- Email invitations bounce back: 30
- Email invitations unanswered: 20,375
- Number of in-scope non-responding units: 380
- Non-response from eligible respondents: 0
- Respondent refusals: 199
- Language problem: 0
- Selected respondent not available (illness; leave of absence; vacation; other): 0
- Early breakoffs: 181
- Number of in-scope responding units: 2,383
- Surveys disqualified – quota filled: 80
- Completed surveys disqualified for other reasons: 24
- Completed interviews: 2,279
Typical participation rates for web surveys are between 20% and 30%. A participation rate of 10.28% may seem a bit low, but due to time constraints, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate.
Unweighted and Weighted Samples
Based on data from Statistics Canada’s 2021 national census, Leger weighted the results of this survey by gender and age within each province, in order to reflect the actual distribution of the Canadian adult population.
A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower participation rates among specific demographic subgroups (see tables below).
The table below presents the geographic distribution of respondents, before and after weighting. Quotas were set for each province, and the weighting adjusted for oversampling in certain provinces so that each province has a representative importance in the results.
| Unweighted | Weighted | |
|---|---|---|
| British Columbia | 12% | 14% |
| Alberta | 11% | 11% |
| Saskatchewan | 5% | 3% |
| Manitoba | 6% | 4% |
| Ontario | 34% | 39% |
| Quebec | 21% | 23% |
| New Brunswick | 4% | 2% |
| Nova Scotia | 4% | 3% |
| Prince Edward Island | <1% | <1% |
| Newfoundland | 3% | 1% |
The following tables present the demographic distribution of respondents, according to gender and age.
First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.
| Unweighted | Weighted | |
|---|---|---|
| Male | 48% | 49% |
| Female | 51% | 51% |
| Another gender | <1% | <1% |
| Prefer not to say | <1% | <1% |
Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.
| Unweighted | Weighted | |
|---|---|---|
| 18-24 | 10% | 10% |
| 25-34 | 16% | 17% |
| 35-44 | 17% | 16% |
| 45-54 | 16% | 16% |
| 55-64 | 18% | 18% |
| 65+ | 23% | 24% |
There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study.
The relatively small weight factors and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.
Appendix 2: Quantitative Methodology (Wave 2)
As a Canadian Research Insights Council (CRIC) Member, Leger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research— Series A – Fieldwork and Data Tabulation for Online Surveys.
Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Leger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.
Sampling Procedure
Leger conducted a panel-based Internet survey with a sample of adult Canadians, using Computer Aided Web Interviewing (CAWI) technology. A total of 2,253 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo’s online panel.
Leger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of Web users profiled on different sociodemographic variables. The majority of Leger’s panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.
Data Collection
Fieldwork for the survey was carried out from June 10th to June 23rd, 2024. A pre-test of 55 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. A total of 2,253 Canadians with demographic characteristics reflective of the Canadian population were surveyed. An average of 14 minutes was required for respondents to complete the survey. The sample was drawn randomly from the Leo panel and the overall participation rate for the survey was 10.87%.
Since a sample drawn from an Internet panel is not probabilistic in nature, a margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.
During data collection, regional quotas were set, and soft quotas were set in order to ensure a sufficient subsample size for Immigrants and Indigenous Peoples. The tables below detail the distribution of respondents by region, gender, age, place of birth, and Indigenous self-identification.
| Number of respondents (unweighted) | |
|---|---|
| British Columbia | 280 |
| Alberta | 241 |
| Saskatchewan | 120 |
| Manitoba | 130 |
| Ontario | 771 |
| Quebec | 460 |
| New Brunswick | 80 |
| Nova Scotia | 100 |
| Prince Edward Island | 11 |
| Newfoundland | 60 |
| Number of respondents (unweighted) | |
|---|---|
| Woman | 1,142 |
| Man | 1,096 |
| Another gender | 11 |
| Prefer not to say | 4 |
| Number of respondents (unweighted) | |
|---|---|
| 18-24 | 228 |
| 25-34 | 373 |
| 35-44 | 372 |
| 45-54 | 354 |
| 55-64 | 395 |
| 65+ | 531 |
| Number of respondents (unweighted) | |
|---|---|
| Canada | 1,758 |
| Outside Canada | 489 |
| Prefer not to say | 6 |
| Number of respondents (unweighted) | |
|---|---|
| Indigenous | 105 |
| Non-Indigenous | 2,140 |
| Prefer not to say | 8 |
Participation Rate
The overall participation rate for this study is 10.87%.
The participation rate is calculated as the number of in-scope responding units (n=2,343), divided by the sum of unresolved units, in-scope non-responding units, and in-scope responding units (n=21,543). The detailed completion results are as follows:
- Total number of sample units invited to participate: 21,545
- Number of invalid cases: 2
- Invitations mistakenly sent to people who did not qualify for the study: 2
- Incomplete or missing email addresses: 0
- Number of unresolved units: 18,969
- Email invitations bounce back: 17
- Email invitations unanswered: 18,952
- Number of in-scope non-responding units: 231
- Non-response from eligible respondents: 0
- Respondent refusals: 52
- Language problem: 0
- Selected respondent not available (illness; leave of absence; vacation; other): 0
- Early breakoffs: 179
- Number of in-scope responding units: 2,343
- Surveys disqualified – quota filled: 51
- Completed surveys disqualified for other reasons: 39
- Completed interviews: 2,253
Typical participation rates for web surveys are between 20% and 30%. A participation rate of 10.87% may seem a bit low, but due to time constraints, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate.
Unweighted and Weighted Samples
Based on data from Statistics Canada’s 2021 national census, Leger weighted the results of this survey by gender and age within each province, in order to reflect the actual distribution of the Canadian adult population.
A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower participation rates among specific demographic subgroups (see tables below).
The table below presents the geographic distribution of respondents, before and after weighting. Quotas were set for each province, and the weighting adjusted for oversampling in certain provinces so that each province has a representative importance in the results.
| Unweighted | Weighted | |
|---|---|---|
| British Columbia | 12% | 14% |
| Alberta | 11% | 11% |
| Saskatchewan | 5% | 3% |
| Manitoba | 6% | 4% |
| Ontario | 34% | 39% |
| Quebec | 20% | 23% |
| New Brunswick | 4% | 2% |
| Nova Scotia | 4% | 3% |
| Prince Edward Island | <1% | <1% |
| Newfoundland | 3% | 1% |
The following tables present the demographic distribution of respondents, according to gender and age.
First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.
| Unweighted | Weighted | |
|---|---|---|
| Male | 49% | 49% |
| Female | 51% | 51% |
| Another gender | <1% | <1% |
| Prefer not to say | <1% | <1% |
Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.
| Unweighted | Weighted | |
|---|---|---|
| 18-24 | 10% | 10% |
| 25-34 | 17% | 17% |
| 35-44 | 17% | 16% |
| 45-54 | 16% | 16% |
| 55-64 | 18% | 18% |
| 65+ | 24% | 24% |
There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study.
The relatively small weight factors and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.
Appendix 3: Quantitative Instruments
English and French quantitative instruments are provided under separate cover.