Supplier name: Ipsos
Contract Number: CW2230192
Contract value: $42,488.57
Award Date: April 27th, 2024
Delivery Date: May 24th, 2024
Registration number: POR 143-23
For more information on this report, please contact Canadian Revenue Agency at:
cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français
Prepared for Canadian Revenue Agency (CRA)
Supplier name: Ipsos
May 2024
This public opinion research report presents the methodology of the 2023-2024 CRA Benefits and Credits Campaign – ACET online survey conducted by Ipsos on behalf of the Canadian Revenue Agency (CRA).
The online survey was conducted with 2,148 respondents between May 2nd and 15th, 2024.
Cette publication est aussi disponible en français sous le titre Campagne publicitaire sur les prestations et les crédits de l'ARC de 2023-2024 Outil d'évaluation des campagnes publicitaires (OECP) – Rapport méthodologique
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Canada Revenue Agency. For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca or at:
101 Colonel By Drive
Ottawa, Ontario K1A 0K2
Canada
Catalogue Number: Rv4-137/1-2024E-PDF
International Standard Book Number (ISBN): 978-0-660-72621-2
Related publications (registration number: POR 143-23):
Catalogue Number: Rv4-137/1-2024F-PDF (Methodological Report, French)
ISBN: 978-0-660-72622-9
© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2024
POLITICAL NEUTRALITY STATEMENT
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Brad Griffin
President
Ipsos Public Affairs
The Canada Revenue Agency (CRA) administers taxes, benefits, and related programs for governments across Canada. These services and programs help residents of Canada comply with tax obligations and receive benefits, contributing to socio-economic well-being. The CRA is undertaking significant efforts to increase awareness of and access to CRA benefits and services, participation in the tax system, and understanding of roles and responsibilities within the tax and benefit system.
Benefits and credits provide additional income or tax relief to individuals to help make life more affordable. Many depend on benefit payments and tax credits for a significant part of their household income. For modest-income households, benefits can increase income by as much as 50 percent.
However, certain segments of the vulnerable population, such as modest-income households, Indigenous Peoples, newcomers to Canada, and youth aged 18–24, face various barriers when filing their taxes and receiving their benefits and credits. These barriers include a lack of general awareness of benefits and credits, a lack of support to complete their taxes, living in remote locations, language barriers, low financial literacy, and/or mental or physical health issues. Given the range of backgrounds and circumstances, it is important for the Government of Canada (GC) to make proactive and targeted efforts to reach vulnerable populations with important tax-related information. Moreover, greater awareness is needed so these populations know that filing their taxes every year is necessary to be eligible for these benefits and credits. Increasing awareness of benefits and credits will help contribute to the government's priorities of poverty reduction, health, and well-being, as found in the GC's Gender Results Framework.
This Benefits and Credits advertising campaign is part of a sustained effort by the CRA to raise awareness among target audiences about the benefits and credits they are entitled to when they do their taxes. The campaign featured the Canada Child Benefit (CCB), Canada workers benefit (CWB), GST/HST credit, the disability tax credit (DTC), and the Community Volunteer Income Tax Program's (CVITP) free tax clinics. The campaign also promoted housing programs, including the First Home Savings Account, the Multigenerational Home Renovation Tax Credit, the Home Buyers' Plan, the GST/HST new housing rebate, and the home buyers' amount
Advertising campaign evaluation is mandatory for all advertising campaigns in which media buys exceed $1 million. The Advertising Campaign Evaluation Tool (ACET) is used for all major Government of Canada (GC) advertising campaigns in order to consistently evaluate these campaigns and comply with key requirements in the GC Communications Policy.
Ipsos was contracted to conduct the data collection and tabulation, and to provide a methodology report. Ipsos programmed, hosted, and provided sample management services, while the CRA provided the online questionnaire. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The data collection from Canadians was handled in accordance with government-wide Public Opinion Research procedures.
The research consisted of an online post-campaign survey conducted in English and French, between May 2nd and 15th, 2024.
The total contract value of this research was $42,488.57, including HST.
Advertising campaign evaluations assist in determining the effectiveness of the selected advertising campaigns by measuring (and comparing) awareness of the subject matter with the audiences. The results will be used to evaluate the effectiveness of the Benefits and Credits advertising campaign with the target audience. The results will also help assess whether the ads were compelling / caught the audiences' attention, and whether the campaign raised awareness of the benefits and credits being promoted. Finally, the evaluation will assist the CRA in improving or modifying future ad campaigns.
For this research, CRA was seeking to evaluate the campaign which ended in April 2024. This post-campaign survey used the standardized ACET Post-Campaign online survey to assess:
The target audience of the research was as follows:
Primary Target Audience:
Secondary Target Audience:
Respondents for the survey samples were drawn from a trusted partner panel vendor, Canadian Viewpoint Inc. The survey was conducted with a sample of 2,148 Canadians ages 18+. Interviews were conducted in the language of their choice, either English or French.
A pre-test was launched on May 2nd, 2024, which garnered 37 completes (25 English / 12 French). An open-ended question was asked at the end of the survey where any problems, questions, or unclear questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between May 3rd and 15th, 2024.
Quotas were set to ensure representation by region, age, and gender, according to the latest Census information.
The sampling methodology utilized email invitations and router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the online interviews took 10 minutes to complete.
Ipsos partnered with sub-contractor Canadian Viewpoint Inc. Canadian Viewpoint has one of the largest consumer panels with ~300,000 active panelists. Like Ipsos, Canadian Viewpoint uses mixed-medium strategies (phone/online/in-person) to build its panel to allow more representative samples. Canadian Viewpoint's online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted lists and in-person studies. Incentives were not used for recruitment purposes to ensure quality, but respondents were incentivized for completing the survey, which is directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources.
The comprehensive background profiling data were gathered when respondents joined the panel through screener questions, which allowed for the targeting of respondents based on key criteria, such as region, age, gender, education, and income level, intention to buy a home within 12 months, profession, and other characteristics.
The online survey was conducted using Computer Assisted Web Interviewing (CAWI). Every panelist opted-in or chose to participate in research surveys. Panelists that meet the criteria we were looking for were randomly selected and sent an email invitation to complete the survey. Participants were offered a choice to complete the survey in either English or French.
The survey platform was Accessibility for Ontarians with Disabilities Act (AODA) compliant, according to Web Content Accessibility Guidelines (WCAG2.0AA).
Sample Weighting
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2021 Census. A Random Iterative Method (RIM) technique was applied for weighing.
| Demographics | Post-wave Survey | |
| Unweighted Sample Size | Weighted Sample Size | |
| Canada | 2148 | 2148 |
| Region | ||
| British Columbia/Yukon | 283 | 301 |
| Alberta/Northwest Territories | 239 | 236 |
| Prairies (MB/SK) | 161 | 129 |
| Ontario/ Nunavut | 850 | 838 |
| Quebec | 473 | 494 |
| Atlantic Canada | 142 | 150 |
| Gender | ||
| Male | 1008 | 1048 |
| Female | 1133 | 1092 |
| Diverse / Prefer not to answer | 7 | 8 |
| Age | ||
| 18-24 | 236 | 253 |
| 25-34 | 302 | 327 |
| 35-44 | 252 | 347 |
| 45-54 | 247 | 340 |
| 55-64 | 276 | 223 |
| 65+ | 835 | 658 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples, except for the age group 35-64. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.38, which is well within acceptable ranges for a survey of the general population.
Within natural fallout, we obtained our quota targets, apart from an oversample of Indigenous respondents (n=50).
The unweighted counts are shown below.
| Primary target audience | Target | Achieved |
| Indigenous | 100 | 109 |
| Moderate-income (<$50K) | 400 – 500 | 664 |
| Newcomers <7years | ~50 | 108 |
| People with disabilities | 400 – 440 | 431 |
| Youth aged 18-24 | 200 – 250 | 226 |
| Secondary target audience | ||
| Potential new homebuyers | -- | 453 |
Incentives and Quality Control Measures
Canadian Viewpoint use a multi-faceted approach to quality, integrating controls and best practices into every phase of the process. Panel members come from diverse fully certified sources. A dedicated team monitors performance by source using feedback loops and continuous reporting to prevent fraud. Our dedicated team monitors the panels for potentially fraudulent behavior, and blocks these people from ever entering a survey via safeguards that includes:
Email Statistics
For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 91%, which is calculated as follows: (qualified completes + over quota + terminates)/click-through).
| Completions | Post-wave Survey |
| Click-Through | 3987 |
| - Partial Completes | 56 |
| - Terminates | 696 |
| - Over quota | 793 |
| Qualified Completes | 2148 |
| Participation Rate | 91% |
Non-Response Analysis
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.
The table below compares the unweighted survey samples to the 2021 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Demographics | Post-wave Survey | |
| Unweighted Percentage | Census 2021 Proportions | |
| Canada | ||
| Region | ||
| British Columbia/Yukon | 13% | 14% |
| Alberta/Northwest Territories | 11% | 11% |
| Prairies (MB/SK) | 7% | 7% |
| Ontario/ Nunavut | 40% | 38% |
| Quebec | 22% | 23% |
| Atlantic Canada | 7% | 7% |
| Gender | ||
| Male | 47% | 49% |
| Female | 53% | 51% |
| Diverse / Prefer not to answer | <1% | <1% |
| Age | ||
| 18-24 | 11% | 11% |
| 25-34 | 14% | 17% |
| 35-44 | 12% | 17% |
| 45-54 | 11% | 16% |
| 55-64 | 13% | 18% |
| 65+ | 39% | 24% |
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with slight differences observed between the unweighted percentages and the 2021 Census data.
ADVERTISING CAMPAIGN EVALUATION TOOL
POST-CAMPAIGN SURVEY
(CRA Post-Campaign ACET 2023-2024)
INTRODUCTION
Thank you for taking a few minutes to complete this survey on current issues that matter to Canadians. Please select the language you would like to take this survey in /Si vous préférez répondre à ce sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].
Your participation is voluntary, and your responses will be kept entirely confidential. The survey takes about 10 minutes to complete.
START SURVEY
Click here if you wish to verify the authenticity of this survey. To view our privacy policy, click here.
If you require any technical assistance, please contact daniel.kunasingam@ipsos.com
a) Does anyone in your household work for any of the following organizations?
SELECT ALL THAT APPLY
IF "NONE OF THESE ORGANIZATIONS" CONTINUE, OTHERWISE THANK AND TERMINATE.
b) What is your gender?
c) In what year were you born?
YYYY
ADMISSIBLE RANGE 1900-2006
IF > 2006, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
d) In which of the following age categories do you belong?
SELECT ONE ONLY
IF "LESS THAN 18 YEARS OLD" OR "BLANK", THANK AND TERMINATE
e) In which province or territory do you live?
SELECT ONE ONLY
IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE
f) Where were you born?
ASK IF g=BORN OUTSIDE CANADA
g) In what year did you first move to Canada?
YYYY
ADMISSIBLE RANGE: 1900-2024
h) Are you an Indigenous person? (An Indigenous person is a North American Indian or a member of a First Nation, a Métis or an Inuk (Inuit). North American Indians or members of a First Nation include status, treaty or registered Indians, as well as non-status and non-registered Indians.)
SELECT ONE ONLY
i) Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
j) Are you a person with a disability?
A person with a disability has a long-term or recurring physical, mental, sensory, psychiatric or learning impairment and:
Persons with disabilities are also those whose functional limitations owing to their impairment have been accommodated in their current job or workplace. Disabilities include:
k) Are you considering purchasing your first home in the next 5 years?
CORE QUESTIONS
ASK ALL RESPONDENTS
Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Q2: Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?
Don't remember
CAMPAIGN-SPECIFIC QUESTIONS
ASK ALL RESPONDENTS
T1A: Over the past three weeks, have you seen, read or heard any Government of Canada advertising about benefits, tax credits and/or free tax help?
T1B: Where have you seen, read or heard this ad about benefits, tax credits and/or free tax help?
[NOTE: SELECT FROM LIST ALL MEDIA USED IN THE CAMPAIGN. YOU MAY ALSO INCLUDE OTHER MEDIA OF YOUR CHOICE. HEADINGS ARE FOR GUIDANCE ONLY AND NOT TO BE USED IN THE FINAL VERSION OF THE QUESTIONNAIRE]
SELECT ALL THAT APPLY
[Randomize]
T1C: What do you remember about this ad?
ASK ALL RESPONDENTS
CAMPAIGN SPECIFIC ATTITUDINAL AND BEHAVIOURAL QUESTIONS MAY BE ADDED HERE
T1D: Are you aware that you may qualify for Government of Canada benefits and tax credits?
T2D: Are you aware that there is free tax help for eligible people to get their taxes done by volunteers at free tax clinics?
T2DA: What motivates you to do your taxes?
T3D: Do you feel confident to do your own taxes, whether on paper or using tax software yourself?
T4D: Who does your taxes?
T4E: [IF DO ON MY OWN IN T4D ASK] Do you do taxes for anyone else in addition to yourself (select all that apply)
T4F: [IF DO ON MY OWN NOT SELECTED IN T4D] Why do you not do your own taxes?
AD RECALL QUESTIONS
T1H: Here are some ads that have recently been broadcast on various media. Click here to watch.
[INSERT VIDEO, PRINT, AND RADIO ADS]
cra_max_15e_unslated_web
cra_paul_15e_unslated_web
cra_priya_15e_unslated_web
[CLICK TO GO TO THE NEXT PAGE]
Over the past three weeks, have you seen, read or heard these ads?
T1I: Where have you seen, read or heard these ads?
[NOTE: SELECT FROM LIST ALL MEDIA USED IN THE CAMPAIGN. YOU MAY ALSO INCLUDE OTHER MEDIA OF YOUR CHOICE. HEADINGS ARE FOR GUIDANCE ONLY AND NOT TO BE USED IN THE FINAL VERSION OF THE QUESTIONNAIRE]
SELECT ALL THAT APPLY
[Randomize]
T1J: What do you think is the main point these ads are trying to get across?
T1K: Please indicate your level of agreement with the following statements about these ads?
|
1 Strongly Disagree |
2 | 3 | 4 |
5 Strongly Agree |
|
| These ads catch my attention | o | o | o | o | o |
| These ads are relevant to me | o | o | o | o | o |
| These ads are difficult to follow | o | o | o | o | o |
| These ads do not favour one political party over another | o | o | o | o | o |
| These ads talk about an important topic | o | o | o | o | o |
| These ads provide new information | o | o | o | o | o |
| These ads clearly convey that I may qualify for Government of Canada benefits, tax credits and/or free tax help. | o | o | o | o | o |
T1L: [ASK IF YES IN T1H] Which of the following actions did you take as a result of seeing/hearing this advertising? [ACCEPT MULTIPLE RESPONSES]
DEMOGRAPHIC QUESTIONS
D1: Which of the following categories best describes your current employment status? Are you…
SELECT ONE ONLY
D2: What is the highest level of formal education that you have completed?
SELECT ONE ONLY
D3: Are there any children under the age of 18 currently living in your household?
D4: What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of the Canada Revenue Agency. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
[1] According to the 2017 Canadian Survey on Disability, more than 6 million Canadians aged 15 and over (22% of the population) identify as having a disability, and it is expected actual numbers are likely higher. (https://www.canada.ca/en/employment-social-development/programs/accessible-canada.html) The CSD definition of disability includes anyone who reported being "sometimes," "often" or "always" limited in their daily activities due to a long-term condition or health problem, as well as anyone who reported being "rarely" limited if they were also unable to do certain tasks or could only do them with a lot of difficulties.
[2] The CRA is also looking for potential new homebuyers earning $60K or more (as part of the natural fallout in the total sample of the 2000 general population; not part of set quotas). There will be questions about the intention to purchase a home among this income segment within the next three years to capture the natural fallout within the sample.