Pre-testing and Evaluation of the “Make Change
that Counts: Managing Your Money in a Changing World” Ad Campaign
Executive Summary
Prepared
for the Financial Consumer Agency of Canada (FCAC)
Supplier: Leger Marketing Inc.
Contract
Number: CW2335352
Contract Value: $100,567.74
Award Date:
October 25, 2023
Delivery Date: June 10, 2024
POR
Registration Number: POR 078-23
For more information on this report, please contact the Financial
Consumer Agency of Canada at:
info@fcac-acfc.gc.ca
Ce rapport est
aussi disponible en français
Pre-testing and Evaluation of the
“Make Change that Counts: Managing Your Money in a Changing World” Ad Campaign
Executive Summary
Prepared for the Financial
Consumer Agency of Canada
Supplier Name: Leger
Marketing Inc.
May 2024
This public opinion research report provides the results of the focus groups conducted pre-campaign, as well as a methodological report for pre- and post-campaign online surveys conducted by Leger Marketing Inc. on behalf of the Financial Consumer Agency of Canada. The focus group research was conducted with 64 Canadians between November 21st and November 23rd, 2024. The ACET pre-campaign survey was conducted with 2,003 respondents from the Canadian general public aged 18 years and older, between January 5 and January 12, 2024. The ACET post-campaign survey was conducted with 2,014 respondents from the Canadian general population aged 18 years and older, between April 1 and 9, 2024.
Cette publication
est aussi disponible en français sous le titre : Essai préliminaire et évaluation de la campagne publicitaire « Faisons des
changements qui comptent : Gérer votre argent dans un monde en évolution » :
Sommaire exécutif.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Financial Consumer Agency of Canada. For more information on this report, please contact the Financial Consumer Agency of Canada at: info@fcac-acfc.gc.ca
Financial Consumer Agency of Canada
427 Laurier Avenue West, 5th Floor
Ottawa ON K1R 7Y2
Catalogue Number: FC5-84/2024E-PDF
International Standard Book Number (ISBN): 978-0-660-72365-5
Related publications (registration number: POR 078-23):
© His Majesty the King
in Right of Canada, as represented by the Minister of Finance Canada, 2024.
Supplier: Leger Marketing
Inc.
Contract
Number: CW2335352
Contract Value: $100,567.74
Award Date:
October 25, 2023
Delivery Date: June 10, 2024
Leger was contracted by the Financial Consumer Agency of
Canada (FCAC) to conduct concept
testing and campaign evaluation using the Advertising
Campaign Evaluation Tool (ACET) for the 2023-2024 ad campaign “Make
Change that Counts: Managing Your Money in a Changing World.” A total of eight (8) online focus groups
were conducted from November 21st to November 23rd, 2023,
with Canadian residents 18 or older. Participants were recruited from across
Canada to take part in a 90-minute focus group, held virtually. A total of six
(6) groups were conducted in English and two (2) groups were conducted in
French. In addition, the
pre-campaign survey was conducted with 2,003 respondents from the Canadian
general population aged 18 years and older, between January 5 and January 12,
2024. The post-campaign survey was conducted with 2,014 respondents from the
Canadian general population aged 18 years and older, between April 1 and 9,
2024. The following summarizes the findings from the qualitative
research study and the methodological
report on findings from the ACET surveys designed to determine the effectiveness of the advertising campaign in
shifting attitudes, beliefs, and behaviours by measuring awareness of the
subject matter with the target audiences.
The Financial Consumer Agency of Canada (FCAC) is an independent federal government agency mandated to protect Canadian financial consumers. FCAC supervises federally regulated financial entities, ensuring that they comply with consumer protection measures set out in legislation, public commitments, and codes of conduct. It also promotes financial education to strengthen the knowledge, skills and confidence of Canadians and raise awareness of consumer rights and responsibilities.
Strengthening the financial literacy of Canadians is a
key pillar of FCAC’s consumer protection mandate. Canadians face an
increasingly complex and digital financial marketplace. For this reason, it is
more important than ever that Canadians have the knowledge, skills, and
confidence to make informed decisions about their personal finances. Financial
literacy is important not only for the financial well-being of individuals, but
also for the economy.
The rising cost of living has left many Canadians
increasingly stressed and worried about their financial futures. While
inflation in Canada has slowed in recent months, the cost of living remains
high. As of July 2023, inflation rose 3.3% year over year. In addition, the
Bank of Canada rapidly increased its policy rate from 0.25% in March 2022 to
the level of 4.75%, bringing higher mortgage rates along with it. Higher
interest rates have resulted in many homeowners experiencing financial
difficulties. They have also put additional pressure on Canada’s rental market.
Higher rates have not only created more demand for rental properties as
homebuyers wait for borrowing costs to come down, but they have also increased
rent prices as landlords pass off higher costs to tenants.
At the same time, the number of highly indebted
households in Canada has risen. This is on top of already historically high
debt levels in Canada. According to the Bank of Canada’s Financial System
Review, more households are carrying large mortgage debt and households with
high debt burdens are especially vulnerable if their incomes decline. Renters,
who represent 41.2% of Canadians, are also struggling to keep up with their
financial commitments and are more likely than homeowners to report that their
debt had increased since 2019.
Given that mortgage holders and renters are increasingly
facing financial challenges, it is important that they have access to
authoritative and unbiased financial information, tools, and resources to help
them confidently make informed decisions. Strengthening the financial literacy
of Canadians will help decrease their risk of vulnerability and increase their
resilience in financially challenging times.
On top of personal financial challenges, Canadians face a
daunting financial landscape. The marketplace for financial products, services,
and advice is complex, constantly changing and increasingly digital. There is a
wide range of product choices, an overwhelming amount of information available
through financial websites and financial apps, and more people and companies
providing advice on money and finances. As a result, it is more difficult to
navigate the financial marketplace, know who to trust, and how to make the best
decisions for one’s circumstances. This is especially problematic for people
with limited financial knowledge.
National
advertising campaign
FCAC’s national multimedia campaign
that ran from January 15 to March 31, 2024, aimed to strengthen the financial
literacy of Canadians and help them build financial resilience in a continually
changing economic landscape. Under the theme of “Managing Your Money in a
Changing World”, FCAC’s national campaign focused on helping Canadian financial
consumers manage their money and navigate the financial marketplace as they
consider their housing and home financing options in today’s financially
challenging times.
The purpose of the qualitative
research was to conduct a pre-test of the advertising creative concepts to
guide decisions related to the selection of the most effective creative concept
and assist with the finalization of creative elements for the ads. The purpose
of the quantitative research was to conduct the campaign evaluation using the
standardized Advertising Campaign Evaluation Tool (ACET). Pre- and
post-campaign ACETs are used for all major Government of Canada advertising
campaigns to consistently evaluate campaigns and comply with key requirements
in the Government of Canada Communications Policy.
The campaign’s research objectives included three components:
1.
Qualitative
pre-testing of creative concepts: The goal
was to choose the creative concept that resonated the most with the target
audiences and performed the best in terms of meeting the campaign objectives.
2.
Quantitative pre-campaign evaluation using the
standardized ACET baseline online survey: The goal was to assess pre-campaign
awareness of the subject matter, including:
·
Aided and unaided awareness of general Government of
Canada advertisements or other advertisements related to the campaign topic;
·
Aided and unaided awareness of the subject matter.
3.
Quantitative post-campaign
evaluation using the standardized ACET post-campaign online survey: The goal was to
measure the post-campaign awareness of the subject matter, including:
·
Aided and unaided awareness of FCAC’s ads and general Government of
Canada ads;
·
Recall of key campaign messages;
·
Whether the ads were attention grabbing, relevant, difficult to follow,
about an important topic, and provided new information;
·
Whether any action was taken as a result of seeing or hearing FCAC’s ads;
· Awareness of who was responsible for creating the ads.
Pre-testing of creative concepts (qualitative
research)
A qualitative (focus group) research approach was used
to pre-test the creative
concepts for final feedback before being released in the media. The
target audience for the research included working-aged Canadians (18 to 65+)
with participants recruited from the following targets:
· Homeowners
with a mortgage
· Canadians
considering buying a home in the next 12 to 24 months
· Renters
The qualitative research to pre-test the creative concepts comprised of eight (8) online focus groups. The groups took place from November 21st to November 23rd, 2023, with Canadian residents 18 or older (with a maximum of one participant 65+ in each group). Participants were recruited from across Canada to take part in a 90-to-100-minute focus group, held virtually. A total of six (6) groups were conducted in English and two (2) groups were conducted in French. A total of sixty-four (64) participants took part in the focus groups and all participants were provided with $100 as an honorarium. All participants were recruited per established industry standards and per the recruitment specifications of the Standards for the Conduct of Government of Canada Public Opinion Research—Qualitative Research. The participants were screened using Leger’s online panel and then validated and recruited by phone.
The focus groups evaluated three video ads and nine social
media ads, distributed across three distinct concepts: Housing on the Mind, Home
Finances Toolbox, and Keeping up with the Joneses. Each concept was represented
by one storyboard video and three social media ads. All participants viewed all
the ads from each concept, and the sequence of the concepts presented was
varied.
This report presents the
findings from these focus groups. Caution must be exercised when interpreting
the results from this research, as qualitative research is directional only.
Results cannot be attributed or extrapolated to the overall population under
study, with any degree of confidence.
Pre- and post-campaign ACET (quantitative
research)
For the ACET, the target audience was Canadians
18 years of age and older. Data for both the pre- and post-campaign surveys was
gathered from Leger’s panel LEO. LEO has approximately 400,000 members
nationally and has a retention rate of 90%. The panel is made up of Web users
profiled on different sociodemographic variables. The
majority of Leger’s panel members (61%) have been recruited randomly
over the phone over the past decade, making it highly similar to the actual
Canadian population on many demographic characteristics. Leger ensured that the
sample reflected the key demographics of Canadians by age, gender, and region.
All recruiting, fieldwork procedures followed the Standards for the Conduct of Government of
Canada Public Opinion Research—Online Surveys.
The pre-campaign ACET baseline survey of
Canadians aged 18+ years of age was undertaken online from January 5 to 12, 2024.
It required an average of approximately four (4) minutes to administer for both
the French and English versions. There was a total of 2,003 useable surveys
completed in the pre-campaign evaluation.
The post-campaign ACET survey of Canadians aged
18+ years of age was undertaken online from April 1 to 9, 2024. It required an
average of approximately six (6) minutes to administer for both the French and
English versions. There was a total of 2,014 useable surveys completed in the
post-campaign evaluation.
No margin of error can be associated with a non-probability
sample (i.e., a web panel in this case, where respondents
opt-in to the panel). For comparative purposes, though, probability samples of
2,003/2,014 respondents would have a margin of error of ±2.2%, 19 times out of
20.
Given that the
pre- and post-campaign ACET was conducted using online survey methodology and
the sample drawn from an internet panel is a non-probability sample, the data
collected cannot be extrapolated to the Canadian general public
adult population. Respondents for this survey were selected from among those
who had volunteered or registered to participate in online surveys. The results
of such surveys cannot be described as statistically projectable to the target
population. The data has been weighted to reflect the demographic composition
of the target population.
The pre-testing
helped to select a concept to move forward with and checked that the creative
concepts did not contain any hidden disasters and the overall reactions to them
were positive.
Among the
three concepts shown (Concepts 1-3), Housing on the Mind (Concept 1) was
the preferred by the majority of participants and
received high praise across all demographic segments. Home Finances Toolbox (Concept 2) was
the second most preferred concept of the three and evoked positive to neutral
reaction. While keeping up with the Joneses (concept 3) was
the least preferred among the participants, it was appreciated by participants
for the look and feel of the static social media ads, finding them
"eye-catching" and "memorable”.
Although
none of the concepts were deemed problematic, it's crucial to approach them
with care, particularly when addressing renters and individuals with lower
incomes. Participants with fixed incomes or financial challenges were often the
first to question the effectiveness of the tools and resources offered by the
government.
Given some questions about the effectiveness of the tools
and resources offered, providing
information on the type of support that is being offered would be a valuable
aspect to include, as well as emphasizing the utility of the tools regardless
of the user situation (renter, homeowner, or prospective home buyer).
“Housing
on the Mind” (Concept 1)
The Housing on the Mind concept garnered significant approval from the majority of participants, cutting across all demographic groups, and was particularly praised for its compelling and clear messaging. The concept’s approach, which featured literal imagery of houses on individuals' heads, resonated deeply with participants, especially those between the ages of 18-40, who found the images both unique and memorable. While some described the images as “jarring”, this was not necessarily a negative element since most indicated the ad would prompt a double take.
The concept was widely perceived as impactful and effective due to its literal and unique approach that resonated with the majority of participants. Many participants believed the Housing on the Mind concept demonstrated what Canadians are worried about in terms of housing and finances and would encourage them to take action (visit the website/investigate the resources).
The few participants who didn't prefer Housing on the Mind cited the imagery as off-putting and inappropriate for government messaging. They found the ads initially confusing, requiring a complete viewing for comprehension and criticized social media versions for busy visuals and a lack of a clear call to action.
Overall, while the concept received some critical reaction in terms of clarity and the appeal of visuals, it was most likely to capture the attention of a broad audience among the tested concepts and effectively communicated its key message about government resources available to tackle housing affordability issues.
“Home Finances Toolbox” (Concept 2)
The Home Finances Toolbox concept was also well liked and ranked as the second most preferred concept among the three presented. The focus on the toolbox was seen as interesting and the imagery was generally liked. Participants concurred that the core message of the campaign was to provide access to financial tools that aid in making informed decisions across various housing scenarios. The toolbox analogy was well-received, with many considering it an effective concept for the campaign. The idea of categorizing the tools into three groups, each tailored to specific informational needs, was also met with approval.
Several participants felt that this concept might appeal predominantly to older Canadians, critiquing it as somewhat boring and lacking the engaging elements for widespread attention. Some mixed reactions also emerged regarding toolbox symbolism, with some participants finding the imagery irrelevant or limited to those interested in actual tools. Some participants also found the storyboard concept confusing, especially concerning the implied connection between homeownership tasks and a renter ordering takeout, which was perceived as unclear and potentially negative. Critiques included a lack of information in the video leaving participants desiring more clarity, as well as the perception that the messaging in the video and the social media ads were not cohesive.
Overall, feedback on this concept was generally positive or neutral. Participants in Montreal and Halifax were more likely to select Home Finances Toolbox as the preferred concept in comparison to GTA/Ottawa and Vancouver.
“Keeping up with the Joneses” (Concept 3)
Although Keeping up with the Joneses was the least
preferred concept among the participants, it was acknowledged for the visual
appeal of its static social media ads, which were described as
"eye-catching" and "memorable." Most participants found the
message of the concept clear and easy to understand, interpreting the main
message as the government providing tools to help Canadians make informed
financial decisions, particularly in relation to purchasing a home or mortgage
renewal.
Participants, especially in the French language groups, perceived a lack of proposed solutions in the Keeping up with the Joneses concept. While the portrayal of Canadian families living the same lifestyle but not being at the same level of preparedness was well liked, the comparison of two families and the idea of life being about comparison were disliked across all groups. The visuals also evoked some skepticism about the government website's potential to significantly improve financial situations and concerns that the concept may not resonate with renters.
Opinions were divided on whether the Keeping up with the Joneses concept would appeal exclusively to homeowners or if its messaging would also resonate with renters and potential home buyers. Some participants, especially concerning the storyboard, believed that the concept did not target renters.
The total
project expenditure was $100,567.74 including HST, for qualitative
concept testing and the quantitative pre- and post-campaign ACET surveys.
I hereby certify as a representative of Leger Marketing Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed
Llisa Morrow, Associate Vice-President
Leger Marketing Inc.
May 14, 2024