2023 Staffing and Non-Partisanship Survey Methodological Report

January 2024

Report

Contract Information

  • Supplier: Advanis Inc.
  • Contract number: CW2267133
  • Contract Value: $299,986.75 (including HST)
  • Award date: 2022-12-30
  • Delivery date: 2024-03-06
Prepared For:
Public Service Commission of Canada

Accessibility

As of March 22, 2024, I, Nael Hajjar, confirm that the accessibility of this document has been verified.

Catalogue Number:  SC1-11E-PDF (Electronic PDF, English)

ISSN 2816-3982

Registration number: POR 106-22

This publication may be reproduced for non-commercial purposes only. Written permission must be obtained first from the Public Service Commission of Canada. For more information on this report, please contact: cfp.sdip-snps.psc@cfp-psc.gc.ca.

© His Majesty the King in Right of Canada, as represented by the President of the Public Service Commission of Canada, 2024

Ce rapport est aussi disponible en français.

Prepared for the Public Service Commission of Canada
Supplier Name: Advanis Inc.
January 2024

This report presents the methodological details for the 2023 Staffing and Non-Partisanship Survey conducted by Advanis Inc. on behalf of the Public Service Commission of Canada. The survey was administered to 273,186 employees of the Government of Canada, between September 20 and November 17, 2023.

Ce rapport est aussi disponible en français sous le titre: Sondage sur la dotation et l’impartialité politique de 2023 : rapport méthodologique.

Public Service Commission of Canada
22 Eddy Street
Gatineau, QC
K1A 0M7
Canada

1. Executive Summary

1.1 Background

The Staffing and Non-Partisanship Survey was first introduced in 2018 as a biennial survey covering all federal departments and agencies that fall under the Public Service Employment Act. This online survey helps the Public Service Commission of Canada (PSC) fulfill its mandate and responsibilities to oversee the integrity of the federal public service staffing system and ensure non-partisanship. It gathers key information on the integrity of the public service staffing system and is currently the only tool the PSC can use to assess employees’ perceptions of the public service staffing system and the political impartiality of the public service.

The PSC contracted Advanis to administer the third cycle of the survey in 2023.

1.2 Objectives

The survey results are important to achieving the PSC’s mandate. Answers to the 2023 survey questions help the PSC explore and understand employees’ perceptions of the federal public service staffing system and assess their awareness of rights and responsibilities related to political activities and non-partisanship.

Key themes covered in this survey include perceptions of employees, hiring managers, supervisors and staffing advisors on:

In 2023 new lines of questioning were added to reflect current priorities such as addressing biases and barriers facing equity-seeking groups.

1.3 Methodology

The PSC developed the 2023 Staffing and Non-Partisanship Survey questionnaire. The content was updated from the 2021 survey cycle. Part of survey development was a pretest where participants were encouraged to provide feedback and screenshots on the survey content while completing the survey.

The 2023 questionnaire was formatted as an electronic survey (to be completed online) that was fully compliant with Web Content Accessibility Guidelines 2.0, level AA or higher. Employees also had the opportunity to phone Advanis to complete the survey with an interviewer.

The survey was administered from September 20 to November 17, 2023, to public service employees who worked in federal departments and agencies which fall under the Public Service Employment Act. The survey was also administered to:

The sample frame was created using the most current available lists of all eligible employees provided by human resources services in each participating department or agency. The PSC then verified, cleaned, and combined these lists to create the entire sample frame and provided it to Advanis. All eligible employees with valid email addresses were sent an invitation to complete the electronic questionnaire. Invitations were sent out over a period of 5 days. Reminder emails were also sent to participants who had not responded.

During fielding, departments and the PSC could also add new employees to the sample frame through a self-serve support website hosted by Advanis throughout the data collection period.

Of the 273 186 employees eligible to participate, 86 717 completed the survey, for a response rate of 31.7%. Among respondents, 77% (66 403) completed the survey in English and 23% (20 314) completed it in French.

Considering that the survey was administered as a census, the results can be extrapolated to the broader population of employees within the participating federal departments and agencies that fall under the Public Service Employment Act.

Although all eligible employees were included in the sample for the 2023 survey, because of the levels of non-response, it was necessary to weight the responding units by department or agency so that they were representative of the target population. The weight adjusts for the non-response within gender and age groups, within each department or agency. Although the value of the bootstrap was discussed for a census type study like this one, 1 000 bootstrap weights were provided in a separate file.

1.4 Contract value

The contract value for this survey was $299 986.75 (including HST).

1.5 Political Neutrality Certification

I hereby certify as a Senior Officer of Advanis that the deliverables fully comply with the Government of Canada’s political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on Management of Communications’ requirements for public opinion research reports. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Signed:
Nicolas Toutant
Vice-President, Research and Evaluation
Advanis
nicolas.toutant@advanis.ca

2. Background

The PSC is responsible for promoting and safeguarding a merit-based, representative and non-partisan public service that serves all Canadians. To achieve these goals, the PSC seeks to:

As part of the PSC’s mandate and responsibilities to oversee the integrity of the federal public service staffing system and ensure non-partisanship, the Staffing and Non-Partisanship Survey was first introduced in 2018 as a biennial public service-wide survey. This online survey gathers key information on the integrity of the public service staffing system. The survey is currently the only means by which the PSC can gauge employees’ perceptions of the public service staffing system and the political impartiality of the public service.

The survey was administered by Statistics Canada on behalf of the PSC in 2018, and with a delay due to the pandemic in 2021. The first cycle of the survey was conducted between February and April 2018; it was a redesign of the Survey of Staffing that was also conducted by Statistics Canada on several occasions in the past, starting in 2009.

For the third cycle of the survey in 2023, the PSC contracted Advanis to administer the survey.

All public servants employed in July 2023, by departments and agencies that conduct staffing under the Public Service Employment Act were eligible to take part in the survey. Employees of separate agencies (for example, the Canada Revenue Agency) were not eligible to participate in the survey.

The survey gathers information from a broad range of audiences (employees, hiring managers and staffing advisors) on key aspects of the staffing system (for example, merit). As in previous cycles, the survey also gathers critical information on employees’ understanding of their rights and responsibilities regarding political activities and non-partisanship.

3. Objectives

The survey results are key to achieving the PSC’s mandate. Answers to the survey questions help the PSC explore and understand employees’ perceptions of the federal public service staffing system and assess their awareness of their rights and responsibilities related to political activities and non-partisanship.

Key themes covered in this survey include perceptions of employees, hiring managers, supervisors and staffing advisors on:

In 2023, new lines of questioning were added to reflect current priorities such as addressing biases and barriers facing equity-seeking groups.

The 2023 survey results support the PSC, as well as participating departments and agencies, in identifying areas where progress was made, as well as gaps and opportunities for improvement. The information collected helps the PSC ensure that its policies, programs and practices are responsive to the public service population it serves. Survey results will help support efforts to safeguard non-partisanship and promote diversity and inclusion within the public service.

The survey will provide important information to managers, human resources advisors and deputy ministers to improve staffing within their own departments and agencies. The information will also be used to support reporting to Parliament in the PSC’s annual report.

4. Survey methodology

The survey was administered from September 20 to November 17, 2023, to public service employees who worked in federal departments and agencies that fall under the Public Service Employment Act, and to regular and civilian members of the Royal Canadian Mounted Police and members of the Canadian Armed Forces with direct reports hired under the Public Service Employment Act. Each person in the sample was contacted by email and asked to complete an electronic questionnaire hosted by Advanis.

4.1 Population coverage

The survey targeted public service employees who worked in federal departments and agencies that fell under the Public Service Employment Act as of July 2023.

The survey also targeted:

All eligible employees from the in-scope departments and agencies with valid email addresses were sent an invitation to complete the electronic questionnaire. Invitations were sent out over a period of 5 days.

The survey questionnaire targeted 3 distinct populations:

4.2 Participating departments and agencies

For the list of participating departments and agencies, please refer to the table in section 7.1 Response rates: departments and agencies.

4.3 Sample design

This survey is a voluntary census with a cross-sectional design.

The sampling unit is the employee.

The survey frame is made up of all in-scope employees who were on the July 2023 list that the PSC provided to Advanis and all members of the Canadian Armed Forces and the Royal Canadian Mounted Police who had at least 1 civilian employee reporting to them.

The sample frame was created using the most current available lists of all eligible employees provided by human resources (HR) services in each participating department or agency. The PSC provided an Excel template to the HR contact that specified the fields required to conduct the survey. The PSC then verified, cleaned and combined these lists to create the entire sample frame and provided it to Advanis.

Twice during the fieldwork period, Advanis provided lists of bounced and otherwise invalid email addresses to the PSC, and the PSC provided these to departments and agencies to have them corrected, if possible. During this same review, departments and agencies could indicate email addresses that should be deleted (for example, for employees who left the public service during fieldwork).

After September 20, 2023, departments and agencies and the PSC could also add new employees to the sample frame through a self-serve support website hosted by Advanis throughout the data collection period. Employees that were added through the support website were reviewed by the PSC to ensure they qualified to participate in the study. The support website is described in greater detail in section 5.3: Employee support.

5. Data collection

5.1 Questionnaire design

The PSC developed the Staffing and Non-Partisanship Survey questionnaire. The content was updated from the 2021 survey cycle.

A pretest was conducted between March 21 and 27, 2023, to test the survey content. Participants were encouraged to provide feedback and screenshots on the survey content while completing the survey. A total of 58 questionnaires (37 in English and 21 in French) were completed.

The 2023 questionnaire was formatted as an electronic survey (to be completed online). Employees also had the opportunity to phone Advanis to complete the survey with an interviewer. In total, just 2 surveys were completed over the phone. The average time to complete the survey online was 14.6 minutes.

The online survey was required to be fully compliant with Web Content Accessibility Guidelines, 2.0 level AA or higher. Thus, the online survey development process was exhaustive. Advanis worked with the Government of Canada to subject the survey to thorough testing and review, and the result was a fully compliant and accessible survey. The questionnaire was also fully tested by Advanis and the PSC to ensure that all questions in the online version matched those in the questionnaire provided to Advanis and that all skip and branching in the survey worked as intended.

5.2 Data collection

Responding to this survey was voluntary. Data were collected directly from survey respondents.

The PSC announced the launch of the survey to deputy heads and heads of HR of all participating departments and agencies. Template messages to promote participation were shared with participating departments and agencies. Weekly response rates were provided to the PSC.

The PSC also arranged 2 group meetings, between:

At these meetings, the PSC and Advanis shared information about the timeline, logistics and support. In particular, Advanis shared details to allow departments and agencies to ensure that emails from Advanis would be accepted by Government of Canada servers and reach employees. Advanis also sent a specially designed survey to the IT representatives, which allowed them to test the process themselves.

Advanis used its proprietary software to send email invitations and reminders. Each employee in the sample frame received an email containing an invitation to complete the electronic questionnaire with a link to the questionnaire hosted by Advanis. The link had an embedded access code that provided access to the survey questionnaire. This unique link allowed the employee to stop the survey and return to it at another time, exactly where they left off. It also meant that each employee could only complete the survey once.

A week before the main launch of the survey, to ensure that the process worked properly, Advanis sent email invitations to:

This soft launch occurred from September 13 to September 14, 2023. Advanis then sent the email invitations to the remaining departments and agencies over 5 business days, from September 20 to September 26, 2023. For some large departments, such as Employment and Social Development Canada, Advanis sent the emails over a few days to reduce overload on their servers.

During collection, 6 reminder emails were sent to participants who had not responded.

5.3 Employee Support

Advanis provided support to employees in 3 ways: through a customized support website, by email and by telephone.

Advanis created a support website so that employees could get help with common issues very quickly, and at any time of the day or night, 7 days a week, throughout the data collection period. When departments and agencies communicated about the survey to employees, they provided the link to this site. By visiting this support website, employees could:

Hundreds of employees visited the support site. Of note, a total of 362 employees chose the “Request a link to the survey” option, doing so a total of 454 times. Employees “requested a link” for the following reasons:

Advanis monitored and responded to 265 emails from employees who needed support. Some employees emailed for multiple reasons. Of these 265 emails, there were 195 individual requests. The breakdown of the reasons for emailing is as follows:

Advanis provided a toll-free number that employees could call with questions. Employees could also leave a voicemail, to which an Advanis interviewer would respond the same or next business day. In total, Advanis dealt with 8 calls that included 9 issues. Employees called for these reasons:

6. Data processing

The main outputs of the Staffing and Non-Partisanship Survey include a “clean” microdata file. The microdata file consists of data processed from the electronic questionnaire. This section presents a brief summary of the processing steps involved in producing these files.

6.1 Data capture

Responses to all survey questions were entered directly by the respondents. The electronic questionnaire reduces processing time and costs associated with data entry, transcription errors and data transmission. The responses were secured through industry standard encryption protocols, firewalls, and encryption layers.

Some editing was done directly when the electronic questionnaire was completed. When a response was outside the range (too large or small) of expected values, or inconsistent with the previous entries, the respondent was prompted, through message screens, to verify the information. The respondents also had the option of not answering questions if they did not know the answer or refused to answer. In these cases, the data were subjected to further edit processes after they were submitted.

6.2 Data cleaning

The information from all respondents who submitted the electronic questionnaire was retained and the data processed.

Editing can occur at several points throughout the survey process. It ranges from simple preliminary checks to more complex automated verifications performed by a computer program after the data have been captured. In general, edit rules are based on what is logically or validly possible.

There were 2 categories of edits used for this survey: validity and consistency edits. Validity edits:

Consistency edits verify that relationships between questions are respected. Consistency edits can be based on logical, or structural relationships between questions or parts of a question.

Example

The first type of error treated involved errors in questionnaire flow, where questions that did not apply to the respondent (and should therefore not have been answered) were found to contain answers. In this case a computer automatically eliminated superfluous data by following the flow of the questionnaire implied by answers to previous questions.

The second type of error treated involved a lack of information in questions that should have been answered. For this type of error, a non-response or “not-stated” code was assigned to the item.

6.2.1 Pre-edits

For all records where values were missing (blank) from the collection, the value of “9,” “99,” “999” and so on, was assigned to indicate that no information was collected.

6.2.2 Flow edits

The flow edits replicate the flow patterns from the questionnaire. Variables that are skipped based on flows were converted from “Skipped” to “Valid skip” codes (“6,” “96,” “996,” and so on). For skips based on the answer to certain questions, skipped questions were set to “Valid skip.”

6.3 Coding of open-ended questions

There were 2 open-ended questions in the survey (COM_10B and MAN_S25, MAN_Q25_5) and 8 “other-specify” open text (GEN_35A_151_, GEN_35B_24_, STA_30_18_, STA_58_8_, GDR_10_3_, GDR_11_12_, DIS_01_12_, and PG_05L_). The answers written into these questions were reviewed and any personal identifying information was redacted. We also recoded the “other-specify” open-ended text from GEN_35A_151_ into the existing level of GEN_35A when possible.

6.4 Imputation

There was no imputation of data in the survey.

6.5 Creation of derived variables

A number of variables included on the microdata file have been derived by grouping answer categories or by combining variables on the questionnaire to facilitate data analysis.

6.6 Weights

Statistical weights were calculated for each respondent to indicate the number of employees in the population that the respondent represents.

The weights appear on the microdata file and must be used to derive estimates from the survey. For example, if the number of employees who worked in Quebec and participated in a staffing process in the last 12 months is to be estimated, it is done by selecting the records referring to employees in the sample with these characteristics and summing the weights entered on those records.

Details of the method used to calculate these weights are presented in section 8.

6.7 Suppression of confidential information

As described in section 6.3, answers to open-ended questions were reviewed by Advanis, and any personal identifying information was redacted. All results made available to the public through reports, data visualization tools, or the Open Government Portal are suppressed when there are fewer than 10 responses to the question.

7. Data quality

7.1 Response rates: departments and agencies

Of the 273 186 employees eligible to participate, 86 717 completed the survey, for a response rate of 31.7%.

77% (66 403 people) completed the survey in English and 23% (20 314 people) completed it in French.

The response rate is calculated as follows: r=R/N; r is the response rate, R is the number of completed surveys, and N is the number of employees eligible to participate.

Response rates: departments and agencies
Department Completed surveys Valid sample records Response rate
Accessibility Standards Canada 28 67 41.8%
Administrative Tribunals Support Service of Canada 276 656 42.1%
Agriculture and Agri-Food Canada 1 653 5 929 27.9%
Atlantic Canada Opportunities Agency 288 610 47.2%
Canada Border Services Agency 4 643 1 7356 26.8%
Canada Economic Development for Quebec Regions 202 409 49.4%
Canada Energy Regulator 219 585 37.4%
Canada School of Public Service 274 681 40.2%
Canadian Grain Commission 179 456 39.3%
Canadian Heritage 796 1 907 41.7%
Canadian Human Rights Commission 136 310 43.9%
Canadian Intergovernmental Conference Secretariat 6 27 22.2%
Canadian Northern Economic Development Agency 47 113 41.6%
Canadian Radio-television and Telecommunications Commission 236 562 42.0%
Canadian Space Agency 308 977 31.5%
Canadian Transportation Agency 159 345 46.1%
Civilian Review and Complaints Commission for the Royal Canadian Mounted Police 47 96 49.0%
Copyright Board of Canada 11 25 44.0%
Correctional Service Canada 4 776 19 365 24.7%
Courts Administration Service 266 771 34.5%
Crown-Indigenous Relations and Northern Affairs Canada 720 2 134 33.7%
Department of Finance Canada 366 900 40.7%
Department of Justice Canada 1 794 5 453 32.9%
Employment and Social Development Canada 14 249 36 001 39.6%
Environment and Climate Change Canada 2 799 8 277 33.8%
Farm Products Council of Canada 4 11 36.4%
Federal Economic Development Agency for Northern Ontario 53 83 63.9%
Federal Economic Development Agency for Southern Ontario 141 296 47.6%
Financial Consumer Agency of Canada 93 210 44.3%
Fisheries and Oceans Canada 3 885 15 022 25.9%
Global Affairs Canada 2 331 6 529 35.7%
Health Canada 3 155 9 744 32.4%
Immigration and Refugee Board of Canada 850 2 423 35.1%
Immigration, Refugees and Citizenship Canada 3 807 12 129 31.4%
Impact Assessment Agency of Canada 228 493 46.2%
Indian Oil and Gas Canada 38 78 48.7%
Indigenous Services Canada 2 181 7 655 28.5%
Infrastructure Canada 491 1 503 32.7%
Innovation, Science and Economic Development Canada 2 299 6 379 36.0%
International Joint Commission 16 44 36.4%
Library and Archives of Canada 467 1 066 43.8%
Military Grievances External Review Committee 26 47 55.3%
National Defense 7 032 27 594 25.5%
Natural Resources Canada 1 488 5 035 29.6%
Office of the Chief Electoral Officer 425 1 110 38.3%
Office of the Commissioner for Federal Judicial Affairs Canada 18 62 29.0%
Office of the Commissioner of Canada Elections 19 38 50.0%
Office of the Commissioner of Lobbying of Canada 17 27 63.0%
Office of the Commissioner of Official Languages 63 148 42.6%
Office of the Correctional Investigator 12 35 34.3%
Office of the Information Commissioner of Canada 57 109 52.3%
Office of the Privacy Commissioner of Canada 76 207 36.7%
Office of the Public Sector Integrity Commissioner of Canada 15 31 48.4%
Office of the Secretary to the Governor General 66 212 31.1%
Office of the Superintendent of Financial Institutions 573 1 331 43.1%
Pacific Economic Development Canada 65 192 33.9%
Parole Board of Canada 213 576 37.0%
Patented Medicine Prices Review Board Canada 42 85 49.4%
Prairies Economic Development Canada 181 364 49.7%
Privy Council Office 405 1 244 32.6%
Public Health Agency of Canada 1 118 3 840 29.1%
Public Prosecution Service of Canada 381 1 112 34.3%
Public Safety Canada 604 1 587 38.1%
Public Service Commission of Canada 427 906 47.1%
Public Services and Procurement Canada 5 776 17 970 32.1%
Registrar of the Supreme Court of Canada 64 211 30.3%
Royal Canadian Mounted Police 3 537 12 234 28.9%
Royal Canadian Mounted Police External Review Committee 13 28 46.4%
Secretariat of the National Security and Intelligence Committee of Parliamentarians 4 8 50.0%
Shared Services Canada 2 346 9 506 24.7%
Statistics Canada 2 328 5 814 40.0%
Transport Canada 2 326 6 742 34.5%
Transportation Safety Board of Canada 93 225 41.3%
Treasury Board of Canada Secretariat 918 2 588 35.5%
Veteran Affairs Canada 1 241 3 698 33.6%
Veterans Review and Appeal Board 39 134 29.1%
Women and Gender Equality Canada 192 459 41.8%
Total 86 717 273 186 31.7%

7.2 Survey errors

The survey is subject to various survey errors that may occur at almost every phase of a survey operation. Respondents may misunderstand instructions or make errors in answering questions; the answers may be incorrectly entered on the questionnaire and errors may be introduced in processing and tabulating the data.

Over a large number of observations, randomly occurring errors will have little effect on estimates derived from the survey. However, errors occurring systematically will contribute to biases in the survey estimates. Considerable time and effort were taken to reduce non-sampling errors in the survey. Quality assurance measures were implemented at each step of the questionnaire development, data collection and processing cycle to monitor the quality of the data. These measures include survey pre-testing to detect problems of questionnaire design or misunderstanding of instructions, highly tested computerized questionnaire applications, procedures to ensure that data capture errors were minimized, and edit quality checks to verify the processing logic.

7.3 Data collection

A description of the survey objectives was provided to the respondents on the survey login page. The Advanis survey support website also provided links to the Government of Canada survey information page as well as contact information in case respondents had any questions for the PSC.

The Advanis support team (accessed through a toll-free number and email address) provided support for participants who had questions during collection or needed technical assistance.

7.4 Data processing

Data processing of the survey was done in a number of steps including verification, editing, estimation and confidentiality. At each step, a copy of the output files is kept, and an easy verification can be made comparing files at the current and previous step.

7.5 Non-response

There is the potential for non-response bias in that response rates varied by department and agency and for certain demographic characteristics (for example, response rates for younger employees tend to be lower than for older employees). To adjust for this possible source of error, Advanis weighted the data by age, sex and department.

The extent of non-response varies from partial non-response (failure to answer just one or some questions) to total non-response. Total non-response occurred because employee contact information from the department or agency was not obtained, the contact information was incorrect, the respondent had problems accessing the electronic questionnaire, or the respondent refused to participate in the survey. Total non-response was handled by adjusting the weight of employees who responded to the survey to compensate for those who did not respond.

In most cases, item non-response to the survey occurred when the respondent did not understand or misinterpreted a question, refused to answer a question, or could not recall the requested information. For item non-response, a “Not stated” code was assigned to the item.

7.6 Measurement of sampling error

Although all eligible employees were included in the sample for the 2023 Staffing and Non-Partisanship Survey, because of the levels of non-response, one can consider the respondents as a sample of the population. Under this assumption, a measure of sampling error can be calculated that can be used in producing quality indicators. For the purposes of the 2023 survey, the term “sampling error” will be used interchangeably with “survey error.”

This section of the documentation outlines the measures of sampling error that users producing estimates from this microdata file should also use.

The basis for measuring the potential size of sampling errors is the standard error of the estimates derived from survey results.

However, because of the large variety of estimates that can be produced from a survey, the standard error of an estimate is usually expressed relative to the estimate to which it pertains. This resulting measure, known as the coefficient of variation (CV) of an estimate, is obtained by dividing the standard error of the estimate by the estimate itself and is expressed as a percentage of the estimate.

For example, suppose that one estimates that 52% of federal public servants participated in a staffing activity and this estimate is found to have a standard error of 0.005. Then the coefficient of variation of the estimate is calculated as:

Formula: 0.005 divided by 0.52 multiplied by 100% equals 0.96%

Determining the quality of the estimates and calculating coefficients of variation (CV) for the survey requires the use of the bootstrap method of variance estimation.

8. Weighting

Although all eligible employees were included in the sample for the 2023 survey, because of the levels of non-response it was necessary to weight the responding units by department or agency so that they were representative of the target population. The weight adjusts for the non-response within gender and age groups, within each department and agency. There is one weight for each record.

8.1 Non-response adjustment

Total non-response can be a major source of non-sampling error in many surveys, depending on the degree to which respondents and non-respondents differ with respect to the characteristics of interest. In the survey, total non-response occurred when an eligible employee did not participate in the survey. In 2023, 31.7% of the eligible employees in the survey population responded to the survey. Non-response groups were formed using sex and age groupings to compute non-response weight adjustment factors for each department and agency.

Weights were calculated using the ratio of the population and respondent counts for each age and gender combination, within each department and agency. Where age and/or gender were unknown or non-binary, values were randomly assigned based on the overall age and gender proportions within all participating departments and agencies. When age and gender population were missing, weights were developed by combining with age groups that were older if possible (and younger if not possible). A weight was then calculated for each respondent.

The purpose of adjusting for non-response is to account for the under-representation of age and gender groups in the respondent group by aligning to the population counts. In this way, the respondent group will better represent the department’s or agency’s composition of age and gender groups.

8.2 Post-stratification adjustment and final weight

Post-stratification is one of the calibration estimation techniques widely used in social surveys. The weights produced for the 2023 survey were calibrated to the number of employees in the public service population at each department or agency. Post-strata were additionally defined using age and gender information.

For each response in department or agency i and non-response group j, the weight Variable w with subscripts i and j is equal to:


Formula: w with subscripts i and j equals uppercase N with subscripts i and j divided by lowercase n with subscripts i and j

where:

Variable uppercase N with subscripts i and j is the number of public service employees in department or agency i and non-response group j, and

Variable lowercase n with subscripts i and j is the number of respondents in department or agency i and non-response group j.

The total weighted count of respondents in a non-response group adds up to the public service population of that group. For example, the total weighted count of the respondents from the female, 25 to 29 age group at Employment and Social Development Canada adds up to the total count of all females aged 25 to 29 in that department.

9. Appendix

Appendix A: Questionnaire

2023 Staffing and Non-Partisanship Survey