Written evidence from The Cabinet Office (CSP12)
Public Administration and Constitutional Affairs Committee
Civil Service People Survey inquiry
INTRODUCTION
SURVEY DESIGN
What is best practice for ensuring respondent anonymity in workplace staff surveys?
● Assessment against the NCSC standards
● Cloud Security Principles
● ISO27001 certificate and Statement Of Applicability
● Penetration test reports
● Web application and External Network and remediation plan for their penetration test
● Cyber Essentials certificate
● Internal risk register summary
● Details of clearances for staff who can access data
What is the best design and method for delivering a staff survey – in particular, across multiple organisations/divisions/Departments?
● The scope of the survey - whether it should focus on employee engagement or employee experience;
● Coverage of the survey - whether there should be a census or a sampling approach, and whether wider public sector organisations should be permitted to participate
● Timing and frequency of the survey
● Outputs - the type and content of reporting products, and publication of results
● Delivery - pros and cons around provision via an independent supplier vs in-house delivery
● The Question Set
● Accountability
● Benchmarking
● Open comments questions and suppression rules
● Taking action and sharing good practice.
● core attitudinal questions - these are asked to all respondents, and cover perceptions and experiences of working for a Civil Service organisation; future intentions to stay or leave; awareness of the Civil Service Code (the standards of behaviour expected), Civil Service Vision; experiences of discrimination, bullying and harassment; and ratings of individual subjective wellbeing. The core attitudinal questions also include an opportunity to provide free-text comments and the five questions that are used to calculate the survey’s headline measure, the ‘Employee Engagement Index’.
● local optional attitudinal questions - a bank of standardised questions on specific topics that each participating organisation can select (up to three blocks per organisation). Themes covered include: safe to challenge; health and safety; performance management; change management; support for managers; continuous learning; career progression; customer service; my organisation; and collaboration and team performance.
● demographic questions - these questions collect information from respondents about their job and personal characteristics, such as their working location and age. These questions are used to filter and compare results within organisations by different demographic characteristics so that results can be better understood and action targeted appropriately. The vast majority of demographic questions are standardised across the survey to enable analysis not only at organisation-level but also across the Civil Service; the demographic questions are also harmonised with the Office for National Statistics (ONS) and the Government Statistical Service (GSS) standards.
● topical/test questions - these questions focus on the “newest” themes, which usually align with current government priorities. These are not long-standing questions, but offer a cross-government perspective on relevant issues (e.g. benefits and challenges of working in different locations) and the ability to test/ validate new questions. Each question block is asked to a random sample of approximately 14% of respondents.
Aspect | Question | Rationale |
Pride | I am proud when I tell others I am part of [my organisation] | An engaged employee feels proud to be associated with their organisation, by feeling part of it rather than just ‘working for’ it. |
Advocacy | I would recommend [my organisation] as a great place to work | An engaged employee will be an advocate of their organisation and the way it works. |
Attachment | I feel a strong personal attachment to [my organisation] | An engaged employee has a strong, and emotional, sense of belonging to their organisation. |
Inspiration | [My organisation] inspires me to do the best in my job | An engaged employee will contribute their best, and it is important that their organisation plays a role in inspiring this. |
Motivation | [My organisation] motivates me to help it achieve its objectives | An engaged employee is committed to ensuring their organisation is successful in what it sets out to do. |
Please note that [my organisation] is a variable term, so in the actual survey it is replaced by the name of the organisation the respondent is part of e.g. DWP, Cabinet Office etc.
Theme | Question |
My work | I am interested in my work I am sufficiently challenged by my work My work gives me a sense of personal accomplishment I feel involved in the decisions that affect my work I have a choice in deciding how I do my work |
Organisational objectives and purpose | I have a clear understanding of [my organisation’s] objectives I understand how my work contributes to [my organisation’s] objectives |
My manager | My manager motivates me to be more effective in my job My manager is considerate of my life outside work My manager is open to my ideas My manager helps me to understand how I contribute to [my organisation’s] objectives Overall, I have confidence in the decisions made by my manager My manager recognises when I have done my job well I receive regular feedback on my performance The feedback I receive helps me to improve my performance I think that my performance is evaluated fairly |
My team | The people in my team can be relied upon to help when things get difficult in my job The people in my team work together to find ways to improve the service we provide The people in my team are encouraged to come up with new and better ways of doing things |
Learning and development | I am able to access the right learning and development opportunities when I need to Learning and development activities I have completed in the past 12 months have helped to improve my performance There are opportunities for me to develop my career in [my organisation] Learning and development activities I have completed while working for [my organisation] are helping me to develop my career |
Inclusion and fair treatment | I am treated fairly at work I am treated with respect by the people I work with I feel valued for the work I do I think that [my organisation] respects individual differences (for example cultures, working styles, backgrounds, ideas, etc.) |
Resources and workload | I get the information I need to do my job well I have clear work objectives I have the skills I need to do my job effectively I have the tools I need to do my job effectively I have an acceptable workload I achieve a good balance between my work life and my private life |
Pay and benefits | I feel that my pay adequately reflects my performance I am satisfied with the total benefits package Compared to people doing a similar job in other organisations I feel my pay is reasonable |
Leadership and managing change | Senior managers in [my organisation] are sufficiently visible I believe the actions of senior managers are consistent with [my organisation]’s values I believe that [my organisation's] senior leaders have a clear vision for the future of [my organisation]. Overall, I have confidence in the decisions made by [my organisation]’s senior managers I feel that change is managed well in [my organisation] When changes are made in [my organisation] they are usually for the better [My organisation] keeps me informed about matters that affect me I have the opportunity to contribute my views before decisions are made that affect me I think it is safe to challenge the way things are done in [my organisation] |
Note - [my organisation] is a variable term replaced in questions by the name or acronym of each participating organisation.
38.1. Since it started in 2009, the Civil Service People Survey has adopted a census based approach, which means that the entire Civil Service workforce is invited to take part. Each participating organisation builds a hierarchy with the headcount for each team so that we can track response rates at a team, organisation and Civil Service level. This is not, however, the only available method.
38.2. There are two main approaches that can be used for conducting surveys based on the research questions and needs: census based and sample based.
38.3. The census approach aims to count everyone in a given population, so it is a ‘complete enumeration’ of the population in analysis. A census is used when we want accurate information for many subdivisions of the population. Such a survey usually requires a very large sample size and often a census offers the best solution.
38.4. In the sample-based survey only part of the population is approached for information on the topic under study, and it is a partial enumeration because it is a count from part of the population. These data are then 'expanded' or 'weighted' to make inferences about the whole population. Sampling methods can be divided into two main categories: probability sample - when each population element has a known chance of being chosen for the sample; and non-probability sample - when we basically do not know the probability that each element of the population will be chosen. A further categorisation is between convenience sample - when individuals who are easy to reach are chosen; and voluntary sample - when individuals select themselves by responding to a general invitation.
38.5. There are five common types of sampling:
● Simple random: each member of the study population has an equal probability of being selected;
● Systematic: each member of the study population is either assemble or listed, a random start is designated, then members of the population are selected at equal intervals;
● Stratified: each member of the study population is assigned to a group or stratum, then a simple random sample is selected from each stratum;
● Cluster: each member of the study population is assigned to a group or cluster, then clusters are selected at random and all members of selected clusters are included in the sample;
● Multistage: clusters are selected as in the cluster sample, then sample members are selected from the cluster members by simple random sampling. Clustering may be done at more than one stage.
38.6. Depending on the approach chosen, there are both advantages and disadvantages:
| Advantages | Disadvantages |
Census based | - There are no sampling errors - From an ethical perspective it ensures that everyone can anonymously share their view on sensitive topics (e.g. bullying and harassment) - Provides a true measure of the population - Allows for benchmark and trend comparison - Data for small areas may be available, assuming satisfactory response rates are achieved - Data for sub-populations may be available, assuming satisfactory response rates are achieved - (Because of the above reasons) detailed cross-tabulations may be possible | - Higher costs, both in monetary terms and staffing requirements - Longer time needed to collect, process the data and produce the results - May be difficult to enumerate all units of the population within the available time |
Sample based | - Reduced cost - both in monetary terms and staffing requirements - Reduced time needed to collect and process the data and produce results as it requires a smaller scale of operation - Results could be made available more quickly due to less data cleaning required, but time may be required to make inference to the whole population | - Data on sub-populations (such as a particular ethnic group or gender identity) may be too unreliable to be useful - Data for small geographical areas also may be too unreliable to be useful - Detailed cross-tabulations may not be statistically valid - Estimates are subject to sampling error which arises as the estimates are calculated from a part (sample) of the population - May have difficulty communicating the precision (accuracy) of the estimates to users - Not all employees have an opportunity to have their say |
38.7. When measuring employee engagement, it seems that the tendency is to adopt a census-based approach in the long term. For example, both the NHS and US Federal survey started as a sample based survey and then moved to a census survey (with a population of 1.3 and 1.5 million people respectively). The Australian, Canadian, Irish and New Zealand surveys have a census-based approach as well.
38.8. Although a census-based approach is quite common among other countries which measure employee engagement in the public sector, it seems that there is a mixed picture in relation to the frequency of employee engagement surveys (see table below).
Annual | Triennal | Other |
UK US (since 2010 [biennial 02-08]) Australia New Zealand Latvia Northern Ireland Canada (considering triennial as in 99-17) [also NHS & UK armed forces surveys] | Belgium Denmark Norway Switzerland (with shorter survey to a sample annually) Ireland (considering frequency) | Luxembourg (census in 2017 and 2023 - smaller surveys delivered on demand) |
38.9. Running an annual survey has the main limitation of being retrospective, capturing how employees felt in a relatively wide time span. There are even further limitations if an organisation measures employee engagement less frequently, for example every three years. These include the risk of: i. losing the experience of team members; ii. not being able to capture the actual reasons for variations in the scores because of the wide time frame; iii. employees not feeling listened; iv. having outdated scores; v. not having actionable results and an effective strategy plan to address feedback.
38.10. Many of the organisations who participate in the Civil Service People Survey run pulse surveys and listening activities in between the annual survey to measure progress on the actions they are taking. They often also use this as an opportunity to pose departmental specific questions. In a large organisation these can be run regularly with different samples of staff. In small and medium sized organisations there is a risk that the same people are constantly asked to participate.
38.11. It is important to evaluate the impact on staff time and survey fatigue. If staff are frequently required to complete surveys, this may become challenging both from a cost and practical perspective, especially for those colleagues in operational and delivery roles. It may also lead to survey fatigue, which is the lack or loss of interest in completing a survey. There are two main types of survey fatigue: i. Pre-response: this is caused by overwhelming potential respondents and discouraging them from giving feedback since it is asked too often; ii. Survey-taking: it occurs when a respondent is completing a survey and triggered by some elements of it, e.g. its length, question type, overall design. The main outcome of survey fatigue is respondents not completing a survey or abandoning it, leading to an inability for an organisation to collect insightful feedback.
38.12. All those points were discussed during the strategy workshops we conducted over the summer on the future of the Civil Service People Survey. Among the members there was full agreement that the:
● Civil Service People Survey should continue to measure employee engagement as well as its drivers and outcome;
● surveys should continue to be census-based by default in order to allow all employees to have their say; a sample-based approach may be used for additional pulse surveys to follow up on specific topics;
● surveys should continue to be annual as a minimum;
● presentation of the findings should be tailored to the audience.
Who should be involved in the design and development of large-scale surveys like the People Survey?
● topics: organisations wanted to keep the majority of the questions the same in order to track changes in employee attitudes and experiences over time;
● local questions: each organisation would now be able to select three blocks from a standard list - instead of adding their own local questions - to facilitate cross-government comparison and simplify the process. This streamlined the previous approach where each organisations could submit their own question (up to 1000 questions to manage centrally);
● test/topical questions: a review of the topics covered was suggested to improve their usability and reduce their complexity;
● logos and local terminology: it was recommended to use only the Civil Service logo and align to the GOV.UK design template, as well as adopt a standard definition for senior managers and leadership team;
● digital and paper surveys: it was decided to continue for the survey to be digital by default while offering paper surveys to organisations only if there is a clear operational need for them (e.g. for prison officers).
● Organisations found the open text questions useful for further analysis and the production of topical packs for leaders
● The benchmark and the demographic publications were mainly used only for internal purposes instead of external reference
● Organisations were overall satisfied with the 2021 Dashboard and their data visualisations
● We also received open comments on the main changes organisations suggested for our products such as our dashboards and publications.
How relevant are the metrics included within the People Survey? Are there additional topics or metrics that would add value to Departments in collecting the information sought?
● core attitudinal questions (asked to all respondents);
● local optional attitudinal questions (asked if an organisation selects them);
● demographic questions (asked to all respondents);
● topical/test questions (asked to a random sample of respondents).
Type of Questions | Topics covered |
Core questions | My work Organisational objectives and purpose My manager My team Learning and development Inclusion and fair treatment Resources and workload Pay and benefits Leadership and managing change Engagement Taking action Organisational culture Civil Service reform Intention to stay The Civil Service Code Discrimination Bullying and Harassment Open comments question (what change organisation should make in next year) Health and Wellbeing |
Demographic questions | Location & hybrid working Time in role / organisation / Civil Service Manager Working pattern Employment status Type of work Cross government professional community Career development scheme Sex / Gender Age band Ethnicity Health Sexuality Religion Grade Socio-economic background |
Local question blocks (organisations can choose up to 3 blocks with around four questions in each block) | My organisation Safe to challenge Change management Health and safety Performance management Support for managers Continuous learning Career progression Collaboration and team performance Customer service |
Test/topical question blocks (respondents are randomly allocated a block of around five questions) | Inclusion and Fair Treatment OECD international comparison questions Wellbeing Benefits and challenges of locations Use of data Talent Diversity |
Aspect | Question |
Work engagement | Overall, I am satisfied with my job My job inspires me The work I do gives me a sense of accomplishment |
Organisational engagement | I feel a strong personal attachment to my organisation I identify with the mission of my organisation; and one on public |
Public service motivation | It is important to me that my work contributes to the common good
|
SURVEY DELIVERY
How does the Cabinet Office work with other Government Departments to administer the survey and gather responses?
● local grades (and their Civil Service equivalents to allow cross-CS comparisons);
● local question block choices;
● the hierarchy (family tree) of teams;
● the mapping of teams to the previous survey for comparisons over time (trend);
● translations if required (e.g. into Welsh);
● reviews of paper surveys if required (e.g. for prison officers); and
● tests of the online survey.
How is the survey delivered within individual Departments, and how are respondents encouraged to take part within those Departments?
80.1. A separate team based in HR is in charge of delivering the survey within the department. Cabinet Office respondents are encouraged to take part in the People Survey every year through a variety of means.
● People Survey Champions are appointed to represent each Business Unit, whose role is to liaise between People Survey managers and their Business Unit staff to deliver promotional messages, encourage completion of the survey, and communicate any changes within team structures.
● In advance of the survey launching staff are reminded of the action taken on the back of the previous survey - the People Survey Action Plan.
● Once the survey has launched a range of routes are deployed to encourage completion including the development of a People Survey hub page with promotional blogs, a countdown to the end of the survey, and a regularly updated list of response rates for each Business Unit and messages from Permanent Secretary and Chief Operating Officer
80.2. Below is the communications plan for the 2021 survey in the Cabinet Office which is typical of a plan used in organisations. This communications plan helped the Cabinet Office reach a 76% response rate in 2021.
2021 People Survey communications plan in Cabinet Office | |
13-28 Sep | Promote the survey via People Survey Champions in all Business Units |
28 Sep | CS COO message to whole Civil Service will be sent out to all Cabinet Office staff by email and shared on our intranet |
Cabinet Office launch message from department COO Sarah Harrison posted on intranet, pointing towards the work undertaken in response to last year’s feedback and encouraging people to participate | |
Email banner on intranet and dedicated People Survey Hub | |
Digital screen messages across CO buildings encouraging taking part | |
People Survey Champions in directorates and teams will send out a message containing their specific team codes to fill in the survey. While the survey is live team leaders ask their staff to complete it | |
4, 11, 18, 25 Oct | Weekly update at Cabinet Office Monday Morning Meeting on response rate levels |
Throughout Oct | Response rates league table on intranet and regular messages to Senior Civil Servants showing response rates for each Business Unit and updates in all-staff email |
‘I’ve taken part’ pictures on our digital screens and intranet of people who have filled in the survey, highlighting a couple of issues important to them. Email banners created and shared that people can add to their signatures, to encourage others to take part. | |
29 Oct | Countdown clock on intranet and digital screens - one week left to participate |
1 Nov | Final all-staff email with a message from Cabinet Office Permanent Secretary highlighting response rates and encouraging people to fill the survey in |
Post-survey |
|
4 Nov | Message of thanks to those who have filled in survey, reminder of when results will come out, info on how they will be accessed |
Example email banner for staff to use. Posters are also available.
81.1. A similar approach is also adopted in other organisations, such as DWP, where the Permanent Secretary writes to all colleagues on the day the People Survey opens reflecting on the Department’s journey over the last year and encouraging participation. There is a permanent survey link to the DWP People Survey intranet pages (link to the survey, list of team codes, FAQs, previous results, message from the Permanent Secretary etc.) on the intranet homepage throughout the survey period and there are weekly reminders in the all-colleague email “DWP Connect”. The survey is promoted and discussed at national leaders calls (for example, directors and SCS calls). Directors General will also highlight the People Survey in directorate comms.
81.2. Below this level local managers and leaders distribute comms as they see fit, with advice from HR Business Partners. Examples Include: bespoke email shots, including a link to the People Survey Team code; reminders in Service Delivery Line Managers Update (SDLMU). The SDLMU is used to highlight important information to operational managers that they may need to share with their team during meetings; local newsletters, all-colleague calls and senior leaders’ updates are other examples of communications channels utilised.
81.3. Unique comms approaches are taken where possible. In 2021 a member of the People Insight Team was the feature of a “Day in the life of” intranet headline news article to give a sense of what goes on behind the survey and to encourage participation.
81.4. In the months leading up to the People Survey DWP also recommend a “You said, we did” approach to communications, encouraging local leaders to link actions taken to colleague feedback. This approach seems to encourage participation and is taken throughout the year, not just during the period of the survey, so that colleagues are regularly reminded of the importance of their feedback.
81.5. Finally, completion rates are monitored throughout the People Survey, both centrally by the People Insight Team and by local survey managers (LSMs). National response rates are used to determine if further communications are required, or if particular drops in response might highlight a problem with access. LSMs communicate with their directors and senior leaders to help push local comms or highlight particular areas where responses are low. This could lead to, for example, a leader visiting a site to help encourage completion.
82.1. Within the Ministry of Defence (MoD), all Civil Servants, Military staff who line manage Civil Servants and all Military staff in Head Office are invited to participate in the Civil Service People Survey. MoDTop Level Budgets (TLBs) make the decision if they wish to invite Contractors to share their views. The People Survey is coordinated centrally through the Head Office Engagement Policy team and delivered in collaboration with a Reference Group of Engagement Leads based in MoD Head Office and TLBs.
82.2. Recognising the scale and complexity of the MOD, there is not a single mandated approach to engagement activity across the Department. TLBs include the business areas within the MOD run by Directors General and Chiefs of our Front Line Commands. The Engagement Policy team in the Head Office act as SPOCs for the Cabinet Office, facilitate corporate activity and coordinate Engagement Leads who are based in each TLB. Engagement Leads drive forward local activity which will have the greatest impact on engagement for that area.
82.3. Five of the Enabling Organisations and Arms Length Bodies take a similar local approach to engagement activity but work independently and deliver their own Civil Service People Surveys. They include Defence Equipment & Support, Submarine Delivery Agency, Defence Science and Technology Laboratory, the UK Hydrographic Office and Defence Electronics and Components Agency. We share best practice with these areas.
82.4. During the pre-survey period the Head Office Engagement Policy team communicates with relevant Senior Boards and Trade Unions, providing information about any updates to the annual People Survey. They also engage with Senior Leaders, ensuring they are sighted on the survey timeline, and they fully understand their responsibility for encouraging survey participation in their area.
82.5. During the survey live period:
● Articles are published on intranet from the start of the survey month, highlighting that the People Survey is coming, reminding colleagues of what action has been taken through the year, informing staff of how to participate in the survey when it is live, and reminding them of the value of sharing their views. Other tools like blogs, banners, digital screens and FAQs are used through the survey period to keep the survey visible and easy to complete.
● Direct emails from the Permanent Secretary are shared at the start and end of the survey window encouraging participation in the survey and highlighting how senior leaders pay attention to the survey results. A direct email from the Cabinet Secretary are shared with all Civil Servants to encourage participation.
● Local communication plans are also developed, delivered and sponsored by local leaders, utilising both digital and physical tools and taking into account the variety of environments our staff work in across the country.
● Response rates are shared locally with Senior Leads, HRBPs and Engagement Leads through the survey period to highlight areas where further communications might be needed and to spotlight areas of good practice in survey communications.
82.6. Once the survey closes,final response rates and the timeline for results are shared with Senior Boards, HRBPs and Engagement Lead Reference Group. The Defence Communications colleagues conduct a review of the impact of various communication methods to understand how improvements to the communications approach can be made for next year’s People Survey.
83.1. Within HM Revenue and Customs (HMRC) the survey as an end-to-end process is delivered through a programme of work:
● scoping: roles and responsibilities, resources, timeline, governance, risks, issues and dependencies, Memorandum of Understanding with Valuations Office Agency are identified;
● survey-build: the HMRC team works both externally with Cabinet Office and internally with stakeholders and delivery partners to ensure all requirements are designed in and delivered to time;
● survey-live: as part of the communications approach, HMRC liaises with Cabinet Office and internal delivery partners to monitor and promote completion of the survey with HMRC people and other more senior stakeholders.
83.2. In relation to the comms, HMRC communication objectives are to i. ensure colleagues know when and how to complete the survey and to proactively address any issues they may havel; ii. encourage completion by demonstrating the positive impact surveys have had; iii. encourage managers to support their teams to complete the survey – particularly around setting aside time in operational areas.
83.3. HMRC communication approach is to:
● focus on encouraging colleagues to complete the survey;
● be clear on the practicalities: what colleagues need to do and when and how long it will take;
● showcase the benefits: demonstrate that engagement with the People Survey has a direct impact on making HMRC a great place to work, signposting to a page of stories (subject to policy support);
● use leaders across HMRC to deliver clear, human and transparent engagement with colleagues.
● encourage managers to make time for their teams to complete the survey;
● create content under ‘people survey’ visuals – including intranet banner and weekly HMRC-level and group totaliser;
● use a cross-channel approach with targeted communications across central, group and directorate channels to reach and engage colleagues at different levels of the organisation;
● work with and through key diversity networks to encourage completion;
● encourage colleagues to share on Yammer when they have completed the People Survey using CountMeln topic with leaders role modelling;
● engage with Departmental Trade Unions ensure they understand HMRC approach and flag any issues on the ground which could erode trust.
84.1. In 2021, 101 organisations took part in the survey. A total of 546,261 people were invited to take part and 327,388 did so, an overall response rate of 62%.
84.2. Half of participating organisations in 2021 achieved a response rate of 76% or greater.
84.3. The table below shows how the overall response rate of 62% compares to selected other public sector engagement surveys. Among the surveys included in the list we are the third largest survey, behind the US and the NHS surveys with respectively 1.5M and 1.3M headcounts:
Survey (date) | Response rate |
|---|---|
34% | |
37% | |
48% | |
52% | |
61% | |
62% | |
63% | |
65% | |
77% |
84.4. Response rates vary greatly across Civil Service People Survey organisations from 21% in the Scottish Prison Service to 100% in the Government Property Agency. This wide variation is also seen across other nations. While the United States federal survey had an overall response rate of 34% in 2021, the response rate in participating organisations varied from 16% to 94% and in Australia, which had an overall response rate of 77% in 2021, response rates varied from 24% to 100%.
84.5. The low response rates seen in the Scottish Prison Service (21%) and HM Prison Service (HMPS, 23%) are partly explained by the unique challenges these organisations face (the Ministry of Justice will cover the complex challenges faced by HMPS, and the impact these have on participation in the People Survey, in a separate submission to the Committee). To encourage engagement with the survey, HM Prison Service are supplied with paper surveys for staff to use.
84.6. With regard to the Scottish Prison Service (SPS), analysis has indicated a number of factors influencing response rates, including the continued delivery of services for those in custody during the COVID pandemic. SPS has committed to increasing response rates. A range of activities will be undertaken as part of an organisational wide promotion campaign designed to promote the benefits of participation for employees.
84.7. These issues with engaging staff working in prisons in the survey are not, however, unique to the UK prison services. For example, the US Department of Justice (including the Bureau of Prisons) response rate in 2021 was 27%.
84.8. The Cabinet Office has reached out to the Queensland Corrective Services survey manager and will share any advice received with the UK prison services on how their peers in Australia received a 49% response rate in 2020.
What are the strengths/weaknesses of delivering the survey in the current manner?
● It is likely to be the most cost efficient way - especially in the short-term, realising economies of scale.
● It is likely that an external provider has proven their capacity and expertise to deliver a project with this degree of complexity.
● In the event of re-procuring the same provider, the knowledge of building the survey would be retained.
● It is more likely that an external provider would guarantee the delivery of the local departmental requirements such as the provision of the paper surveys and translation into foreign languages, therefore having the broadest reach in giving all civil servants the greatest opportunity to participate.
● It is less likely that the delivery of the survey will be affected by potential lack of internal resources within the Civil Service.
● We would not automatically own the Intellectual Property of any survey platform or analytical tools developed and used. This has previously caused some difficulties, in particular when an Analyst Tool from a third party provider was discontinued in 2019, leaving departments without an interactive tool to interrogate their results. This has been in part mitigated by adding clauses in the most recent contract (2020/21/22) and it will be looked at for the delivery of the 2023 survey.
● Buying an off-shelf product may reduce the likelihood that all our requirements (both essential and desirable) are met.
● There may be the risk that some suppliers may be excluded because they do not meet or pass the information assurance (security) standards or do not comply with GDPR, e.g. American based companies which store data outside of the UK/Europe.
● the current Civil Service People Survey provides a reliable way to collect employee lived experience;
● being run as a single exercise with a validated set of measures, the survey allows us to track changes over time and at departmental level, and make comparisons among relevant groups (e.g. grade);
● having a centrally managed contract is cost efficient and the best use of money; it optimises the number of resources needed for procuring a contract, purchasing a licence, building and delivering a survey;
● having a single survey with a census-based approach also guarantees that each civil servant has an equal opportunity to complete the survey regardless of the organisation they are part of;
● having a central team in charge of the delivery of the survey facilitates the communication among 100+ organisations and improves accountability on the delivery of the project.
● the timing of the survey may not necessarily be suitable for all organisations because it could conflict with own departmental priorities;
● there may be cases where the questions or topics asked in the survey may not be fully relevant for a specific organisation and the level of customisations requested may not be appropriate for practical or analytical reasons.
VALIDITY OF RESULTS
How can and should data collected through a staff survey be quality-assured?
What sources can this data be cross-referenced with?
Are the People Survey results considered to accurately represent the ‘lived experience’ of Civil Servants? Which areas are relevant or tend to vary?
● all the questions - with the exception of the team code one - are voluntary;
● the demographic questions include a ‘prefer not to say’ option;
● respondents are made aware of our suppression rules and of how their responses are going to be used and reported (e.g. who is going to have access to their individual responses; only aggregated findings where at least 10 responses have been received are going to be shared; etc.)
How does non-response bias impact results, i.e. How does the percentage of civil servants who choose not to respond to the survey impact the overall validity of the data?
APPLICATION OF RESULTS
How should organisations take forward and implement learning from staff surveys, both to learn and build on positive outcomes and take corrective action for negative outcomes?
What is considered best practice for taking forward and acting on employee feedback?
● strategic narrative: visible empowering leaders who can explain where the organisation is going;
● engaging managers: focus on their people, treat them as individuals and coach and stretch them;
● employee voice: all employees are seen as central to the solution of problems;
● integrity: values on the wall reflected in day to day behaviours.
● Pride in employer;
● Satisfaction with employer;
● Job satisfaction;
● Opportunity to perform well at challenging work;
● Recognition and positive feedback for one’s contributions;
● Personal support from one’s supervisor;
● Effort above and beyond the minimum;
● Understanding the link between one’s job and organisations’ mission;
● Prospect for future growth with one’s employer;
● Intention to stay with one’s employer.
● Increase organisation performance;
● Have better employee performance;
● Have higher employee retention and reduced employee turnover;
● Have higher job satisfaction;
● Increase revenue;
● Have a better customer experience;
● Have more innovative behaviours.
● Thank employees for taking part in the survey
● Distribute the results
● Explore the results individually
● Team leaders seek further insights if required e.g. differences by grade
● Talk about the results as a team
● Celebrate success e.g. rising scores because of actions taken since the previous survey
● Identify areas of opportunity for improvement and use the resources available to all organisations to begin to take action on these
● Monitor progress (e.g. via pulse surveys) and report back to the team
● Have senior leaders who set the example
● Showcase and share best practice
Note: the Civil Service Benchmark is the median score across the 101 organisations who participated in the 2021 survey. We use this measure as our benchmark because it prevents the results from being skewed by a small number of organisations who contribute the largest number of respondents.
What data from the People Survey are shared with departments following the delivery of the survey, and at what level of granularity?
133.1. The first set of internal reporting products are for Civil Service senior leaders. A summary of the key findings at Civil Service level is drafted for the Civil Service’s Chief Operating Officer and the Head of the Civil Service. The central team also creates what can be defined as a national report, basically a data pack, which is presented by the head of the team to the Permanent Secretary Heads of Government departments as well as circulated to HR Directors in main organisations. The Cabinet Office also produces other annual internal packs on key topics such as locations, health and wellbeing and diversity and inclusion.
133.2. Between 2009 to 2019, the former supplier used to produce a pdf report for each team who took part in the survey. The pdf included the main scores for the main themes and the comparison with the organisation and with the previous year. An example of the front page of a pdf highlight report received by participating organisations prior to 2020 can be found below.
133.3. Only survey managers and e.g. HR, have access to online tools from the supplier to analyse their results and to the comments left in open text questions.
Front of Cabinet Office 2017 survey report
133.4. Since 2020, Qualtrics, the current contractor for the Civil Service People Survey, has provided each organisation with access to three interactive, digital dashboards. These dashboards contain an increasing amount of data and analytical filters. Any staff member can use the Standard dashboard to see the overall scores for any team in their organisation and the difference from the previous survey. Team managers and leaders of multiple teams can use the Intermediate dashboard to compare teams and conduct simple analysis e.g. differences between grades or demographic groups.
133.5. The Advanced dashboard allows Human Resources to dive deeper into the organisation's results including responses to open comments questions. The dashboards have an export function, so results can be exported as needed. Each organisation can decide who within their organisation can access each of the dashboards. They can either create individual employee logins, or create a universal login to share with their workforce.
133.6. Release of the three types of dashboards - Advanced, Intermediate and Standard - is usually staggered over the course of a week with two main purposes: a. to not overload the system; b. to allow survey managers in each participating organisation time to prepare relevant briefing and comms ahead of the distribution to all staff (the Standard dashboard is launched last). The dashboards contain aggregate, anonymised survey data, and are GDPR and security compliant (only the advanced dashboard includes anonymised individual open comments).
133.7. Although we do not know the time spent on each visit to the dashboard, we do have some information on the total number of logins each day since launch. For example, both in 2020 and 2021 the daily logins peaked at almost 10,000 when the dashboard was launched across participating organisations.
133.8. An example of the headline findings from the Advanced dashboard:
133.9. An example of the responses to a specific question set where also one of the filters is selected.
133.10. As shown above, the dashboards contain the filters (e.g. job and personal characteristics) at the top; the themes on the left-hand side and the findings at the centre. At the top left corner users are able to select the team or work unit they are interested in (organisations define their own work units through work over the summer period, when survey managers in each organisation build or update a hierarchy of work units in the online survey platform before the survey launch).
133.11. The minimum number of respondents for a team to receive a report in the dashboard is 10 staff. In 2021, approximately 12,000 units received 10 or more responses and received a report and 3,000 did not receive enough responses in which case the data ‘moved up’ in the hierarchy to a ‘parent’ unit. Respondents identify their unit by inputting a unique code for each of approximately 15,000 units.
133.12. The dashboard offers team comparison, comparison by demographic, comparison with directorate and organisation level, as well as with previous years. Results are only displayed for work units or job/personal characteristics when there are ten or more respondents. Comments to the open text questions are only displayed when there are twenty or more from respondents. Respondents are asked not to write names in their comments but in case they do, a check is undertaken before the dashboards are released. Names or identifying characteristics are removed by the supplier.
133.13. Users of the Intermediate and Advanced dashboards can compare different job or personal characteristics in a unit as long as each received ten or more responses e.g. ten men and ten women. Examples of comparisons that can be made include: Civil Service Grade; Function or Profession / Job role; Country or Region / Town or City; Main place of work (office / home / both); Time in current job / organisation / Civil Service; Manager or non manager; Working pattern; Type of employment contract; Sex / Gender matches sex; Age; Ethnic Group; Has long-term condition or illness; Sexuality; Religion; Socio-economic background.
133.14. There is also a fourth type of dashboard - a cross-government version - which allows users to interrogate the full dataset at Civil Service level, but doesn’t permit analysis at team level, only at an organisational one. The purpose of this dashboard is to facilitate cross-government comparisons and it is mainly used by Civil Service HR, the diversity and inclusion communities, as well as heads of functions and professions.
133.15. A further internal product that the central Civil Service People Survey team distributes to all participating organisations is an internal benchmark file. This spreadsheet contains:
● The overall results for the Civil Service People Survey for that year;
● the overall results for each of 100+ participating organisations;
● a high-performers benchmark (Upper Quartile [top 25%]); and
● organisations grouped in six size benchmarks (e.g. less than 100 headcount or 6,000+ headcount) and the scores for each size benchmark.
133.16. This file is well-received by participating organisations because it facilitates cross-government comparisons, the drafting of internal briefing and action plans, as well as an opportunity to connect with similar organisations and share best practice.
133.17. Furthermore, for organisations with dedicated analytical support, there is the possibility to receive the microdata i.e. individual level responses, as well as the anonymised individual comments to the open questions included in the survey. This is subject to the signing of a data sharing agreement by a named analyst and a senior responsible officer. Transfer of the microdata uses secure cloud-based managed file transfer software.
133.18. The level of the granularity of the findings provided varies based on the different products - e.g. from being quite high level in the reports to senior level or in the standard dashboard which gets distributed to all staff, to becoming extremely detailed in products such as the advanced dashboards, the topical packs or in the microdata.
134.1. As external products, each year we publish on our public survey hub on GOV.UK two sets of data.
134.2. Civil Service benchmark results: a spreadsheet with mean and median scores for all respondents, plus overall scores for 100+ organisations and a technical guide to that years survey
134.3. Results by all demographic groups and five focus files with results by:
- Ethnicity
- Gender
- Health status
- Sexual orientation
- Socio-economic background
134.4. Statistics are published in an accessible ODS format. The technical guide is published in an accessible HTML format. Organisations are separately encouraged to publish their own results on their public website and many link to the survey hub to do so.
What action takes place within departments following the publication and sharing of results?
Survey | Question | Score % positive |
CSPS | I believe that senior managers in [my organisation] will take action on the results from this survey | 48% |
CSPS | Where I work, I think effective action has been taken on the results of the last survey | 35% |
US | I believe the results of this survey will be used to make my agency a better place to work. | 40% |
Northern Ireland | I believe that senior managers in my department will take action on the results of this survey | 36% |
Northern Ireland | Where I work I think effective action has been taken on the results of the last survey | 25% |
UK armed forces (AFCAS) | I believe the leaders in the [military] will take action on the results of AFCAS | 30% |
UK armed forces (AFCAS) | I think effective action has been taken in the [military] on the results of AFCAS | 26% |
138.1. The Executive Committee, People and Operations Committee and all Heads of Business Units are briefed on the key results, including demographic insights. Ministers and NEDs participate fully in this process with discussions at Cabinet Office Board on the results and action plans. Staff also have access to Cabinet Office-wide findings, guaranteeing transparency within the organisation.
138.2. Following the publication of People Survey results, a department wide People Survey Action Plan is developed in consultation with various stakeholders including senior leaders, staff networks and trade unions. Discussions on the findings are also held at directorate and team levels with each business unit expected to develop their own plans based on findings.
138.3. There are many tangible examples of improvements from the People Survey driving change and monitoring the impact in the Cabinet Office. For instance:
● In 2017 Cabinet Office kickstarted a review of its purpose off the back of fairly low scores relating to Objectives and Purpose (73%). Through the development of a new purpose, and carrying out organisation wide engagement through the Cabinet Office ‘Our Conversation’ programme, understanding of the department's purpose and how people’s work contributes to that improved by 4pp to 77% in 2018.
● In 2019 through interrogation of bullying, harassment and discrimination scores, we identified areas of best practice through high performing units, and targeted poor performing units for particular action. Within one Business Unit, scores in 2019 were high (16% discrimination, 22% bullying/harassment) targeted work was carried out with them including piloting an organisational change intervention which saw significant improvements in their scores year on year (in 2021 their scores were 9% discrimination and 12% bullying/harassment). A case study was developed from their experiences and shared at a best practice round table in May 2022 alongside other examples from Units who have low scores in this area.
● In 2021 scores for learning and development for those at EO grade increased from 55% to 58%, with 5pp gains in some key questions about learning opportunities after the launch of a new talent model and improved opportunities such as delivery of the ‘GROW’ career management programme for EOs and the introduction of Career Conversations for all staff. People Survey scores will also form a key success measure for an ambitious agenda for Cabinet Office to have the best learning offer in government and double the typical amount of time spent in Learning & Development.
● By having this comparable data the survey provides a means for senior leaders to be held accountable for people management in a consistent way in the Cabinet Office. People Survey scores relating to Leading and Managing Change, Bullying, Harassment and Discrimination and overall Engagement are considered as part of SCS performance management process and a mandatory SCS Leadership Objective sets out expectations in relation to taking action on the back of the survey insights.
139.1. In the case of DWP, the People Insight Team undertakes several regular activities in the weeks and months following the release of People Survey results:
● A day 1 download of results to the Permanent Secretary and People Capability and Place Director General on the day results are released to departments;
● A senior leaders (SCS) briefing, led by the Permanent Secretary, highlighting areas of celebration and concern in the results. This pack also includes actions to be taken by senior leaders to engage their colleagues with the feedback and start action planning. This occurs the day results are released to colleagues.
● An all-colleague call, using a slightly altered version of the pack shared with SCS, giving the Permanent Secretary the opportunity to walk colleagues through the results. This normally occurs the same day, or day after, the SCS call.
● The DWP Executive team receive a more in depth analysis of all results in the weeks following the release, and they use this as the basis for discussion on what needs to be focused on as a result of the survey.
● Around January, the Executive Team receive a second report highlighting key areas of focus and more in depth analysis. The content of this often depends on the results themselves, but is sometimes affected by wider events; for example, following the Black Lives Matter protests and national conversation, this pack focused on the experiences of BAME colleagues.
● The final part of DWP regular results work is each DWP Champion for a protected characteristic receives a bespoke report, highlighting the changes in results from the previous year to this one for the particular group they champion.
● Throughout the rest of the year, bespoke analysis can be requested from the People Insight Team and they use People Survey results in a great deal of their analytical work. Alongside this, monitoring of colleague wellbeing is done via an in house Pulse Survey.
● Below national level, approaches vary with the size of the team considered. Most directorates are large enough to take on a similar level of analysis to that of DWP as a whole, and this is generally a joint effort between directors general and HR Directors partnered with them. Many areas of DWP have representative groups known as People Groups, made up of colleagues of all grades but normally with support of a local manager / senior leader. These groups action plan and take forward results at a local level (for example Jobcentre level).
139.2. There are tangible examples of work undertaken as a direct result of the People Survey Results in DWP:
● Case study: In 2020, most health and wellbeing results had reduced compared to 2019, and new questions about the impact of Covid-19 showed negative effects on physical health (35% of respondents) and mental health (60% of respondents). DWP developed immediate and longer-term wellbeing interventions due to Civil Service People Survey 2020 data and wider supporting insight.
● The recent launch of a future vision and story for the department to 2025 was influenced heavily by colleagues’ feedback in the 2021 People Survey that they wanted a clear vision for the future.
● In Change and Resilience Group, the rollout of the Leadership Essential products demonstrates commitment to learning and growing line management capabilities in areas like performance management and developing the line management relationship.
● People, Capability and Place have been embedding Behaviour Matters and Micro Behaviours products on the back of scores in relation to bullying, harassment and discrimination and inclusive leadership.
● Finance Group (FG) have developed their own Leadership Essentials Refresh programme. This new programme runs alongside the DWP programme, using their material, but a few tweaks have been added to make it as relevant as possible for all FG colleagues. FG coaches are used to deliver the programme and they use examples in the sessions which are FG specific.
● Service Excellence has started running a “cultural inquiry” on the back of People Survey results to help understand the culture of Service Excellence and how best to move forward as a group.
140.1. On the release of the CSPS results for the MOD, analyst and HR colleagues work to develop an initial highlights results pack for senior boards, followed by a further in-depth presentation highlighting any trends, insights gathered from the survey and identifying priority areas for action, which will support the improvement of employee experience within the MOD. Deep dives into key priority areas are commissioned by Senior Boards and insight is fed into corporate activity owners to inform activity for the year ahead. Regular communication is maintained with corporate activity areas to ensure activity is progressing.
140.2. Additional activities include:
● An article from the Permanent Secretaries is published on intranet sharing the People Survey results. This is followed up by a later article reminding colleagues of engagement action taken since the last survey and highlighting any new action to be taken as a result of feedback from this survey.
● As part of the annual appraisal and objective setting round, MoD Permanent Secretaries look at the People Survey results of their Directors General to ensure staff engagement forms a part of the conversation.
● The Head Office Engagement Policy team meets quarterly with the Cabinet Office and the ‘Big Five’ government departments to discuss the People Survey updates and best practice.
● Monthly Reference Group meetings are organised where Engagement Leads representing our TLBs come together to collaborate by sharing progress on engagement activity, identifying common challenges and spotlighting best practice.
● Engagement Pages are available on intranet and highlight the ‘Listen, Act, Track’ methodology, providing a ‘Policy Bites’ video on engagement that helps all staff to understand the value of activity to improve employee engagement and experience. Best practice resources such as Engage for Success for teams to continually learn from are also highlighted.
● At local level, results are shared with local teams via the Survey provider’s digital dashboard and all local Engagement Leads have access to the dashboard, empowering them to conduct further analysis of their data and identify priority areas.
● Teams are encouraged to discuss their local results and review engagement action plans to identify areas of success and progress, along with areas of improvement. Where needed, further local focus groups are set up to discuss the survey results and identify actions.
● TLBs utilise a range of tools to drive engagement activity forwards including local action plans, regular newsletters highlighting engagement activity and sharing information, local networks of Engagement Champions for driving and communicating action across our full estate, face to face road shows providing opportunities to discuss engagement activity, workshops on topics highlighted within the survey etc. MoD is also piloting a Self Service use of the Qualtrics digital dashboard for local leaders to build greater engagement ownership of their engagement results.
● In terms of reporting, as part of the ‘People Survey Takes Action’ series MoD publish articles on the Department wide intranet spotlighting activity that has taken place at a corporate level to improve employee experience as a result of the People Survey, monthly rolling articles and blogs spotlighting activity that has taken place in each TLB, helping to share best practice across the MOD.
● Local and corporate activity is reported back to Senior Boards on a quarterly basis, enabling Senior Leaders to monitor progress and highlight potential areas where further work may be required.
141.1. Once the Civil Service People Survey findings are cascaded from the Cabinet Office to participating organisations, there are some steps that take place within HMR. These include:
● Reporting: HMRC survey managers prepare a sequence of senior leader briefings at the top of HMRC on results in the wider context of the HMRC Strategic Objective ‘Making HMRC a great place to work’. Results are also disseminated to report owners (managers with a team of 20 or more and where a minimum of 10 people respond) and survey managers ensure complete transparency of all results across HMRC for all staff.
● Further analysis: the HMRC survey managers manage deeper analysis for HMRC programmes of work and other bespoke requests.
● After Action Review: survey managers draft a model for continuously improving year-on-year.
● Funding: HMRC survey managers ensure payment and secure budget for the following year.
141.2. It is worth mentioning that in HMRC actions are planned following the Listen, Learn, Act and Communicate cycle and the organisation uses the People Survey, other HMRC surveys and approaches as a major part of how they listen to their staff. HMRC learns by creating ground level action plans in collaboration with employees to foster ownership, grounded in data, to drive the other two elements of the cycle.
141.3. Specifically, HMRC survey managers take forward and implement learning from the Civil Service People Survey by working in conjunction with staff to deliver on the action plan, building their capability throughout in organisational development and design, people analytics, listening tools, techniques and listening platforms and channels (acting); and providing regular, useful and insightful updates to staff with a ‘You said, we/you did’ format, making clear how location action relates to the data and insight that’s been gathered (communicating).
141.4. This approach of action planning and implementing learning acts as a barometer and becomes a part of HMRC evidence base ensuring that action is driven in line on HMRC.
141.5. Strategic Objective to make HMRC a great place to work and its five underpinning pillars ( connection to purpose; a great environment; enabled to do best work; attractive employment offer; and continuous learning culture).
141.6. The Magenta book is used as a guide for developing success measures and evaluation for various initiatives. For HMRC’s ‘Great Place to Work’ strategic objective, various People Survey questions provide the measurement for the Key Performance Indicators used to measure this strategic objective, to enable to track the impact of changes. Some Civil Service People Survey questions are replicated in a monthly sample Pulse survey to allow a continuous tracking. This approach also features in how the HMRC Executive Committee post-People Survey is briefed.
How is this recorded, monitored and kept consistent across departments?
What should the Cabinet Office’s role be in enabling departments to understand and benefit from the People Survey results?
● Learning and development - Identify informal training opportunities e.g. from a member of the team that has expertise in a certain area; complete a skills audit with members of the team. Arrange coaching/mentoring for team members and job shadowing opportunities. Feedback about L&D undertaken at team meetings.
● Inclusion and fair treatment - Recognise individual achievement appropriately e.g. a face-to-face thank you, a mention at a meeting or event, or nominations for awards. Also encourage peer recognition.
● Resources and workload - below is a screenshot of a part of the Further Reading and Resources section of the 2021 results dashboard on this topic.
August 2022
57