Evaluation of the National Child Protection Inspection programme on policing – full research report

Executive summary

Introduction and background

This report presents the findings of an independent evaluation of the National Child Protection Inspection (NCPI) programme. This programme was introduced in April 2014 and is run by Her Majesty’s Inspectorate of Constabulary and Fire & Rescue Services (HMICFRS), as a rolling programme for police forces across England and Wales.

The overall aim of the NCPI programme is to improve the experiences of and outcomes for children and young people who come into contact with the police when there are concerns about their safety or well-being. The specific objectives of the inspection programme are to:

• assess how effectively police forces safeguard children at risk;
• make recommendations to police forces for improving child protection practices;
• highlight good practice in child protection work; and
• drive improvement in police forces’ child protection practices.

The programme’s methodology builds on key strengths of earlier multi-agency inspections. It uses multiple modes of assessment to assess the child’s experiences of and interactions with the police across different stages of their journey through the system.

HMICFRS commissioned a mixed method evaluation to explore the perceived impact of the NCPI programme on policing practices since 2014. Findings have implications for supporting future development of the NCPI programme, as well as any ongoing evaluation.

Evaluation methodology

Multiple data collection methods were used to capture evidence on the perceived impact of the NCPI programme. At the outset of the evaluation, the research team worked in collaboration with HMICFRS to understand the intended outcomes of the NCPI programme and refine the programme theory using a logic model approach. The final model (see appendix B) sets out a logical description of how and why HMICFRS assumes that the inspection activities will lead to intended benefits. Alongside this, an indicator matrix was developed to measure success against these inspection programme outcomes (and gaps).

A qualitative case study design was also used to obtain an in-depth and contextualised understanding of the perceived impacts of the NCPI programme based on the views and experiences of key police staff, wider safeguarding partners, and the HMICFRS inspection team. This included 37 in-depth interviews with individuals from or working with 10 inspected police forces; a focus group and two interviews with individuals from or working with uninspected forces and a focus group with HMICFRS inspectors.

Alongside this, the research team administered an online survey aimed at staff and officers from all ranks working in neighbourhood policing, custody, and child protection units across all 43 police force areas. It gathered views on respondents’ backgrounds, their awareness of the NCPI programme, their views on its impact, and their forces’ approaches to child protection. Thirty four forces responded to the request and distributed the online survey within their forces. A total of 856 participants completed the survey.

The final strand of the evaluation involved the systematic review of inspection and post-inspection reports from 24 police forces, to extract information on impacts from the NCPI programme and identify improvements across inspection assessment criteria.

Key findings

Findings reported in this summary explore evidence and progress against key outcomes in the NCPI logic model, using triangulated data from the range of primary and secondary sources used for the evaluation.

Understanding and awareness of the NCPI programme

Understanding of the aims and benefits of the NCPI programme is the first outcome in the logic model and was considered important in facilitating positive change in how forces safeguard children and prioritise child protection work. Findings from this evaluation suggest that understanding and awareness is higher within inspected forces than non-inspected forces; higher among senior staff than frontline staff; and higher among those directly involved in child protection work than those working in areas with a broader remit (e.g. custody units). This is an important finding and suggests that while HMICFRS may be helping to increase awareness of the NCPI programme among individuals most involved in the inspections, more could be done to raise awareness among all staff and partners through different channels. This could for example include the current learning events run by HMICFRS.

Perceived impacts of the NCPI programme

The survey found that around one quarter (23%) of respondents felt that their force’s approach to child protection had improved because of the NCPI programme. This was higher among inspected forces and those inspected more recently.

Individuals from inspected forces and the inspection team believed that the NCPI programme helped to embed change in forces’ approaches to child protection; supporting progress in key areas such as understanding of child protection and safeguarding issues, information sharing and partnership work. Underpinning these changes was increased commitment from leadership and management teams to improving their child-centred approach, which was one of the early intended outcomes in the logic model.

Changes as a result of the inspection programme were identified across the inspections’ key areas of focus, including for example ‘leadership, management and governance’ and the ‘child’s experience of the force’. For instance, in relation to the child’s experience of the force, the reported impacts included:

• Internal awareness campaigns and training for officers and staff to improve knowledge and understanding around child protection. This aligns with the survey data showing a significant relationship between a force’s inspection status and the recency of training, though respondents felt improvements could be made to ensure training was of consistent high quality.
• An improved approach to identifying children at risk, assessing risks and making onward referrals.
• Improved awareness and approaches around capturing detailed information about the child, including from the child’s perspective. The survey highlighted a small difference between inspected forces, 66% of whom agreed that their force had a culture of listening to children as compared with 60% of those in uninspected forces. This indicates that some improvement might be felt from strategies implemented within forces following inspection activity. However, more could be done to ensure HMICFRS continues to support improvements in this area.

Underpinning these changes were a range of facilitators which linked together and closely followed the sequence of outcomes detailed in the logic model. The first condition perceived to be important was strategic engagement with and buy-in to the NCPI programme, which influenced the acceptance of recommendations and prioritisation of child protection work. This in turn was perceived to support investment in resources designed to safeguard children across the force.

Alongside this, a range of barriers were also identified that could limit the ability of the NCPI programme to achieve longer-term positive change. These included a force’s internal working practices and systems; the working practices of partner agencies; the wider inspection regime (e.g. other inspections); and the force’s perceptions of the NCPI programme’s findings. Furthermore, the survey findings highlighted that changes linked to the NCPI programme were perhaps not as strongly experienced by those working in frontline roles and that more positive views tended to be reported by those working in child protection units in comparison to other policing teams (particularly, custody teams). This suggests that changes driven by the NCPI programme could be better communicated and filtered down to all staff across forces to ensure impacts are as widespread as possible.

Unintended consequences of the NCPI programme were also reported. These could facilitate or impede child protection work and should be considered as the programme develops and adapts to work with more forces over time. Positive unintended consequences included:

• Some forces had reviewed their practice (drawing on the inspection methodology) to support ongoing monitoring and improvement across a range of areas, including those not identified through the inspection.
• A better awareness and understanding of how to work with other vulnerable groups and perpetrators. For example, one force reported how the NCPI programme had prompted them to reassess and broaden practices for assessing domestic violence perpetrators.
• Among uninspected forces, the NCPI was thought to have raised the profile of child protection.

Individuals from inspected forces had more mixed views on a number of other unintended consequences, including:

• Changing perceptions of the force across internal staff, external partners and the wider public, which could be both positive and negative in nature.
• Reviewing the allocation and use of resources to support child protection and safeguarding work which was perceived by some to be positive, but others as diverting resources away from other important areas.

Experiences of the NCPI programme

In the programme’s logic model there is an assumption that inspection work being better tailored to each force should have a direct impact on how forces implement changes and sustain and improve their child-centred approach.

Overall the NCPI programme was perceived to be different in approach from other inspections and participants were surprised by the scope of inspection activities. Three key elements of the inspection approach were thought to support effective assessment of force systems and performance in relation to child protection. They were:

• multiple evidence collection methods which included case file reviews, self-assessment activities and interviews that captured a range of perspectives;
• the detailed and discursive nature of evidence collection, for example case audits, some of which entailed a self-assessment element. This approach was thought to enable the inspection team to understand the operating context of the force and explore complexities; and
• the two-stage design of the methodology, including re-inspection activity, was thought to be effective in supporting ongoing improvement, monitoring how changes have been addressed and sustaining momentum.

The NCPI approach was considered engaging and comprehensive, and a greater sense of collaboration with forces was highlighted since the changes to the inspection approach in 2017 (see Appendix B). Elements of this approach that were thought to be impactful included the willingness of the inspection team to engage actively with forces and partners, and to discuss findings and challenges openly.

Challenges with the NCPI methodology and approach were also highlighted, relating to concerns about the:

• scope and scale of evidence collection: some questioned whether the inspection collected and triangulated all information necessary to build an accurate picture of policy, practice and key contextual issues. A lack of transparency around how conclusions were made was also highlighted.
• extent to which the force context, such as limited resources or the role of external agencies was taken into account when generating evidence and developing recommendations. This was felt across almost all elements of the inspection activity, and consequently effected the perceived efficacy of the recommendations made.

A key theme which emerged across the findings reported in this section was a desire for greater transparency on how the methods were used to develop recommendations. While information about the inspection methodology and assessment criteria is widely available online and shared in advance with forces, there was a sense that more clarity on NCPI processes could help with engagement in the inspection programme.

The provision of recommendations and areas of notable practice were perceived to be important in driving and sustaining improvements, as was the increased sense of collaboration between HMICFRS and inspected forces since 2017 (when the NCPI methodology changed). Participants valued the willingness to engage collaboratively with the force and felt that the supportive approach of the inspection team enabled forces to discuss issues and challenges comfortably. Others, however, felt that evidence may not have been as fully explored in reports as they might have hoped, linking to findings highlighted earlier around confusion as to how decisions are reached by the inspection team. In light of this, it seems that HMICFRS could usefully communicate more detailed information about how the inspection process works to ensure there is clarity and transparency across the sector.

Recommendations

Overall participants spoke positively about the NCPI programme and thought it provided an important opportunity to reflect and improve on child protection work. However, suggestions were also raised about how the NCPI methodology and approach might develop in the future to ensure it continues to support and collaborate with forces in the most effective way. Key recommendations include:

• Raising awareness of the aims and benefits of the inspection programme across staff grades and police units, and working with forces to ensure information is cascaded in an accessible way. Sustaining senior buy-in is also key to ensuring improvements are felt across programme outcomes, including disseminating information.
• Revisiting and improving the current opportunities in place for dissemination of findings and for forces and staff to learn from each other.
• Consideration of the timing of activities within the wider inspection regime. Linked to this, it was suggested that HMICFRS could provide more clarity over the range and purpose of all inspection work undertaken with forces.
• Building in an element of more iterative and continuous monitoring into the NCPI programme.
• Building on the information provided to forces after inspections including considering whether HMICFRS could do more to support the practical implementation of changes.

1. Introduction

This report presents the findings of an independent evaluation of the National Child Protection Inspection (NPCI) programme, which is run by Her Majesty’s Inspectorate of Constabulary and Fire & Rescue Services (HMICFRS) as a rolling programme for police forces across England and Wales. This chapter sets out the context for the research, its aims and objectives, the research methodology and associated limitations.

Research aims and objectives

HMICFRS has a statutory responsibility to conduct inspections of the police across England and Wales, and since 2017 the Fire & Rescue Services in England. Along with annual inspections of police forces’ general practices, their work includes rolling thematic inspections focusing on specific areas of policing such as child protection.

The overall aim of this evaluation was to explore the perceived impact of the NCPI programme on policing practices since 2014, with a focus on whether there is evidence:

• that the child protection policies of inspected forces have improved as a result of the inspection regime;
• that through greater involvement and engagement with HMICFRS inspection activity forces made more changes to their child protection policies and practices;
• that the programme has had an impact on the policies and practices of the forces that have not yet been inspected; and
• of unintended consequences of the NCPI programme, whether positive or negative.

It should be noted that this research tries to capture perceived impact from inspections that have been conducted since 2014. This evaluation provides an indication of how the NCPI programme supported and prompted change across the forces.

The inspection programme is ongoing, with just under half of the police forces in England and Wales still to be inspected at the time of writing. This research therefore provides useful tools for continuing evaluation work going forward. More information on options and key considerations for future evaluation are detailed in section 5: ‘Key considerations for future monitoring and impact evaluation’.

Background and context

Background to the development of the NCPI programme

The Children Act 2004 puts a duty on police and other agencies in England and Wales to safeguard and secure the welfare of children who come into contact with their services.[1] This includes local authorities, children’s social care and health and education services. The Act also states that the police and other key agencies have a specific statutory duty to:

• identify children who might be at risk;
• investigate alleged offences against children;
• engage in inter-agency working and information sharing to protect children; and
• use emergency powers to protect children.

Responsibilities for these agencies to work together were formalised in the Working Together to Safeguard Children guidance, first published in 2015.

Previous inspection approaches

In 2010, the Department for Education commissioned an independent review of child protection in England, which covered all local services that come into contact with children, including health, education and the justice system (Munroe, E. 2011). Findings from the review found that:

• Practitioners had become overly preoccupied with compliance and the system had become over-bureaucratised rather than focusing on the safety and welfare of children and young people;
• There should be a move from compliance to a learning culture which would enable professionals to have more freedom and use their expertise in assessing need and providing the correct help; and
• That the experiences of children, young people, and their families should be put at the heart of the inspection system.

The report concluded that:

‘instead of “doing things right” (i.e. following procedures) the system needed to be focused on doing the right thing (i.e. checking whether children and young people are being helped)’ (Munro, 2011)

Based on the findings from pilot inspections undertaken by HMICFRS, which resulted from the review, it was apparent that there was a need for a full programme of single-agency inspection that focused specifically on policing alongside multi-agency inspections such as the joint targeted area inspections (JTAIs), which are carried out by HMICFRS in collaboration with other agencies such as Ofsted and examine how well local agencies work together in an area to protect children. As a result the NCPI programme was introduced in April 2014.

Purpose and nature of the NCPI programme

The overall aim of the NCPI programme is to improve the experiences of and outcomes for children and young people who come into contact with the police when there are concerns about their safety or well-being. The specific objectives of the programme are to:

• assess how effectively police forces safeguard children at risk;
• make recommendations to police forces for improving child protection practices;
• highlight good practice in child protection work; and
• drive improvement in police forces’ child protection practices.[2]

Overview of NCPI methodology

The NCPI methodology builds on key strengths of earlier multi-agency inspections.[3] The inspections assess the child’s experiences of and interactions with the police across different stages of their journey through the system (i.e. initial contact, criminal investigation process, etc.).

The exact methodology used is flexible and can evolve with findings over the course of the inspection, depending on areas that require further exploration. For example, case audits are used to assess force performance in detail in particular areas of interest that have been raised earlier in the inspection process. More information on the inspection methodology is included at appendix E.

Following the inspection, HMICFRS writes a report detailing findings and including recommendations with expected timescales (immediate, within three months, or within six months). Within six weeks of the report being published, police forces must provide HMICFRS with an action plan outlining how they will respond to the recommendations. Inspectors then review these plans, which helps to inform the level of post-inspection work required. There are three levels of post inspection activity:

• Full reinspection is used in exceptional circumstances where there are significant concerns around improvement. This option uses the full NCPI methodology.
• Post-inspection review is used when there are concerns. The review is likely to include interviews with staff, members of the local safeguarding partnership[4] and the Director of Children’s Services (DCS), audits of selected cases, and a document review.
• A post-inspection visit is more light-touch and used in cases where findings from the initial inspection were broadly positive. The visit may include interviews with senior leaders, members of the local safeguarding partnership and DCS, and a limited review of key documents.

Changes to the inspection methodology (pre/post 2017)

In 2017 HMICFRS changed the way in which NCPI and JTAI inspections were carried out. While the assessment criteria and reporting remained broadly consistent, the way in which forces were engaged and how the fieldwork was conducted changed, with the aim of it being more collaborative, as described below.

Pre 2017, forces were notified of an inspection in advance, but there was little direct engagement until the fieldwork began. Contact between the force and HMICFRS was still limited once the fieldwork had started. Following data collection, forces would receive a draft version of the report and be asked to provide a factual response. Often the response was lengthy, did not reflect a full or shared understanding of the evidence and sometimes portrayed an unnecessarily defensive relationship between the force and HMICFRS.

Since 2017 each force to be inspected is contacted by the lead inspector and offered a ‘pre-fieldwork’ visit. This visit sets out the aims of the programme and provides details about how the inspection will be conducted as well as guidance on how to conduct the self-assessment. During the fieldwork, regular ‘keep in touch meetings’ are held to share findings on a regular basis. Challenge is invited and discussion of evidence encouraged at these meetings. Following the fieldwork, further visits are offered to support the force to develop and focus their improvement activity. HMICFRS also offers wider learning events for staff across the organisation.  These changes were made following feedback to HMICFRS on the pre 2017 methodology.

Research methodology

Multiple methods of data collection were used to explore the perceived impact of the NCPI programme, which included the development of the programme’s logic model, a document review, online survey and qualitative case studies, as outlined in figure 1.1.

Figure 1.1: Evaluation methodology

A brief overview of the methods is provided below.

• Logic model development. The final model developed by the research team sets out a logical description of how and why the inspection activities will lead to intended benefits (see appendix B). Alongside this, an indicator matrix was developed to map associated outcome measures and gaps. Together, the model and matrix can be used to communicate the aims and logic of the NCPI programme, support programme design and planning, and measure the progress and success of the NCPI programme going forward. Progress towards outcomes in the model, drawing on data collected for the evaluation, is highlighted throughout the report.
• Document review. Inspection and post-inspection reports from 24 police forces were systematically reviewed to extract information on impacts from the NCPI programme and identify improvements across inspection assessment criteria.
• Qualitative fieldwork. Depth interviews were conducted with 37 individuals from or working with 10 inspected police forces. Participants included force child protection / abuse leads, local authority / multi-agency assessment leads, business improvement leads, members of the local safeguarding partnerships, chairs or Directors of Children’s Services (DCS) and custody leads. We also carried out a focus group and interviews with two uninspected forces and a focus group with HMICFRS inspectors.
• Online survey. The survey was aimed at staff and officers working in police units where officers and staff were most likely to come into contact with children, such as neighbourhood policing, custody, and child protection units across police force areas. It gathered views on respondents’ backgrounds, their awareness of the NCPI programme, their views on its impact, and their forces’ approaches to child protection. HMICFRS invited all 43 police forces to participate in the survey. Thirty four forces responded to the request and distributed the online survey within their forces. A total of 856 participants completed the survey. Twenty five police forces had more than one survey response – of these 16 were inspected forces and 9 were uninspected forces. Survey findings are provided throughout the report and in a separate chapter detailing police staff and officers’ views on and approaches to child protection at appendix C.

Interpreting the evaluation findings

It is noted in the report where findings relate to either the survey, qualitative fieldwork or document review. Across the chapters, qualitative and quantitative findings are reported on concurrently to provide a detailed understanding of key issues and themes. Two points are important to consider when interpreting these findings:

1. The survey data provides useful context and background information about where forces felt they were in relation to key outcomes in the police force pathway. However, the survey sample was self-selecting and therefore not representative of the wider police force population. In the quantitative analysis that follows, the data has been statistically tested to identify differences that are statistically significant at the 95% level and these differences are highlighted in the report unless otherwise stated.[5] However, it should be noted that due to the sampling methodology any differences identified between subgroups (e.g. inspected and uninspected forces) may not be directly related to the NCPI programme but instead to a range of other factors.
2. In relation to the qualitative data, the number of people who hold a particular view is not reported as it offers no indication of the extent to which these views are held in the wider population.[6] The report distinguishes between different types of participants where this helps to illuminate findings and would not breach anonymity.

Report overview

The following chapters are organised thematically and discuss findings across the four strands of the evaluation. Each chapter begins with discussion of the relevant outcomes from the logic model.

• Chapter 2 examines awareness of the NCPI programme among staff, frontline officers, and senior managers.
• Chapter 3 explores perceptions of the impact of the NCPI programme on inspected forces’ safeguarding policy and practice.
• Chapter 4 outlines the experiences and views of those involved in the inspection activities.
• Chapter 5 sets out the report’s conclusions.
• Appendix A provides a reference list.
• Appendix B provides the NCPI’s logic model.
• Appendix C provides a summary of the survey analysis.
• Appendix D includes further detail on the research methodology used.
• Appendix E provides more detail on the NCPI methodology.
• Appendix F provides a glossary of key terms.

2. Awareness and dissemination of the NCPI programme

Chapter summary

The aim of this chapter is to explore awareness of the NCPI programme and associated learning events, among frontline officers and staff as well as individuals in more senior roles. It brings together findings from both the online survey and the qualitative data.

Figure 2.1: Logic model pathways

Understanding and awareness of the NCPI programme and its aims is the first outcome in the logic model for both the police force and the inspection team outcome pathways (appendix B). The assumption is that with greater awareness and understanding of the goals of the NCPI programme and better dissemination, police forces will engage with the programme more, leading to changes to child protection practices that are aligned with the aims of the programme. These are important outcomes for HMICFRS to measure and monitor.

The qualitative findings highlighted that awareness of the NCPI programme and its aims was generally higher among those working in child protection roles, as well as those directly involved in implementing the inspection recommendations.

Frontline officers and staff were perceived to have lower awareness of the NCPI programme, and findings indicated that awareness was lower among uninspected forces compared with inspected ones. In addition, awareness was lower among individuals working in custody and among external safeguarding partners, as these roles tended to be more involved in other inspections, which may affect their ability to disentangle specific inspection activity.

This suggests that while HMICFRS may be helping to increase awareness of the NCPI programme among strategic staff and those more heavily involved in the inspections, more could be done to raise awareness among all staff and partners through different channels. For example, through the current learning events run by HMICFRS. While awareness of learning events was low among all participants, there was interest in attending them, and it was felt that these may be a good avenue for the sharing of common challenges and best practice. The inspection team also highlighted that more work could be done to promote learning events and inspection findings, including through local meetings and networks and suggested that this sort of activity could be built into the NCPI programme more widely.

General awareness

The qualitative data indicated that awareness and understanding of the NCPI programme was generally higher among those more closely involved in child protection or wider safeguarding work, as well as senior leaders involved in implementing the inspection recommendations (for example Child Protection leads). Individuals from inspected forces thought this was to be expected, as only those with roles relevant to the inspection would have detailed knowledge about it.

There were two specific roles or areas of responsibility where awareness seemed to be lower:

• Custody: participants in custody lead roles appeared to have less awareness of the NCPI programme, underpinned by their more limited role in the inspection process, not necessarily being considered by forces as operating within a child protection environment, and because their remit involves safeguarding concerns which are not solely limited to children.
• External safeguarding partners: awareness of the NCPI programme was lower among external safeguarding partners; this was highlighted by participants in this role as well as police participants. This may be due to a perception of having less involvement in the inspections, the more limited relevance of NCPI recommendations to their organisations, and because their focus was instead on work for other inspectorates such as Ofsted, or on a JTAI.

The qualitative data indicated that frontline officers and staff were also perceived to have a lower awareness of the NCPI programme than case study participants in strategic roles, as involvement in inspection activities did not usually form part of their day-to-day responsibilities. It was felt that frontline officers and staff may just view the NCPI programme as part of an ongoing wider inspection regime, rather than a specific standalone inspection.

The survey results reinforced that those in frontline roles were less likely to be aware of the NCPI programme: results indicated that more than half of respondents (55%) were not aware of the NCPI programme prior to the survey. For those that were, the most common ways of becoming aware of the NCPI programme were by being informed by their senior manager or another senior leader (45% of respondents) or by reading about it via a communication channel such as the intranet or a newsletter (24% of respondents).

Awareness according to force inspection status

Another indicator of awareness is whether individuals in frontline roles were aware of whether their own force had been inspected as part of the NCPI programme. In line with the findings discussed above, awareness was low, with three in five (60%) respondents not knowing whether their force had been inspected. As shown in figure 2.2, the proportion of respondents who did not know whether their force had been inspected was also higher in uninspected forces (69%) compared with inspected forces (54%).

Figure 2.2: Awareness of whether force has been inspected, by inspection status

Base: 850 respondents

Survey findings also identified that almost a quarter (23%) of respondents stated that their force had been inspected on its child protection policies and practices, but that they were not sure whether this was the NCPI programme. Interviews with individuals in more senior roles in the inspected forces also suggested that forces find it challenging to distinguish between the NCPI and other inspections with a child protection focus. Some initially spoke about another inspection (such as a thematic child protection inspection) as opposed to the NCPI programme or found it hard to isolate one inspection from another. This highlights an opportunity to communicate more effectively with frontline practitioners about the purpose and aims of the NCPI inspections and how they differ from other inspection activity conducted by HMICFRS and partners.

Awareness of aims and findings

The first outcome in the logic model also highlights the need to be aware of the specific aims of the programme (see ‘Overview of NCPI methodology’ for programme aims), with the goal of leading to greater buy-in to the programme. The survey therefore asked respondents who were aware of the NCPI programme (45% of respondents) to what extent they were aware of its specific aims.

Awareness of the aims of the NCPI programme varied in relation to whether forces had been inspected. More than half (55%) of respondents in uninspected forces reported that they were not informed of the NCPI programme’s aims, compared with 40% of those in inspected forces.

Respondents from forces inspected after 2017, when the inspection methodology was changed (see chapter 1) were the most likely to report that they were informed about the NCPI programme aims. As shown in figure 2.3, 46% said this, compared with 34% in forces inspected before 2017 and 30% in uninspected forces. This suggests that HMICFRS could be doing more to maintain awareness of the inspection programme over time to keep momentum going, including among those who have had inspections over a year ago. However, levels of awareness may in part be related to respondents’ direct involvement in inspections and/or their ability to recall this information.

Figure 2.3 Extent of awareness of the aims of the NCPI programme by inspection timing

Base: 846 respondents

Individuals’ from inspected forces understanding of the aims of the NCPI programme was broadly consistent with the programme’s actual aims and specific objectives. The qualitative data highlighted that some participants understood there to be additional aims which included providing public reassurance; ensuring the provision of a consistent child protection service across all forces; and the sharing of knowledge and best practice between forces, which could be linked to objectives of other inspection programmes (for example the JTAI). This indicates that awareness among senior leaders, who are generally more engaged in inspection activities, is likely to be higher than frontline staff.

Low awareness of inspection findings was further reinforced by individuals from the inspection team, who discussed how uninspected forces may not have been engaged with NCPI findings unless they were due to have their own inspection. Inspection team participants also highlighted that the findings from inspected forces were not necessarily viewed by forces yet to have an inspection. Low levels of awareness of learning events and the formal dissemination of the inspection reports (which usually involved publishing them on the HMICFRS website), may have contributed to this.[7]

Dissemination and learning events

Improving the dissemination and use of evidence are outcomes in the logic model. The assumption is that as data and evidence is more effectively collated and shared, both inspected and non-inspected forces can better adapt their practice.

Figure 2.4: Evidence and dissemination outcomes in NCPI programme logic model

Individuals from inspected forces discussed awareness of inspection findings, which varied. Some individuals noted that recommendations and the force’s response and plans were shared, for example, via email, on the force intranet, and in key meetings such as partnership boards. In line with other findings in this chapter, teams directly involved in child protection were anticipated to have engaged most with the recommendations.

Quarterly learning events are one way in which HMICFRS formally supports dissemination of best practice. They also discuss common areas of challenge in child protection policy and practice. They are typically communicated via the National Police Chiefs’ Council and through HMICFRS’s national stakeholder newsletters.

Findings highlighted that there was a lack of awareness about learning events among individuals from both inspected and uninspected forces. This was the case among the more senior staff that took part in qualitative encounters (regardless of their role and force inspection status), as well as frontline officers and staff who took part in the online survey. Most survey respondents reported that they had either not heard or did not know whether they had heard about NCPI learning events. Only 5% said that they had either attended an event or had heard about but not attended one. Nevertheless, there seemed to be a general appetite for this kind of forum. Two thirds (68%) of respondents said that they would be interested in attending an NCPI learning event, with only one in ten (11%) saying that they would not. Interest was higher among respondents from uninspected forces (74%) compared with respondents from inspected forces (64%).

A participant in a partner safeguarding role described having attended learning events run by another inspectorate and finding these to be useful in identifying national themes (e.g. around county lines or serious youth violence) and what works in other forces. Learning events could also highlight that issues and challenges are shared, even by forces who may otherwise seem different.

Individuals from both inspected and uninspected forces also described other ways in which findings from the inspection reports of nearby forces had been effectively shared, which included regional meetings. These meetings were seen as an effective way of sharing findings locally, as they enabled uninspected forces to learn about and act on areas identified as needing improvement in inspected forces. Building on this, and in addition to the learning events, it was felt that a more targeted approach could be used to facilitate the dissemination of inspection findings and improving awareness of the NCPI programme. This could for example involve HMICFRS emailing inspection reports directly to relevant strategic individuals in uninspected forces.

Finally, it was suggested that collating inspection findings across multiple forces would allow for a thematic overview of best practice. Individuals from the inspection team suggested that learning events and dissemination (which could include regional meetings) should be formalised as part of the wider NCPI programme. Doing so would facilitate working towards achieving the two evidence and dissemination related outcomes in the inspection team outcome pathway (appendix B). The two outcomes outline data and evidence from forces being more effectively collated for the purpose of wider learning, and HMICFRS more effectively disseminating evidence from inspection activities.

3. Perceived impact and outcomes of the NCPI programme

Chapter summary

The overarching goal of the NCPI programme is ‘improved outcomes for children’. The logic model outlines the pathway of change that needs to occur for a child-centred approach to become embedded within police forces, and to facilitate effective partnership working with key stakeholders, as outlined in figure 3.1. This includes a force having the relevant training, systems and processes in place to enable staff and officers to:

• feel confident about making child-centred decisions;
• have a better understanding of individuals who pose a risk to children; and
• be better able to identify children and families in need and provide the necessary support and signposting to partner agencies.

Figure 3.1: Police level outcome pathway

This chapter describes the range of perceived impacts across forces who have been inspected. Data from the survey showed that around one quarter (23%) of respondents felt that their force’s approach to child protection had improved because of the NCPI programme (6% felt the force’s approach to child protection had not changed, 2% that it had worsened, and 69% did not know). This was higher among inspected forces both before the 2017 changes (33%, compared with 12% of uninspected forces) and those inspected after the 2017 changes (26%), see
figure 3.2.

Individuals from the inspection team and inspected forces felt that the NCPI programme had contributed to changes in three main areas:

• Embedding cultural change in forces’ approaches to child protection. This included both senior and frontline staff having a better understanding of child protection and safeguarding issues (though the survey highlighted mixed findings about awareness).
• Improving information sharing and partnership working around safeguarding children in inspected forces.
• Among uninspected forces, the NCPI programme was thought to have raised the profile of child protection.

Evidence of impact was also identified across specific inspection criteria. For example, in relation to the child’s experience of the force, three areas of change were identified, which included:

• Internal awareness campaigns and training for officers and staff to improve knowledge and understanding around child protection. The survey data showed a significant relationship between a force’s inspection status and the recency of training across two of the key training areas. Firstly 44% from inspected forces had received training on identifying children at risk in the last year, compared to 32% of respondents from uninspected forces. Secondly, 31% from inspected forces had received training on listening and speaking to children in the last year, compared to 24% from uninspected forces. Significant differences were not found across the two other areas, (‘making decisions in the best interest of the child’ and ‘referral pathways for children and their families’) which may indicate that the inspection programme is having less impact in these areas.
• An improved approach to identifying children, assessing risks and making onward referrals. In line with this, survey findings showed respondents in inspected forces were more likely to agree that they could access information in a timely manner (61% compared with 49% who agreed in uninspected forces) and that information captured was of a high quality and sufficient level of detail (41% agreed, compared with 34% in uninspected forces).
• Improved awareness and approaches around capturing detailed information about the child and the ‘child’s voice’. Survey findings showed that there was a small difference between inspected forces, 66% of whom agreed that their force had a culture of listening to children compared with 60% of those in uninspected forces. This indicates that there is some improvement which is trickling down from the senior team to the frontline. However, more could be done to ensure HMICFRS continues to support improvements in this area.

Underpinning these changes were a range of facilitators which linked together and closely followed the sequence of outcomes detailed in the logic model. Strategic engagement and buy-in with the NCPI programme from senior leaders within the forces, was perceived to be important in the acceptance of recommendations and prioritisation of child protection work. This in turn supported investment in resources designed to support the safeguarding of children across the force.

However, a range of barriers were also identified that could limit the ability of the NCPI programme to achieve longer-term change. These included a force’s internal working practices and systems; the working practices of partner agencies; the wider inspection regime; and perceptions of the findings. The survey also indicated that changes linked to the NCPI programme were perhaps not as strongly felt by those working in frontline roles, and that those working in custody teams were less likely to report positive views than those working in other areas. This suggests that more work could be done across forces to ensure impacts of any changes made regarding child protection are communicated and experienced widely.

Unintended consequences related to perceptions of inspections and the force; the use of resources; ongoing internal monitoring; and, how forces worked with other vulnerable groups and perpetrators. One key positive unintended consequence was forces’ drive to carry out continuous monitoring and review activities to support ongoing improvements in child protection work.

Lastly, among forces yet to be inspected there was limited evidence of impact that could be directly linked to the NCPI programme, which may be expected given the lack of direct engagement these forces are likely to have had with the inspection programme. However, staff recognised the role of the NCPI programme in promoting good practice across forces and sharing lessons learned. Drawing on findings from chapter 4, this suggests that there may be opportunities for evidence from other forces’ inspections to be more effectively collated and shared for the purposes of wider learning.

Perceived impacts on inspected forces’ safeguarding policy and practice

This section provides an overview of the perceived impacts of the NCPI programme, before moving on to describe evidence of impact across the specific inspection criteria that the evaluation focused on.

Overall perceived impact

The extent of perceived impact ranged from those who felt that it had been significant and shaped their force’s child protection structure and focus, to those who found it more difficult to identify areas of change. Examples of the former included the NCPI programme being associated with changes in governance structures that supported greater scrutiny and strategic oversight of child protection within the force. As a result of the NCPI programme, one force for example sought to integrate senior leaders from other organisations such as children’s social care within governance structures which was thought to have helped prioritise child protection issues across the force.

Data from the survey showed that around one quarter (23%) of respondents felt that their force’s approach to child protection had improved because of the NCPI programme, with the majority not knowing what impact the NCPI programme had had. However, the proportion of respondents that reported a positive impact on child protection as a result of the NCPI programme was higher among inspected forces at 30%, compared with 12% of uninspected forces. When exploring the data by inspection timing, 33% of those inspected after 2017, reported that the approach had improved, compared with 26% in forces inspected before 2017 (as highlighted in figure 3.2). This indicates that improvements were felt more by forces that had been inspected more recently since the inspection methodology was changed (more detail in chapter 1).

Figure 3.2: Impact respondents think the NCPI programme has had overall on their forces’ approach to child protection

Base: 834 respondents

Some individuals, including in uninspected forces, found it more difficult to identify and disentangle the perceived effect of the NCPI programme from other drivers of change, such as other inspections, internal reviews, and wider national and contextual changes around child protection safeguarding policy and practice. A lack of awareness of impact on uninspected forces was also reflected in the survey findings. Just under four-fifths of respondents (79%) from uninspected forces (62% in inspected forces) did not know whether the NCPI programme had had any impact on their child protection approach, and under one-fifth (12%) said it had improved (30% in inspected forces). Improvements could be linked to wider changes perceived to have been driven by the NCPI programme or due to misinterpretation of findings related to other inspections, for example JTAIs.

Individuals from inspected forces and the inspection team felt that the NCPI programme had contributed to changes in three general areas:

• Firstly, by helping to embed cultural change in forces’ approaches to child protection. This included both senior and frontline staff having a better understanding of child protection and safeguarding issues.
• Secondly, by improving information sharing and partnership working around safeguarding children in inspected forces (both are outcomes included in the logic model).
• Thirdly, among uninspected forces, the NCPI programme was thought to have raised the profile of child protection within the police and among wider partners.

Individuals from inspected teams felt that the NCPI programme had helped to both underpin changes that were already underway at the time of the inspection, as well as change in areas where forces had been unaware that it was needed, suggesting that the inspection programme successfully recognised good practice and also areas of improvement in forces.

Perceived impact across the NCPI programme criteria

Alongside exploring the overall effect of the NCPI programme, encounters with individuals from inspected forces and the inspection team identified evidence of impact across the specific inspection criteria, including:

• leadership, management and governance of the forces;
• the child’s experience with the force;
• child-focused and evidence based decision-making practices within the forces;
• force management of those at risk of harming children;
• treatment of children in police detention; and
• treatment of cases of missing and absent children.

Impacts are reported below under each of the above inspection criterion.

Leadership, management and governance

Commitment from leadership and governance arrangements to support a consistent child-centred approach underpin the police outcome pathway. The theory is that greater ‘buy-in’ from leaders should lead to improvements in other related areas (see logic model, appendix B or at the beginning of this chapter).

The survey explored frontline views of the support offered to them by management and leadership teams and approaches to foster an environment which safeguards children. Key findings included:

• Overall, respondents felt mostly positive about their management and leadership teams’ approach to child protection. Over two thirds (69%) felt the management and leadership team had ‘very much’ or ‘mostly’ fostered an environment which effectively safeguards children and only 3% said they had not. Three quarters (74%) agreed that they felt supported. Those who felt supported by their force’s management and leadership teams were also more likely to feel confident in their ability to make child-centred decisions.
• Findings varied significantly by the areas of the force in which respondents worked. Higher proportions of respondents working in child protection units than those in neighbourhood policing and custody teams felt supported, that their force had a culture of listening to children, and that leadership teams had fostered an effective safeguarding environment. Results were similar among respondents from inspected and uninspected forces which indicates that the NCPI programme may not have had an impact in this area. There was however significant variation in views across all three areas according to the area of the force in which respondents worked, with those in custody units consistently reporting less positive views. For example, less than half (45%) felt their force had a culture of listening to children, compared with, for example, 60% in neighbourhood policing and 80% in child protection teams, highlighting this as an area for HMICFRS to focus on going forward.

The NCPI programme was associated with changes in leadership, management and governance structures in the following five ways.

• The inspection identified or had helped the force to recognise areas that needed greater strategic oversight. New systems and governance structures had been put in place within the force and more widely with partners. This included areas of responsibility being reassigned within the force, the creation of new strategic groups, and ensuring consistent ways of working with safeguarding partners (a key area of improvement which was also highlighted across forces in the document review). For example, one force had introduced regular improvement board meetings around implementing NCPI recommendations which were also attended by key partners. The open-response survey data showed some respondents highlighted timely communication of information with partners as an area of best practice, but that others felt that sometimes partnerships did not work as well, which impacted on the transfer of information and referrals.

• Senior leadership were more involved in internal training and communications around child protection. This included increased monitoring of force performance to support continued improvement. For example, one force had used an internal staff survey to improve understanding of their force’s knowledge around the issues identified by the inspection, and to assess training needs. Training was also highlighted in the document review, where some forces had implemented new training programmes or courses, or had new training plans in place to roll out. Higher proportions of survey respondents from inspected forces reported having training more recently in two key areas: ‘identifying children at risk’ (44% of those in inspected respondents, compared with 32% of respondents in uninspected forces), and ‘listening and speaking to children’ (31% of those in inspected respondents and 24% in uninspected). Evidence of significant differences by inspection status was not found for training on ‘making decisions in the best interests of children’ or on ‘referral pathways for children and families’.

• Closer working between strategic and operational staff, for example through increased supervision and staff being involved in the drafting of force protocols.

The child’s experience of the force and child-focused, evidence based decision-making practices

A key aim of the NCPI programme is to understand the child’s end-to-end experience with the force. As such, the inspection team needs to explore whether staff have the right information and knowledge to effectively safeguard children, how they make decisions and the extent to which these are child-centred and based on good quality evidence. Key outcomes for the police pathway in the logic model are that staff and officers feel confident and supported to make child-centred decisions, and that children are engaged and listened to throughout their interactions with the police, which should lead to better identification of children in need and signposting to support (see logic model, appendix B or at the beginning of this chapter).

Changes relating to how forces consider children’s needs and improve decision making and interactions with children were highlighted in the document review and by participants from inspected forces who felt they could be linked to the work of the NCPI programme. Findings from the survey have been incorporated where appropriate.

• Firstly, internal awareness campaigns and training had been delivered to officers and staff to improve knowledge and understanding around child protection. This included how to interact with children and how to consider ‘the voice of the child’. As mentioned earlier, this aligns with the survey data showing a significant relationship between a force’s inspection status and the recency of training. Furthermore, those who had training more recently were more likely to report feeling confident in making child-centred decisions. However, respondents from both inspected and non-inspected felt improvements could be made to ensure training was of consistent high quality. Only a third (29%) of all respondents felt that they had all the information they needed to effectively safeguard children, and 10% said they did not have the information they needed at all (with no evidence of differences between inspected and uninspected respondents). Therefore, more work could be undertaken to review whether training has effectively supported safeguarding practice on the frontline.

• Secondly, an improved approach to identifying children at risk, assessing risks and making onward referrals. One force had introduced a system to flag and prioritise incidents that involved children. This was also evident in the document review, which highlighted greater consistency and depth in risk assessments. For example, one force had amended a domestic abuse risk assessment form to direct officers to speak to children and assess the impact of the incident on them. Accessing good quality information was also perceived to be important in identifying children and providing onward support. Survey findings showed respondents in inspected forces were more likely to agree that they could access information in a timely manner (61% as compared with 49% who agreed in uninspected forces) and that information captured was of a high quality and sufficient level of detail (41% agreed, compared with 34% in uninspected forces).[8] However, survey respondents reported that they wanted to know more about effectively identifying risk and working with partner agencies.

• Lastly, improved awareness and approaches around capturing detailed information about the child and the ‘child’s voice’. This included information on their demeanour and views, any relevant safeguarding issues to inform decision making. This aligns with survey findings which found variation by inspection status in forces’ culture of listening to children; in inspected forces, 66% of survey respondents agreed with this statement, as compared with 60% of those in uninspected forces. No evidence was found that confidence in making child-centred decisions varied by inspection status, however. The majority (80%) of frontline respondents reported that they felt confident in making these decisions (32% were very confident, 48% somewhat confident). Although, as with several of the other survey questions the proportion of staff working in child protection that reported feeling ‘very confident’ was 48%, more than double the proportions in neighbourhood policing and custody (22 and 23% respectively). Again, results did not vary significantly in relation to inspection status. The inspection team highlighted that there may be contrasting views on what effective child-centred decision making looks like, which may need to be explored further with forces as the inspection programme develops.

Force management of those at risk of harming children

The NCPI programme examines how the force identifies and works with partner agencies to manage those who pose a risk of harm to children. Staff and the inspection team identified two key areas of impact which helped improve forces’ understanding of high-risk offenders. These included:

• Improved information sharing about those at risk of harm to children across the force including with staff who sat outside the specialist sex offender teams. Such information could then be used to inform decision making when staff attended incidents involving a sex offender.
• Improved identification of child sexual exploitation (CSE) perpetrators and information sharing between agencies. As highlighted, 58% of survey respondents agreed that information was effectively shared with partner agencies, however there were no differences between inspected and uninspected forces.
• In contrast some forces reported a lack of impact in this area. One reason given for this was that the NCPI programme was felt to be more focused on the experiences and outcomes of children than those at risk of harming them. (The NCPI programme is less focused on compliance of offender management, for example in relation to legal conditions, and more focused on ensuring that staff within these teams understand the wider child safeguarding risks).

Treatment of children in police detention

A further area of scrutiny for the NCPI programme is police detention. The inspection explores whether and how children are detained and how they are treated when detained. This relates to a range of police force outcomes, including for example, how children are engaged and listened to and subsequently supported.

The reported impacts of the NCPI programme provide evidence in support of the NCPI programme being associated with a reduction in the number of children being held in police custody, in line with the Custody Concordat aims (Home Office, 2017).

• The case study data highlighted that some forces had increased awareness about their responsibilities to children in custody and best practice in this area, (which could, as mentioned above, be linked to other inspection programmes national policy changes and government guidance for example). Specific examples highlighted by individuals in inspected forces included delivering child-focused custodial training and a shift to children no longer being viewed primarily as offenders.
• The survey however found that only half (52%) of respondents who worked in custody units said that they felt supported to make child-centred decisions, compared to 86% of those in child protection units (there were no significant differences between inspected and uninspected forces which suggests the NCPI programme may not have had much of an impact in this area). It therefore seems that more could be done to ensure that safeguarding improvements are implemented and experienced by all teams working with children across the force, including custody units, which play an important safeguarding role, especially in supporting the Custody Concordat aims.
• Some forces had also implemented and reviewed systems in place around children’s access to support and appropriate adults while in custody. For example, one force described how all young people in custody now saw a healthcare professional when detained.
• The NCPI programme was also linked to better multi-agency working and information sharing around children in custody. Forces described how they had introduced new processes to support joint reviews of children in custody by the police and relevant partners and providing alternatives to custody.

However, as detailed throughout this chapter, survey respondents from custody units were less sure about the effectiveness of systems and processes in recording and sharing information than respondents in other areas of the force.

Treatment of missing and absent children

The final area of focus was the criteria around the treatment of missing and absent children. Three areas of impact were identified from interviews and the document review.

• Forces had reviewed and changed the systems in place for identifying, classifying and dealing with missing children. This included the completion of ‘safe and well’ checks when a child was found, to try and understand the reasons why they had gone missing.
• Changes had been made to multi-agency working arrangements. One force described how they had introduced weekly meetings with partner agencies and intelligence teams, to help ensure that the force had up to date information about ‘at risk’ children and would know who to contact if a child went missing.
• The document review highlighted an improved understanding around the links between missing children and CSE.

Facilitators and barriers to the NCPI programme supporting change

For the NCPI findings to support longer-term change the recommendations given to forces need to be achievable. A range of facilitators and barriers were felt to have had a bearing on forces’ ability to action the NCPI recommendations and implement change.

Facilitators to implementing change

Three key facilitators that had helped forces to implement recommendations were identified as described below.

• Firstly, strategic engagement with the inspection team was identified as a key factor as it was thought that forces might be more willing to implement changes if they understood where recommendations had come from and had a chance to discuss findings with the inspection team. For example, one participant described how ongoing dialogue with the inspection team meant that they had anticipated what the recommendations would be. This was felt to be an important part of the inspection process and may have had a bearing on the force’s subsequent level of engagement with implementing the recommendations (see chapter 4 for further information on experiences of engagement during the inspection). In addition, strategic staff also played a role in increasing awareness about inspection findings across the force. One participant described how staff at senior level had driven the required change by delivering seminars and communicating directly with officers and staff. This supports HMICFRS’s rationale that increased awareness and understanding of the inspection programme should drive greater improvements across key intended outcomes. However, as evidenced by findings in chapter 2, work around how HMICFRS disseminates inspection findings and engages with forces may require further development to drive ongoing improvements in this area.

• Secondly, the acceptance of recommendations by operational and frontline staff across a force was another key driver in implementing change, facilitated by strategic engagement as described above. This was thought to be important because staff at all levels have a role to play in delivering high quality services, so being aware of and on board with inspection recommendations could help facilitate change. Linked to this, one view was that a less positive inspection outcome could mean that staff more readily understood and were committed to the need for change. One force also described how tailoring messaging was important in ensuring the force could relate to key issues.
• Finally, participants noted investment and reallocation of resources to specific areas highlighted by the NCPI programme, for example training programmes and the creation of specialist roles. However, the survey findings suggested that frontline staff felt further improvements could be made in these areas, especially in relation to the quality of training to ensure it consistently supports frontline staff to effectively safeguard children.

In linking these facilitators together, it is apparent that the reported conditions for change closely followed the sequence of outcomes detailed in the logic model. Strategic engagement and buy-in with the NCPI programme appears to have influenced the acceptance of recommendations and prioritisation of child protection work. This in turn supported investment in resources designed to support the safeguarding of children across the force, which may be necessary to trigger the confidence and skills of those on the frontline as well as providing better resources (e.g., recording systems).

Barriers to enabling change

Evidence from the document review and interviews with staff also highlighted challenges some forces faced in addressing recommendations. Barriers which could limit the NCPI programme in facilitating change related to several areas: the internal working practices and systems within a force; the working practices of partner agencies; the wider inspection regime; and confidence in the NCPI findings.

Internal working practices and systems

Implementing recommendations around existing practices and systems within a force (including IT systems) were reported as challenging by some individuals from inspected forces, especially within short timeframes. In addition, and as discussed in chapter 4, it was felt that the force’s wider context, (e.g., existing infrastructure, available resources, the timings of other inspections, and working practices across forces) had not always been taken into account.

Changes to practice could also be limited if a force was unable to cascade relevant information about new systems and processes to all relevant staff, though this appeared to differ according to the size of the force. Larger forces reported finding this more difficult due to the scale of the task.

Smaller forces focused on different issues, such as challenging decisions about which recommendations they prioritised to help ensure resources were invested in the most critical areas.

Working practices of partner agencies

Partnership working was also highlighted in the survey and by individuals from inspected forces as an area which forces might focus on improving as it was thought to impact on the timely transfer of information and referrals. One view was that the NCPI programme did not focus sufficiently on partners and forces sometimes found it challenging to implement recommendations where a multi-agency response was required. For example, one force described how they were dependent on partner agencies to access local authority secure accommodation for children and young people in custody.

In line with this, the survey indicated that partnership working might be challenging for some respondents. For example:

• Of the 71% of all respondents who reported that they did not already have all the information they needed to effectively safeguard children, 38% identified partner agencies as an area where they would welcome additional information.
• Overall, 31% of all respondents had not at any time received training on referral pathways for children and their families or did not know when they had done so. Issues relating to partnership working were also commonly reported in open text responses both on areas of good practice and on areas where respondents felt their force could improve their practice. Examples of reported challenges included working relationships with partner agencies and difficulty in accessing information held by them, especially outside standard working hours. The need for more expedient referrals to appropriate services was also highlighted.

The wider inspection regime

Participants also reflected on the implementation of the NCPI programme within a wider quality assurance and inspection regime. At times this could be challenging, especially when it was felt that different professional bodies and inspections had conflicting views about what constituted best practice in child protection.

Confidence in the NCPI findings

Lastly, to implement the recommendations, staff needed to have confidence that they would lead to improved child outcomes. This could be undermined when it was felt that recommendations were not fully supported by evidence collected by the inspection team (also picked up in chapter 4). Force perceptions of the accuracy of findings could be linked to a misunderstanding of the way in which different pieces of evidence are considered and recorded to provide an overall picture of child protection practice.

In addition, while staff welcomed the follow-up element of the NCPI programme, they also noted it was not a continuous inspection programme and questioned the lack of further follow-up if a force had not fulfilled the recommendations. However, the inspection team also noted how they were not a regulator and could only make recommendations and suggest ways in which forces could improve.

It should be acknowledged that some of the changes discussed may have been aimed at very specific groups or areas of the force and thus not noticeable to all units. However, these patterns indicate that more work could be done across forces to ensure impacts or changes made in relation to child protection are communicated and experienced as broadly as possible. Work to increase awareness of the NCPI programme and its aims and prioritise child protection work across forces, is likely to go some way towards supporting this.

Unintended consequences

For any programme or initiative there are always a range of unintended consequences that do not form part of the intended outcomes pathway. To understand the full range of perceived impacts from the NCPI programme it is important that these are also identified. Staff in inspected forces and the inspection team reported five distinct areas of unintended impact from the NCPI programme.

1. Perceptions of inspections within the force. Staff described both positive and less positive views of HMICFRS following the NCPI programme depending on their experiences. One participant described how they now viewed inspections as being genuinely about improvement rather than ‘catching people out’. This may have been linked to the NCPI programme highlighting both positive and less positive practice, and also the more collaborative methodology of the NCPI programme relative to other inspections. In contrast, a participant who felt that their contribution had not been accurately reflected in the inspection report described how they would be less willing to give a genuine picture of the force in subsequent inspections. This perceived lack of transparency could be related to information not being recorded accurately which could negatively influence the overall impact of the inspection programme. A key facilitator of change was acceptance and confidence in HMICFRS’s findings.

2. Perceptions of the force across internal staff, external partners and the wider public. Individuals from inspected forces spoke favourably about how the findings were felt to have legitimatised the work of the police for the wider public and provided reassurance by showing officers that their work was valued. In contrast, a less positive inspection outcome in one force was felt to have made it challenging to recruit staff into child protection roles due to the additional scrutiny in this area. In addition, it was felt to have a negative impact on the way in which the force was viewed by partners, which could undermine partnership working. Finally staff questioned the impact of publicly highlighting gaps in delivery on public confidence and trust in the police more widely.
3. The allocation and effective use of resources. This was both when the inspection was conducted and afterwards when the force implemented recommendations. The findings from the inspection were felt to have provided impetus for resources being invested or reallocated within child protection. In some forces for example, increased resources were used to create specialist teams to offer a more timely, focused and consistent child protection approach.[9] The importance of specialist teams in developing and maintaining the required skills to safeguard children was also highlighted in the survey. However, some questioned whether resources had always been used most effectively, the impact on other areas of policing, and whether some of the changes that forces had implemented as a result of the inspection had actually created more work for the force. For example, staff questioned the necessity of increased staff attendance at child protection conferences, or the recruitment of a dedicated member of staff to oversee progress against recommendations. This highlights that on occasions some staff may not have agreed with the plans that had been put in place to action inspection recommendations. It should be noted that force plans following recommendations would not have been mandated by HMICFRS.

4. Taking part in the NCPI had prompted some forces to conduct their own review of practice and reflect upon it. This had been both in preparation for the NCPI among uninspected forces and after it had taken place. Forces had been able to draw on the inspection methodology to support ongoing monitoring and improvement across a range of areas, including those not identified through the inspection. For example, in reviewing the NCPI findings one force described how they had added an additional layer of safeguarding to the treatment of children in custody. For uninspected forces the NCPI programme was also perceived to have prompted changes when the inspection was more imminent. This included forces reviewing their own practice and investing resources in areas that the NCPI programme would examine.

5. Lastly, it was felt that the NCPI programme had helped create a better awareness and understanding of how to work with other vulnerable groups and perpetrators. For example, one force reported how the gaps identified by the inspection in relation to working with CSE perpetrators had prompted it to reassess and broaden practices for assessing domestic violence perpetrators.

4. Experiences of the NCPI programme

Chapter summary

This chapter provides an overview of the experiences and views of those directly involved in NCPI activities in their force area, as well as all participants’ reflections and responses to the inspection recommendations.

The NCPI programme’s logic model assumes that developments in the inspection approach should lead to improvements in the way forces work to protect children. More comprehensive and tailored inspection work should have a direct impact on how forces implement and sustain a child-centred approach, ultimately leading to better outcomes for children, as set out in the inspection team pathway in figure 4.1.

Figure 4.1: Inspection team level outcome pathway

Overall, the NCPI programme was perceived to be different from other inspections and individuals from inspected forces were surprised by the scope of the inspection. Three key elements of the NCPI approach were felt to support effective assessment of force systems and performance in relation to child protection. They included the:

• multiple evidence collection methods, which included case audits and self-assessment, observations and reality testing, interviews and focus groups across a range of participant groups. The mixed-method approach, element of self-assessment (unique to the NCPI programme) and inclusion of a broad range of perspectives was important in gathering detail across police force areas and helped to develop a rounded understanding of how the force perceived its child protection work, which could be triangulated and compared with case review evidence.
• collection and examination of detailed qualitative evidence was thought to enable the inspection team to explore and understand the complexities of forces’ work. This was perceived to facilitate a more robust and meaningful assessment than might be possible through the monitoring of quantitative data alone.
• two-stage design of the methodology (again unique to the NCPI programme), with a full inspection and subsequent revisit or review process, was thought to be effective in supporting ongoing improvement, monitoring how changes have been addressed and sustaining momentum.

Challenges with the NCPI methodology and approach were also highlighted, relating to concerns about the:

• scope and scale of evidence collection: some questioned whether the inspection collected and triangulated all information necessary to build an accurate picture of policy, practice and key contextual issues. A lack of transparency around how conclusions were made was also highlighted.
• extent to which the force context, such as limited resources or the role of external agencies was taken into account when generating evidence and developing recommendations. This was felt across almost all elements of the inspection activity, and consequently effected the perceived efficacy of the recommendations made.

Views on the fairness of the inspection recommendations were mixed. For example, some felt that overall recommendations were clear and focused on the right areas, whereas others felt that some evidence may have been misinterpreted or not fully explored.

A key theme which emerged across the findings reported in this section was a desire for greater transparency on how the methods were used and evidence was appraised to develop recommendations. While information about the inspection methodology and assessment criteria is widely available online and shared in advance with forces, there was a sense that more clarity on NCPI programme processes could help with engagement in the inspection programme, including for those in roles less directly involved in the day-to-day safeguarding of children.

Finally, the provision of recommendations and areas of notable practice were perceived to be important in driving and sustaining improvements, as was the increased sense of collaboration between HMICFRS and inspected forces since 2017 (when the NCPI methodology changed). Individuals from inspected forces valued the willingness to engage collaboratively with the force and felt that the supportive approach of the inspection team enabled forces to discuss issues and challenges comfortably.

Presentation of findings

It is useful to note that negative and positive views on the inspection approach and methodology were often shared alongside each other. Throughout this chapter we aim to balance positive findings alongside suggestions for improvements in order to present a well-rounded perspective on the inspection methodology. The evidence highlights a broadly consistent positive view from across the participants included in the evaluation. More negative views are reported with greater nuance, reflecting specific issues or challenges that participants had encountered. Several other considerations are helpful to bear in mind when reading this chapter:

• Less positive views were only shared by some forces and individuals within these forces.
• It was clear that these views sometimes related to specific experiences which may have been influenced by other factors external to the inspection programme.
• Less positive views seemed to be shared more by individuals in roles that were external to the force, including local authority safeguarding partners. This highlights that more could be done to raise wider awareness and buy-in of the inspection programme to support positive outcomes across the logic model.

Views on the inspection approach

In participants’ reflections on the inspection as a whole, four characteristics of the NCPI approach emerged as particularly important. These were:

• the combining of multiple methods;
• focus on qualitative evidence;
• range of participants included in inspection interviews or focus groups; and
• the two-stage design, discussed further below.

Multiple inspection methods

Staff welcomed the use of multiple inspection methods in combination, which is standard practice for HMICFRS to ensure that evidence is triangulated, and is key to ensuring the inspection methodology is robust and can identify good practice and areas for improvement. Examining case files alongside interviews with key personnel, for example, was felt to produce a rounded and holistic overview of a force’s work. This was reflected in the document review, which suggested that combining methods enabled the inspection team to compare practice on the ground with the expectations set out in statutory guidance on child protection and staff perceptions to identify any discord between the forces’ intentions and actual, evidenced practice. An example was where inspectors were told in interviews that improvements had been made in particular areas but could find no evidence of progress within case files.

Qualitative approach

The inspection’s flexible qualitative approach was felt to provide the inspection team with a richer insight into how forces operate than through quantitative data alone, a recommendation put forward in the Munro review (Munro, 2011). This was evident in the document review too. One example was where detailed qualitative evaluation of detention certificates highlighted a lack of understanding among police of the uses of alternative accommodation. In this case, even though the correct process of form completion had been followed, subsequent actions to protect children were not in line with best practice.

Some felt that the inspection was markedly different from others they had experienced, and were surprised by the scope of the inspection. Participants reported they had not anticipated that: case reviews would form such a large part of the inspection; that cases other than those specifically focused on child abuse would be included; or that the inspection would explore the journey of children from initial contact through to the end of their involvement.

However, as highlighted, a reported limitation of the approach was the extent to which the context in which the force operated was considered, including resource limitations, or regional contexts, for example. This has previously been recognised as a limitation of inspection activity on single issues, including child protection, responsibility for which falls across several organisations (Munro, 2011) and was one of the reasons that the JTAI was introduced.

Range of participants included in evidence collection

Participants considered it sensible and appropriate for inspection activities to include colleagues at different levels within the force as well as representatives of partner agencies involved in child protection. For example, it was noted that the inspection team observed local and force-level meetings, such as the multi-agency risk assessment conference (MARAC) meeting. This was found to be an effective means of observing problems such as a lack of attendance among key stakeholders at such meetings, which might not otherwise have been detected.

By involving a range of individuals, the inspection team was thought to be able to develop a rounded understanding of how the force perceived its child protection work, which could be triangulated and compared with case review evidence. However, some suggested that the scope of evidence collection could be further broadened to include for example, staff directly involved in specific cases in the case assessment strand of work, to add depth and nuance to inspection findings.

Two-stage design

The two-stage design of the inspection – that is, the main inspection and re-inspection, revisit or review – was regarded as something that supported forces to focus their efforts and motivate them to act upon the recommendations of the main inspection.

The timing of follow-up activity ranged from six to 12 months after the initial inspection, though ‘support visits’ to assist forces with the development of improvement activity could happen before this. Some forces found the follow-up a more positive experience than the main inspection, for the following reasons:

• They knew what to expect: the experience of the main inspection meant they had developed a familiarity with the inspection’s approach, focus, and key personnel;
• Activity was more contained: there were fewer strands of data collection/evidence to review at this stage; attention was focused on the areas highlighted in the inspection recommendations; and in some cases, a smaller team returned to complete the follow-up work. This was described as facilitating more collaborative, positive relationships;
• Having implemented some improvements, confidence was greater; the follow-up was an opportunity to demonstrate change and boost morale. This supports the general aims of the inspection and highlights how increased engagement can lead to improvements for forces.

A contrasting view was that the follow-up stage of the inspection could usefully have been broader in scope. This was because in focusing on a narrower range of evidence, it represented a missed opportunity to fully assess and understand any changes to the force’s practice, culture and context.[10]

Another reported limitation of the follow-up work was that in some instances participants did not think that reports demonstrated enough triangulation of police or partner accounts of changes that had been made with other sources of evidence, highlighting contextual challenges raised earlier. This suggests a lack of understanding from forces about how evidence collected for the inspection follow-up translates into findings and recommendations presented in reports. There is therefore a need for greater transparency in the methods and processes by which decisions are reached to ensure the detail included in reports is understood and used effectively.

Views on methodology

The data in this section provides important evidence across key inspection team outcomes, including effective tailoring of the NCPI methodology to better identify good practice and areas for improvement. As detailed below, the methods used by HMICFRS were perceived to be comprehensive, however, several improvements were identified across the data collection approaches.

Case file audits

Case file audits are one of the main inspection activities carried out as part of the NCPI programme to detail the journey of the child through the police force by assessing a dip-sampled selection of case files. The NCPI programme’s assessments are compared with an assessment carried out by the force itself to support self-reflective learning and identify any disparities between the two. A number of other cases (not self-assessed by the force) are also reviewed by the inspection team to further understanding of the process and systems within a force to inform inspection findings. Case file audits are not used to understand how the systems of the force work, this detail is collected elsewhere in the inspection.

Overall, forces viewed the case audit process as an effective methodology to assess the child’s experience of the force. The self-assessment element of the NCPI approach was regarded as allowing the inspection team to identify issues related to the force’s perception and assessment of the quality of its own work, including whether or not any weaknesses and areas for improvement were recognised by the force themselves. This sometimes led to immediate intervention if there was a cause for concern. For example, evidence from the document review detailed cases that were high-risk, such as those involving domestic violence, that had been mis-categorised as a standard risk, which led to immediate action in the force.

A number of challenges were also identified, which included:

• Some participants considered the self-assessment to be time consuming and resource intensive. This included for example showing inspectors how to use police force systems to locate records.
• Some felt it was difficult to assess cases accurately. While HMICFRS provides forces with assessment criteria and templates before inspections take place, some participants did not know that they were able to access this information which highlights a lack of communication, linking to earlier findings about awareness of the programme.
• There was limited recognition or reflection by the inspection team of the operational context, such as how the force works with partners or prioritisation of activities in response to resource limitations, which could have a bearing on the child’s experiences of and interactions with the force. Though as stated earlier, it is not intended that the case file methodology should be used to assess operational context, this view highlights some confusion from forces about the aims of this exercise which could perhaps be more effectively communicated by HMICFRS.
• The inspection team’s reporting on findings from the case audits did not appear to be consistent across all inspected forces. Participants felt that some reports set out a much more detailed breakdown of the results than others, which might better support understanding and improvements in forces’ work. It should be noted that the reports are only one way which inspection findings from case file audits are shared and it is likely that forces received other feedback from this exercise. However, as previously mentioned, more consistent communication about findings from these methods would help forces understand how conclusions had been reached.

In summary, although this method was valued by the forces as it provided unbiased insight into, for example how cases involving children were handled, it was also felt that communication on the purpose of the method could be improved. This links to the earlier point raised in relation to transparency of methods to ensure forces properly understand the purpose of evidence collection and how it is brought together to form recommendations in reports.

Interviews and focus groups

Interviews and focus groups are also used to gather the perspectives of partners, senior managers and practitioners. The document review revealed that interviews provided insight into local relationships as well as the forces’ involvement in child protection, with detailed feedback on issues such as police attendance at strategic meetings captured from discussions with members of the local safeguarding partnership.

Participant views on the usefulness and accuracy of evidence collected using this methodology were mixed. Three key factors influenced participants’ views on the value of evidence collection through interviews and focus groups and included the selection of participants, scope and content of interviews and mode of evidence collection:

Selection of participants

The efficacy of the approach depended on the individuals who were included. Some felt that the inspectorate took insufficient account of a force’s unique structure, made assumptions about who it would be best to speak to, and may not have accessed the most relevant information. As discussed, participants also felt that including individuals who had been directly involved in cases included in the case review and self-assessment might have added further context and nuance.

Scope and content of interviews

Views on the scope of the interviews were mixed:

• The document review suggested that, in tapping directly into staff knowledge and experience, interviews were particularly effective in accessing underlying reasons for concerns, highlighting issues with workload, capacity, wellbeing, and where these were most prevalent.
• From the interviews, one view was that the scope was fair, appropriate, and left no obvious gaps. Offering an opportunity at the end of the interview to add anything that had not been covered was highlighted as an effective way of ensuring all key topics were raised.
• Others, by contrast reported a different experience and felt the focus of interviews was overly narrow. Some participants for example felt that interviews had not included much discussion of good practice within the force.
• Finally, some felt that questions were sometimes closed or even adversarial, giving the impression that inspectors sought to confirm a hypothesis rather than building and assessing an evidence base through open and impartial questions.

Mode of evidence collection

Interviews were conducted face-to-face where possible and usually only over the phone to minimise inconvenience, in cases where police staff may have to travel long distances for example. Some felt that conducting interviews by telephone presented a barrier to developing rapport between participants and inspectorate staff, which could affect the quality of contributions.

The rapport developed in face-to-face interviews depended on both participants’ knowledge and ability to recall specific details, and on their candour and openness – adding weight, perhaps, to the notion that a combination of inspection methodologies is most appropriate to capturing a rounded and realistic picture of the force’s work in practice. One recommendation to enhance the level of detail that interviews captured was to give participants the questions or topic areas in advance.

Finally, it was noted that interviews were not recorded, and while forces were invited to comment on the accuracy of the reports, some felt there was a lack of opportunity for people to review their contributions at the time of the inspection. Furthermore, some participants did not feel their contribution was reflected in the inspection report, even though the interview itself had seemed fair and the questions appropriate. Given these concerns it seems that more could be done to demonstrate HMICFRS is effectively recording information gathered on inspections, checking details with participants at the time of evidence collection to ensure people felt information had been fairly and accurately relayed. This links to the earlier point raised around the need for greater transparency of the methods and processes used for the inspection.

Engagement with the inspection team

Views on inspection team engagement and support

Increased engagement with the NCPI programme is a key outcome across both the police and inspection pathways. The extent to which police staff and officers and NCPI staff worked together around inspection activities was perceived to contribute to this outcome and support forces to improve their practice. In 2017 the methodology of the inspection was changed to increase the sense of collaboration between HMICFRS and inspected forces.[11] Examples of collaboration and engagement were evident in both pre- and post-2017 inspections, but there seemed to be a greater emphasis on collaboration since changes in 2017. It is however worth noting that the more collaborative and supportive methodology was not always recognised by all individuals from inspected forces, which may be due to a number of factors including for example, individual participants’ involvement in and expectations of the inspection process.

Participants appreciated the inspection team members’ relevant professional expertise and willingness to engage collaboratively with the force and their partners. A sense of support and reciprocity in such relationships enabled the force to discuss issues and challenges comfortably.

Two key supportive approaches were described by participants as having been used during the inspection activities:

• The inspection team returned any high-risk cases that were identified to the force for review and action. Where this happened, one view was that the opportunity to discuss and feed back action taken with the inspectorate was beneficial.
• For those involved in inspections after 2017, the approach also included daily debriefs with senior members of the force. This ongoing dialogue increased awareness of ongoing inspection activities and enabled the force to act on emerging findings at an early stage. However, there was some suggestion that engagement could go further, for example, providing more written information on specific cases to support these verbal debriefs.

Others felt the approach had been less collaborative. An example was where the inspection team visited a partnership meeting without advance notice, which was seen as potentially jeopardising the force’s reputation and way of working there. Other factors that contributed to this view included the perception that the inspection team was somewhat guarded about how they reached their conclusions or reluctant to share detailed feedback with the force. As highlighted earlier in this section, greater transparency and an audit trail of evidence (e.g. through recording interviews) would have benefited the process and supported further learning and development.

Views on post-inspection engagement

Verbal debriefs for the inspection team to feed their emerging findings back to members of the force were held shortly after completion of inspection fieldwork in each force (these are called ‘hot debriefs’ as they take place at the earliest opportunity after evidence collection). Participants described the hot debrief as an opportunity for the inspection team to give the force an immediate response and welcomed its timing which allowed the force to immediately start addressing issues that were highlighted. One view was that the debrief also served as a useful opportunity to correct any misunderstandings in advance of the inspection report.

However, reported limitations of the debrief were that it was delivered verbally and lacked detail relating to how, for example, cases were assessed. One view was that the extensive use of acronyms made debriefs more challenging for staff whose expertise lay outside child protection to understand; another was that the final inspection report did not always accurately reflect the discussion. Though post-inspection support was available, some participants would have appreciated earlier written feedback to facilitate implementation of improvements from the hot debrief. As noted above, since 2017 the NCPI programme has involved verbal daily hot debriefs which participants found valuable, but again it was suggested these could be benefited by including written communication. This suggests that more work could be done to communicate such opportunities and explain changes in the conclusions with greater clarity.

Finally, some participants were frustrated that collaboration between the inspection team and the force did not appear to allow discussion of the draft reports’ wording or recommendations beyond correction of technical errors.

Inspection findings

It is important that the inspection methodology robustly identifies areas of good practice, and areas of improvement, and effectively supports forces to protect children by making useful and actionable recommendations. To focus on this, reports do not outline what might be expected as a minimum standard, only evidence that indicates forces are performing above and below this threshold. Findings in this section discuss the perceived accuracy and fairness of findings and timescales within which findings are delivered. Good practice and areas for improvement are highlighted throughout.

Accuracy and fairness of findings

Views on recommendations arising from the inspection were mixed. Some felt that, overall, recommendations were fair, clear and constructive; focused on the right areas; and that the coverage was thorough, balanced, and without gaps. For some participants, the recommendations reflected their force’s prior understanding of itself; others felt the inspection team’s expert external perspective offered helpful clarity even where the recommendations were as expected.

Others, however, felt that evidence may not have been as fully explored in reports as they might have hoped, linking to findings highlighted earlier around confusion as to how decisions are reached by the inspection team. In light of this, it seems that HMICFRS could usefully communicate more detailed information about how the inspection process works to ensure there is clarity and transparency across the sector.

Feedback on the usefulness of recommendations is outlined below.

• Firstly, and as mentioned previously, participants felt that the recommendations sometimes did not take account of issues relating to the force’s context, such as limited resources or the role of external agencies (including their ability to implement improvements). In addition, it was sometimes felt that blame was unfairly attributed to the force for issues that involved statutory partners that some recommendations were not achievable as they hinged on activity beyond the force’s remit.
• Secondly, for some, the recommendations reflected an aspirational standard beyond what was thought to be feasible and proportionate. One example was the suggestion that safeguarding an individual child should involve exploring the wider risks among the children with whom they were associated, described as a ‘Rolls-Royce service’ unattainable to the force.
• Thirdly, while the flexible and tailored approach of the NCPI programme was communicated to forces and welcomed by some participants, others questioned whether criticism was consistently given to all inspected forces. Perceptions of unfairness may be related to the fact that different areas for improvement are picked up through different inspections and may also indicate a lack of effective communication and information sharing across forces.
• Finally, it was suggested that recommendations could be made more constructive and helpful to forces by including more information on how cases were assessed and by offering suggestions as to how situations might be resolved. However, as mentioned earlier, it is beyond HMICFRS’s role to suggest how forces might solve problems and make decisions to improve practice.

Timescales for reporting

The timescales provided with inspection findings were generally felt to be a useful means of supporting forces to implement improvements.[12] This was because they indicated how activities should be prioritised and acted as a motivating influence.

The timescales provided were generally felt to be appropriate, with one view that it was important to begin work ahead of publication of the inspection report, based on the initial feedback provided in the hot debrief, as making the necessary changes between the report release and reinspection would not be realistic.

However, it appears that participants may have conceptualised the timeframes attached to particular recommendations differently. Some participants said that the timeframes indicated how recommendations should be prioritised and appeared to view them as windows within which to develop and establish each strand of improvement work, rather than a deadline by which that work must be fully completed. Others described the timescales as insufficient time to meet the recommended goals, which suggests they viewed them as final deadlines. This might indicate a need for greater clarity from HMICFRS as to the expectations for delivery against recommendations.

Notable practice

Participants recognised that the inspection findings covered areas of notable practice, which links to a key outcome in the inspection pathway that the methodology can identify good practice and areas of improvement. Examples of areas of notable practice that had been highlighted included:

• the experience, understanding, and commitment of members of the force; and
• progress from previous inspections, for example in areas including partnership working and information management.

The inclusion of good practice in the inspection’s findings provided participants with reassurance that specific existing policies and practices were working effectively. They welcomed the ‘balanced view’ this offered. Some felt that the highlighting of good practice as well as issues to fix lent the conclusions more credibility, and there was a sense for some that this might support staff buy-in or engagement with the recommendations.

Another view was that praise was somewhat limited, and focused more on ‘criticism’. Participants thought this was particularly the case where the inspection report did not include an executive summary (in pre-2017 reports). Instead, the report could appear to be ‘a beginning-to-end critique’, (though as mentioned earlier, it is worth noting that reports do not provide detail on minimum standards, only on where this threshold has been exceeded or not met). This might suggest that more could usefully be included in terms of highlighting good practice, sharing learning between forces, or foregrounding ‘quick wins’ the force could make early in the inspection report, feeding into evidence and dissemination outcomes.

5. Conclusions and key learning

This final chapter summarises key findings discussed throughout the report explaining how the evidence collected relates to outcomes in the logic model. Achievements and challenges as well as unintended consequences are outlined here. The chapter also provides a summary of recommendations, which could support future improvements and developments of the NCPI programme. Finally, the chapter sets out a series of considerations for future evaluation of the NCPI programme, focussing on how best to robustly measure impact.

Overview of key findings

A summary of key findings from the evaluation are included in this section. Links to outcomes in the logic model are referenced throughout with the aim of highlighting progress towards achieving success in key areas.

Awareness of the NCPI programme

Awareness of the NCPI programme is the first shared outcome in the logic model (see appendix B) and considered important in ensuring success in later outcomes in relation to how forces safeguard children and prioritise child protection work. The evidence gathered for this evaluation suggests that awareness is higher within inspected forces among senior staff and those directly involved in child protection work, and lower among non-inspected forces and frontline staff.

These findings suggest that more could be done to raise awareness of the inspection programme among all staff and partners through different channels, including the learning events, a few of which have been held so far. Learning events were discussed positively even if individuals had not heard of them previously. Appetite to improve practice and learn from findings of inspection work was palpable and it was evident there was a desire to ensure as much value as possible was captured from inspection work across forces. The underlying assumption in the logic model is that improvements in awareness will contribute to and support success across other key areas with the ultimate benefit of improving outcomes for children.

Perceived impacts of the NCPI programme

Data from the survey showed that around one quarter of respondents felt that their force’s approach to child protection had improved because of the NCPI programme. This was higher among inspected forces relative to non-inspected forces, and the highest in forces inspected more recently.

Individuals from case study forces believed that the NCPI programme helped to embed change in forces’ approaches to child protection, supporting progress in key areas such as understanding of child protection and safeguarding issues, information sharing and partnership work. Evidence of impact was also identified across specific inspection criteria. For example, in relation to the child’s experience of the force, three areas of change were identified, which included:

• Internal awareness campaigns and training for officers and staff to improve knowledge and understanding around child protection. This aligns with the survey data showing a significant relationship between a force’s inspection status and the recency of training, though respondents felt improvements could be made to ensure training was of consistent high quality.
• An improved approach to identifying children, assessing risks and making onward referrals.
• Improved awareness and approaches around capturing detailed information about the child, which aligns with the survey finding which shows that in inspected forces, 66% of survey respondents agreed that their force had a culture of listening to children as compared with 60% of those in uninspected forces.

Underpinning these changes were a range of facilitators which linked together and closely followed the sequence of outcomes detailed in the logic model. The first condition perceived to be important was strategic engagement and buy-in with the NCPI programme, which influenced the acceptance of recommendations and prioritisation of child protection work. This in turn supported investment in resources designed to support the safeguarding of children across the force.

The survey findings highlight that changes linked to the NCPI programme were perhaps not as strongly felt or experienced by those working in frontline roles and that more positive views tended to be reported by those working in child protection units in comparison to other policing teams (particularly, custody teams). This indicates that more work could usefully be done across forces to ensure impacts of any changes made in relation to child protection are firstly, communicated widely and secondly, experienced as broadly as possible.

Unintended consequences of the NCPI programme were also reported. These were perceived to both facilitate and impede child protection work and should be considered as the programme develops and adapts to work with more forces over time. They included:

• Changing perceptions of the force across internal staff, external partners and the wider public, which could be both positive and negative in nature. It was hoped that even though views may have changed, this focus on key issues would help to drive ongoing and further improvements on a range of child protection issues.
• Reviewing the allocation and use of resources to support child protection and safeguarding work. Individuals from inspected forces valued the opportunity to assess how resources were prioritised but warned that this may have led to the unintended consequence of removing much needed support elsewhere.
• Prompting forces to conduct or review their own monitoring and self-assessment, helping to foster a culture of continuous improvement.

Experiences of the inspection programme

Case study fieldwork with inspected forces explored experiences of the inspection process to understand how the NCPI programme evaluated child protection practice. In the programme’s logic model there is an assumption that improvements in the work undertaken by the inspection team should support forces to improve practice to better safeguard children. More comprehensive and tailored inspection work should have a direct impact on how forces implement changes and sustain and improve their child-centred approach. It is important therefore to understand views and experiences of the inspection process.

Overall, individuals from inspected forces thought the approach of the NCPI programme was engaging and comprehensive. They valued the openness of the inspection team, collaborative approach and commitment to use the inspection framework to drive positive and realistic improvements. In addition, they highlighted the valuable role the NCPI programme could play in the sharing of good practice across forces, inspected and uninspected.

Greater collaboration with forces was highlighted since the changes to the inspection approach in 2017.

Three key elements of the NCPI approach were thought to support effective assessment of force systems and performance in relation to child protection. They included:

• Multiple evidence collection methods which included case file reviews, self-inspection activities, interviews and focus groups. This mixed-method approach with different people was thought to be important in gathering the required range of perspectives and level of detail across police force areas.
• The detailed and discursive nature of evidence collection was thought to enable the inspection team to understand context and meaning, and explore complexities. This was perceived to facilitate a more robust and meaningful assessment than might be possible through the monitoring of performance indicators alone.
• The two-stage design of the methodology identified as unique to the NCPI programme, which includes re-inspection activity was thought to be effective in supporting ongoing improvement, monitoring how changes have been addressed and sustaining momentum.

Challenges with the NCPI methodology and approach were also highlighted, which related to concerns about:

• The scope and scale of evidence collection: though the approach to data collection was broadly welcomed, some questioned whether it was possible to collect and triangulate all information necessary to build an accurate picture of policy, practice and key contextual issues.
• There were mixed views on the validity of recommendations detailed in inspection reports, which links to points made throughout the report about a perceived lack of transparency around how evidence had been used to generate inspection findings. However, both recommendations and areas of notable practice were perceived to be important in driving forward and sustaining improvements.

A comprehensive overview of survey findings which have been reflected on throughout the report is included at appendix C. This appendix provides useful context and background information on child protection systems and practice and a measure of where participating forces feel they are in relation to key outcomes in the police force pathway in the logic model (appendix B).

Recommendations for the NCPI programme

Overall and as detailed throughout the report, individuals from both inspected and uninspected forces and the inspection team spoke positively about the NCPI programme and thought it provided an important opportunity to reflect on child protection work. They valued having an independent, detailed appraisal of force policies and practice with the aim of generating rich, detailed and insightful information to support and sustain improvements across forces. However, suggestions were also raised about how the NCPI methodology and approach might develop in the future to ensure it continues to support and collaborate with forces in the most effective way. Recommendations are set out below:

• Awareness of the NCPI programme was higher among those more closely involved in child protection or wider safeguarding work and senior leaders. Those in frontline roles were less likely to be aware of the NCPI programme; more than half of survey respondents (55%) were not aware of the NCPI programme prior to the survey and three in five (60%) respondents did not know whether their force had been inspected as part of the NCPI programme. This is an important finding and suggests that HMICFRS could perhaps do more to publicise the aims and benefits of the inspection programme across staff grades and police units, working with forces to ensure information is cascaded in a clear and accessible way to support buy-in and prioritisation of safeguarding work.
• Throughout the report, the appetite for more sharing of good practice was highlighted. This is a key outcome within the inspection team pathway and success in this area has the potential to improve a range of other police force and child outcomes. Individuals from both inspected and uninspected forces reported they would value having more opportunities to learn from each other. They suggested that HMICFRS has a key role to play in facilitating this through, for example, the learning events which could be usefully scaled up. However, the limited evidence of awareness of learning events, and the lack of dissemination of inspection findings more widely suggests that HMICFRS should work to improve how evidence is shared to drive improvements through this mechanism.
• Though work is undertaken to coordinate different inspection activities, further consideration should be given to the timing of inspection work and the potential for further coordination with other programmes such as PEEL. The amount of work involved in supporting inspection activities and the challenges of accommodating numerous inspections were highlighted and believed to be especially problematic for smaller and under-resourced police force areas. Greater coordination of scheduling could support effective preparation and ensure that the senior officers required for inspections are available.
• It was suggested that HMICFRS could provide more clarity over the range and purpose of inspection work undertaken with forces, including PEEL and JTAIs. It was felt that an improved understanding of the aims of different inspections and how they fit together might help to increase and sustain engagement with the inspectorate, which in turn should lead to better outcomes for children.
• The possibility of building in an element of more iterative and continuous monitoring was raised. It was felt that HMICFRS might get a more well-rounded and contextualised view of a force if information was collected and built upon over time. However, the extent to which timely recommendations could be made and acted upon with a more iterative programme might be limited. HMICFRS is working on developing a more efficient monitoring and inspection process, which started in 2018. This work also includes the development of a recommendations register, whereby recommendations can be tracked and measured more efficiently across different inspections. It is hoped that this new process will help HMICFRS monitor what forces are doing and assess progress against recommendation, supporting practice improvements.
• Building on the information provided to forces after inspections, some individuals from inspected forces suggested that HMICFRS could do more to support the practical implementation of changes. There was a sense that the reports sometimes focused too heavily on what needed to change and missed an explanation of how things might be improved. Opportunities to learn from other forces (as outlined above) could help operationalise recommendations for forces, drawing on the experiences of others. However, it is important to note that HMICFRS is an inspectorate not a regulator and therefore its powers only include the ability to make recommendations and not be prescriptive in how to implement these recommendations.

Key considerations for future monitoring and impact evaluation

This final section outlines a series of considerations for future evaluation of the NCPI programme, focussing on how best to robustly measure impact.

Producing a causal estimate of the impact of the NCPI programme is likely to be challenging as it is a national programme, and therefore hard to develop a ‘counterfactual’ scenario,[13] which would enable outcomes to be robustly examined. This makes ‘impact evaluation’ approaches such as randomised-controlled trials (RCTs) or quasi-experimental designs difficult to implement.

An alternative approach would be to undertake quantitative analysis comparing outcomes over time or between forces that had been recently inspected and those that had not, though it would be difficult to isolate factors which had directly caused change and make a credible causal claim.

Whatever the approach, the outcomes for any quantitative analysis must be both quantifiable and measurable and specific outcomes should be identified by HMICFRS as a first step.

Finally, it might be possible to evaluate the NCPI programme using a ‘theory-based’ approach such as realist evaluation, contribution analysis or process tracing. The aim of such approaches is not to quantify impact on outcomes, but instead to explore whether causation can be established beyond reasonable doubt by collecting evidence to validate, invalidate, or revise the assumptions in the logic model.

Appendix A. References

Children Act 2004, c. 31. [online] (PDF document) Retrieved on 15 December 2019.

HMICFRS (2013). Joint inspection of multi-agency child protection arrangements. HMICFRS.

HMICFRS. Child protection inspection methodology. (PDF document) HMICFRS. Retrieved on 15 December 2019.

HMICFRS. Assessment criteria. (PDF document) HMICFRS. Retrieved on 15 December 2019.

HMICFRS. Self-assessment template. (PDF document) HMICFRS. Retrieved on 15 December 2019.

HM Government (2018). Working together to safeguard children: A guide to inter-agency working to safeguard and promote the welfare of children. HM Government.

Home Office (2017). Concordat on Children in Custody.

Munro, E. (2011). The Munro review of child protection: Final report. Department for Education.

Ritchie, J., Lewis, J., Nicholls, C. M., & Ormston, R. (Eds.). (2013). Qualitative research practice: A guide for social science students and researchers. London: Sage.

Kellogg Foundation (2004). Using logic models to bring together planning, evaluation, and action: Logic model development guide (PDF document). Purdue University.

Appendix B. NCPI programme logic model

The focus of the NCPI inspection programme is on outcomes for and experiences of children who come into contact with the police when there are concerns about their safety or well-being. The overarching long-term goal articulated in this model is ‘improved outcomes for children’. Positive outcomes for children should be defined as sustainable, ensuring a safe and healthy transition to adulthood and focused on quality of life.

Two outcome pathways are identified in the model; outcomes for the police force and outcomes for the inspection team. All outcomes in the model denote a sense of change and are positioned sequentially to show how short-term outcomes can lead to medium- and longer-term outcomes. There is an expectation that all outcomes contribute in some way the overarching long-term goal of improving outcomes for children.

Appendix C. Current approach to child protection: survey findings

Summary of findings

This appendix provides an overview of frontline officer and staff views on the current approach to child protection in their forces, drawing on the findings from the online survey. It provides useful context and background information on child protection systems and practice and a measure of where participating forces feel they are in relation to key outcomes in the police force pathway in the logic model (appendix B).

The survey results indicated that on the whole, respondents felt positive about their forces’ approach to child protection. However, some specific areas for improvement were also highlighted. These related to the provision and quality of training and information to allow respondents to effectively safeguard children; systems and processes (especially the level of detail of recorded information); and partnership working. Despite this, a sizeable proportion of respondents felt supported by their leadership and management team and felt that there was a culture of listening to children in their forces. Moreover, the majority of respondents felt confident in their ability to make child-centred decisions.

There were some differences between respondents from inspected and uninspected forces. Differences between groups of respondents seemed also to be driven by the policing area in which they worked, and to a lesser extent, the length of time they had been working for the police. Those who worked in custody and those who had worked for the police for up to five years tended to be less likely to have positive views on their forces’ approach to child protection or were more likely to state that they did not know what their views were, compared with those working in other areas and those who had worked for the police for more than five years. This was also the case in relation to views on their own child protection skills. This might suggest that, to ensure that the earlier intended outcomes in the NCPI programme logic model are achieved, there is a need to focus support and capacity building on teams that do not specialise in child protection and individuals who have lower levels of experience of police work.

Interpreting findings

As discussed in chapter 1, while the survey provides insight into respondents’ views and experiences, it is not representative of all police forces in England and Wales as it was not conducted using random probability sampling. The sampling approach also meant that respondents were self-selecting, and it may be that those with particularly strong views were more likely to take part.

Where findings are reported by the forces’ inspection status, the unit in which respondents work, or the length of time they have served in the police, the intention is to give the reader greater insight into the data collected and the current context of child protection practices among participating forces. As discussed in chapter 1, any reported impact cannot necessarily be directly attributed to the NCPI programme.

In the quantitative analysis that follows, the differences highlighted are statistically significant at the 95% level unless otherwise stated.[14]

Views on approach to child protection in forces

Areas of best practice and areas for improvement

As part of the open text questions in the survey, respondents were asked to identify up to three areas of best practice for their forces’ approach to child protection and safeguarding children, as well as up to three areas where they felt their forces’ approach could be improved. For both, the most commonly raised themes were around working with partners and specific systems and processes. Both of these themes are outcomes identified in the NCPI programme logic model, which the inspection programme is aiming to improve. Views were very similar across both inspected and uninspected forces. There were some differences according to the units respondents worked in.

Further detail is provided below.

• Partnership working

Areas of best practice highlighted meaningful information sharing and timely communication, particularly through arrangements like the Multi-Agency Safeguarding Hub (MASH), social workers co-located in police forces, or Triage teams.

Areas for improvement related to similar issues that were felt not to be working well. For example, a lack of effective working relationships with partner agencies and difficulty in accessing information held by them, especially outside of standard working hours. The need for more expedient referrals to appropriate services was also highlighted.

• Systems and processes

Areas of good practice highlighted clear procedures around the recording of risks and vulnerabilities and ensuring that relevant forms (e.g. the Single Combined Assessment of Risk Form) are submitted consistently and completed fully.

Areas of improvement related to the accuracy of recording, especially in relation to the level of detail provided which was perceived to be insufficient, and specific recording systems which posed barriers to collating child related information e.g. family composition.

Another issue that was commonly discussed in relation to both good practice and areas for improvement was the importance of having specialist teams, as these ensure that individuals working in key roles are able to develop and maintain the required skills. Respondents from forces that did not have specialist child protection teams were critical of this, while respondents from forces that did highlighted the need for these to be better resourced.

Senior leadership and culture of listening to children

In order to achieve the long-term intended outcome of a wider cultural shift among police forces and other agencies, leadership and management need to be committed to and supportive of using a child-centred approach. The assumption is that this approach should be increasingly embedded into the culture of police forces, which could lead to improvements in the interactions and experience of children and families. As such, survey respondents were asked a set of questions about the extent to which they:

• felt the management and leadership team in their force had fostered an environment which effectively safeguarded children;
• felt supported by the management and leadership team in their force in the decisions they made about the needs of vulnerable or at-risk children; and
• agreed that there was a culture of listening to children in their police force.

Respondents felt mostly positive about the management and leadership teams’ approach to child protection. Over two thirds (69%) felt the management and leadership team had ‘very much’ or ‘mostly’ fostered an environment which effectively safeguards children. Furthermore, three quarters (74%) of respondents agreed that they felt supported in their decision-making about children. Results did not differ by inspection status in relation to either how supported respondents felt or the extent to which they felt leadership teams had fostered an effective safeguarding environment.

About two thirds of respondents (63%) agreed that there was a culture of listening to children in their police force. This varied in relation to their inspection status: 66% of those in inspected forces agreed with this statement, compared with 60% in uninspected forces.

Figure C.1: Extent to which respondents agreed that there was a culture of listening to children

Base: 849 respondents[15]

There was significant variation in views across all three areas according to the area of the force in which respondents worked, with those in custody units consistently reporting less positive views than those working in other areas:

• Less than half (45%) felt their force had a culture of listening to children, compared with, for example, 60% in neighbourhood policing and 80% in child protection teams.
• Half (50%) felt their leadership team had fostered an effective safeguarding environment, compared with 72% in neighbourhood policing and 77% in child protection teams.
• Just over half (52%) felt supported by their management and leadership teams in their decisions about the needs of vulnerable or at-risk children, compared with 75% of those in neighbourhood policing and 86% in child protection units.

This suggests that capacity-building work focused on child protection skills could be particularly beneficial for teams that do not specialise in child protection, for which children may comprise only a small proportion of the individuals with whom officers and staff regularly interact.

Training

As outlined in section 2, improvements in the provision and quality of training to staff and officers is one of the early intended outcomes of the NCPI programme in the police force pathway of the logic model. One of the key aims of the NCPI programme is to improve interactions between the police and children or families by moving away from a focus on processes to ensuring that decisions are always in the best interest of the child or their family. One route to improving these interactions is to ensure that police officers and staff have high quality and up-to-date training in relevant topics, and that they have the confidence to apply the knowledge they have acquired. Respondents were therefore asked when they received training in four specific areas:

• making decisions in the best interests of a child to ensure they are safe;
• listening and speaking to children when dealing with incidents;
• identifying children who are at risk; and
• referral pathways for children and their families.

Almost half (47%) of respondents had received training in the last year. However, 19% of respondents reported that they had not received training in any of these areas at any time. The proportion of respondents who reported never having received training was highest for training related to ‘listening and speaking to children’ (29% reported not having received this) and lowest for ‘identifying children who are at risk’ (which 16% reported not having received). Respondents’ open text responses also suggested that training provision could be improved, for example with additional training focused specifically on child protection for both specialist child protection officers and non-specialist response officers, and refresher training for those who had already received training.

The survey found that respondents’ training related to the areas of the force in which they worked, as shown in table C.2 below. Frontline staff in child protection were most likely to have had training, and those in custody were least likely. Respondents working in custody units were least likely to report that they had received training in each of the four areas: 35% reported that they had not received training in any of these, as compared with, for example, 19% of those in neighbourhood policing and 9% of those in child protection teams. This may relate to the nature of respondents’ roles, as colleagues in other departments engage with children at earlier stages of their contact with the police than those working in custody.

Table C.2: Proportion of staff who have ever received training by force unit

Training area	Custody unit: % received training	Child protection unit: % received training	Neighbourhood policing unit: % received training	Other unit: % received training
Identifying children who are at risk	75	88	84	86
Listening and speaking to children when dealing with incidents	55	82	67	75
Making decisions in the best interests of a child to ensure they are safe	66	81	73	77
Referral pathways for children and their families	63	83	81	79

Base: ranging from between 845 to 846 respondents

There did appear to be differences between inspected and uninspected forces in the recency of two strands of training: on identifying children who are at risk, and on listening and speaking to children when dealing with incidents (as can be seen in table C.3).

Having an inspection (JTAI or NCPI) of any kind made it more likely that respondents would have had training within the year. However, there was no evidence that the type of inspection had an effect.

Table C.3: Training delivered within the last year[16]

Training area	Inspected forces: % received training in last year		Uninspected forces: % received training in last year
Identifying children who are at risk	44	32
Listening and speaking to children when dealing with incidents	31	24

Base: 845 respondents

Respondents working in custody units in forces that had been inspected as part of the NCPI programme were less likely to have had training than those that had not been inspected. This pattern was similar across both non-inspected and inspected forces.

It is also important that training is of high quality and perceived to be helpful by those receiving it. This is likely to affect individuals’ confidence in their training, in turn influencing their implementation of good practice. Respondents who had received training were therefore asked about their views on the helpfulness of the training provided by their force in supporting them to successfully safeguard children. Overall, 70% felt that the training was helpful, while only 15% felt that it was unhelpful. However, about 15% of respondents who had received training felt the training was neither helpful nor unhelpful, or did not know how helpful it was. Those who had received training more recently were more likely to report positive views about its quality – 81% of those who had received training in the last year felt that it was helpful, as compared with 55% who had been trained a year or more previously, for example. These findings could indicate that training quality and the frequency of its delivery could be improved to ensure that it better supports frontline officers and staff to effectively safeguard children.

Views on systems and processes

Another early intended outcome in the police force pathway of the logic model is the ability of crime and incident systems and processes to record timely, accurate and high-quality information. Better and more timely access to information about risk and vulnerability could improve the nature and quality of decision making and protective planning. Having the right information could allow practitioners to make more informed assessments of need, enabling those in need of help and protection to receive the right support at the right time. Survey respondents were therefore asked to what extent they agreed or disagreed with a set of statements about the systems and processes in place in their force, with regard to the four key areas
of recording, access, quality and sharing of information about incidents relating to children.

As can be seen in table C.4 below, a large majority of respondents (79%) agreed that their systems and processes allowed them to efficiently record all relevant information. Smaller majorities agreed with each of the other areas. However, only 38% of respondents agreed that the information on the systems was of a high quality and contained the necessary level of detail. Lack of relevant detail in recorded information relating to children was also mentioned as an area for improvement in respondents’ open text responses.

Table C.4: Views on systems and processes in forces

Area	% Agree	% Neither agree nor disagree	% Disagree	% Don’t know
Able to efficiently record all the relevant information	79	10	7	4
Able to access all the relevant information in a timely manner	56	18	22	4
Information on system is of high quality and contains the necessary level of detail	38	33	20	9
Detailed information is effectively shared with partner agencies	58	19	10	13

Base: ranging from between 845 to 847 respondents[17]

Respondents in inspected forces were more likely to agree that they could access information in a timely manner (61% as compared with 49% who agreed in uninspected forces) and that information on the system was of a high quality and sufficient level of detail (41% agreed, compared with 34% in uninspected forces). No differences were evident in relation to the other two questions (about respondents’ ability to record all relevant information efficiently and about their forces’ information-sharing with partners).

Across all four areas, a higher proportion of respondents working in child protection units agreed with the statements than in custody and neighbourhood policing units.

Skills and confidence

Information and knowledge

An additional area that the survey explored was respondents’ views on whether they had the right information and knowledge to effectively safeguard children. While about a third (29%) of respondents felt that they had all the information they needed, more than half (56%) wanted to know more about specific areas, and a further 10% said they did not have the information they needed. There was no evidence of differences between respondents in inspected compared with uninspected forces. However, results differed according to respondents’ roles, with 20% of respondents from neighbourhood policing units and 23% from custody units reporting that they had all the information they needed, compared with 43% in child protection teams. Similarly, a smaller proportion of those who had worked for the police for up to five years (17%) said they had all the information the needed, compared to 31% of those with over five years’ experience.

There was very little difference in the areas that respondents in inspected and uninspected forces wanted more information about. These included:

• Partner agencies: respondents were keen to know more about which agencies to contact in case of concerns; and have more clarity on the responsibilities of all statutory organisations involved in child protection.
• Internal processes and procedures: including clarity on the appropriate processes and mechanisms for referrals, and procedures for joint investigations with Children’s Social Care in cases where there are concerns of children suffering or likely to suffer from significant harm.
• Assessment and identification of vulnerability and risk: including how to effectively assess risk and how to identify the safeguarding needs of children. This was particularly felt to be useful for response officers and in relation to neglect or domestic abuse incidents.
• Powers and legislation: including for example, when and how to use police protection powers, which give police the power to remove children from a scene to protect them from significant harm.

Confidence in making child-centred decisions

In order for staff and officers to be able to identify and support children in need and their families, they need to feel confident in their ability to make child-centred decisions[18]. The survey results indicated that the majority of respondents felt confident in making child-centred decisions, with 32% reporting that they felt ‘very confident’, and 48% ‘somewhat confident’ in doing so.

Levels of confidence did not differ significantly according to whether or not forces had been inspected as part of the NCPI programme. In inspected forces, 33% of frontline staff reported feeling very confident, as compared with 31% in uninspected forces.

However, respondents who had received training were more likely to report feeling confident in making child-centred decisions. Similar proportions of those who had received training within a year and more than a year ago reported feeling either ‘very’ or ‘somewhat’ confident (85% and 86% respectively), compared with 54% of those who had either not received training at all or did not know when they had done so.[19] This aligns with the logic of the NCPI programme, suggesting regular training has positive outcomes on the frontline workforce’s confidence in making the types of decisions HMICFRS is promoting.

Those who felt supported by their force’s management and leadership teams in their decisions about the needs of vulnerable or at-risk children were also more likely to feel confident in their decision-making. This was the case for 90% of those who felt supported, whereas 65% of those who did not feel supported by the leadership team nonetheless felt confident in their ability to make child-centred decisions (among those who said they felt very confident, the proportions were 37% and 29% respectively). This is in alignment with the logic of the NCPI programme theory, which suggests that support from leadership may influence the confidence of the frontline workforce.

Again, there was significant variation related to the force area in which respondents worked. Those working in custody roles were less likely to be confident in making child-centred decisions than those working in child protection and neighbourhood policing units. The proportion of staff working in child protection that reported feeling ‘very confident’ was 48%, more than double the proportions in neighbourhood policing and custody (22 and 23% respectively). Results did not vary significantly in relation to inspection status.

Differences were also apparent in relation respondents’ length of service. As shown in figure C.5 below, one in five (20%) of those who had worked for the police for up to five years stated that they did not feel confident in their ability to make child-centred decisions, while the corresponding proportion for those who had worked in the police for over five years was only 6%. This might suggest that this is a skill that frontline staff develop over time. Results did not vary significantly in relation to inspection status.

Figure C.5: Confidence in making child-centred decisions, by length of time worked in police[20]

Base: 846 respondents

The findings collectively demonstrate that, though frontline officers and staff respondents felt that they would like to know more about specific areas of child protection, they still felt overwhelmingly confident in their abilities to effectively safeguard children.

Appendix D. Methodological appendix

An overview of the methods used across the four separate strands of research are included in this appendix.

Methodology

Logic model development

The research team worked in collaboration with HMICFRS to refine the programme theory[21] for the NCPI programme using a logic model approach based on the W.K Kellogg Foundation Logic Model Development Guide.[22] This involved:

• A brief document review to develop the research team’s understanding of the programme as well as the origins, aims, activities and intended outcomes of the NCPI programme.
• Two half day workshops which focused on the NCPI programme’s intended outcomes and impact. Workshops were conducted with HMICFRS staff and inspection team, with the first focusing on mapping and ordering intended outcomes for the programme and the second on identifying evidence sources and gaps.

The final model (see appendix B) sets out a logical description of how and why the inspection activities will lead to intended benefits. Alongside this, an indicator matrix was developed to map associated outcome measures and gaps. Together, the model and matrix can be used to communicate the aims and logic of the NCPI programme, support programme design and planning, and measure the progress and success of the NCPI programme going forward.

Document review

Inspection and post-inspection reports from 24 police forces were systematically reviewed to extract information on impacts from the NCPI programme and identify improvements across inspection assessment criteria. Analysis focused on synthesising information about good practice and areas for improvement across inspection themes. Findings from the document review are referenced where appropriate throughout the report.

Qualitative fieldwork

A qualitative case study design was used to obtain an in-depth and contextualised understanding of the perceived impacts of the NCPI programme based on the views and experiences of a range of key police staff, wider safeguarding partners, and the HMICFRS inspection team. The following evidence collection encounters were carried out:

• 37 in-depth interviews with individuals from or working with 10 inspected police forces;
• a focus group with individuals from or working with an uninspected force, and two interviews with those from another uninspected force; and
• a focus group with HMICFRS inspectors.

Police forces were sampled with the aim of achieving diversity across key characteristics, which included:

• the date each force was inspected (due to changes in the inspection methodology which happened in 2017);
• geographic area;
• whether the force had a Joint Targeted Area Inspection (JTAI, an inspection programme conducted in collaboration with Ofsted, HM Inspectorate of Probation, and the Care Quality Commission); and
• type of post-inspection status, i.e. full reinspection, revisit, review.

Recruitment of participants from across the forces was supported by key contacts within the force who were briefed on the purpose and nature of the research. They identified relevant individuals who could speak to key roles of interest, including:

• force child protection lead / head of child abuse investigation team;
• Local Authority / multi-agency assessment lead;
• business improvement lead;
• member of the local safeguarding partnership or DCS; and
• custody lead.

An overview of the achieved sample is included in the following table.

Role	Interviews completed
Force child protection lead	8
Business improvement lead	7
Custody lead	8
Front door of LA / MASH	6
Children services / member of the local safeguarding partnership	8
TOTAL	37

It was not possible to achieve the full quota of case study participants, for reasons outlined later in this appendix (in methodological challenge). However, significant effort went in to securing evidence collection encounters with relevant individuals.

Fieldwork conduct and analysis

Interviews and focus groups ranged from 30 minutes to two hours and explored participants’ awareness and knowledge of the NCPI programme, their experiences and reflections of the NCPI programme (inspected forces only), and perceived outcomes and impact of the inspections.

Encounters were audio recorded on encrypted digital devices and transcribed verbatim, with consent. Transcripts were managed and analysed using Framework,[23] a systematic approach to qualitative data management that was developed by NatCen and embedded in the software NVivo.

Interview quotations are used throughout the report to illustrate themes and findings. Quotes are labelled according to the role participants held within their police force, as an external partner (which covered safeguarding roles within an LA), HMICFRS staff or a non-inspected force participant.

Online survey

The final strand of the evaluation involved administering an online survey aimed at staff and officers from all ranks working in police units where officers and staff were most likely to come into contact with children, for example, neighbourhood policing, custody, and child protection units across all 43 police force areas. It gathered views on respondents’ backgrounds, their awareness of the NCPI programme, their views on its impact, and their forces’ approaches to child protection. Questionnaires took approximately 10 minutes to complete and involved mostly closed questions, with a small number of open-ended questions.

The survey was conducted for just over one month in February and March 2019. HMICFRS invited all 43 police forces to participate in the survey, and of the 34 that responded to the request the online survey link and a cover letter were shared with an HMICFRS force contact, who then distributed this within their forces. A total of 856 participants fully completed the survey. Twenty-five police forces had more than one survey response – of these, 16 were inspected forces and 9 were uninspected forces. The total number of responses per police force ranged from 1 to 130, with 16 forces having more than 20 responses.[24]

The survey data was analysed by producing descriptive statistics using SPSS.[25]

Methodological challenges and limitations

As with all research, the evaluation methodology had limitations, and it is a marker of high-quality research to acknowledge them.

While engagement with the research was good overall, the main challenge associated involved the recruitment of participants, as outlined below.

Relevant participants for the qualitative fieldwork were sometimes difficult to identify, especially if their force was inspected several years ago, as many staff had since left their roles.

Despite our best efforts, it is possible some staff many not have clearly understood the difference between the independent evaluation and the inspection work of HMICFRS, which may have impacted willingness to participate.

Scheduling interviews and focus groups was challenging due to senior staff having competing priorities on their time.

The online survey was not shared for equal amounts of time across forces because of differences in IT infrastructure, which likely impacted participation rates across forces.

In addition there are two notable limitations in relation to the analysis it has been possible to undertake using the data collected, outlined below:

• The survey data provides useful context and background information about where forces felt they were in relation to key outcomes in the police force pathway.[26] The non-random nature of the sampling meant that findings are not representative of all police forces in England and Wales, however differences between groups have been tested statistically to help understand the data in line with other findings and caution should be taken in attributing causation to differences highlighted. The sampling approach also required participants to self-identify, which could lead to a potential bias in the views that have been measured. Therefore, any differences identified between subgroups (e.g. inspected and uninspected forces) may be due to the NCPI programme or a range of other factors which could have a bearing.
• In relation to the qualitative data, the number of people who hold a particular view is not reported as it offers no indication of the extent to which these views are held in the wider population.[27] The report distinguishes between different types of participants where this helps to illuminate findings and would not breach anonymity.

Topic guides

Tailored topic guides were used to ensure a consistent approach across all the interviews and between members of the research team. The guides were used flexibly to allow researchers to respond to the nature and content of each discussion, so the topics covered and their order varied between interviews. Researchers used open, non-leading questions, and answers were fully probed to elicit greater depth and detail where necessary.

The main headings and subheadings from the topic guides used for interviews with case study participants is included below.

1. Introduction

• Introduce self and NatCen
• Introduce research, aims of study and interview
• Brief overview of topics to be covered in interview
• Voluntary participation
• Confidentiality, anonymity and potential caveats
• Audio recording (including encryption, data storage and deletion)
• Questions
• Verbal consent recorded on tape

2. Background

• Role and responsibilities
• Nature and profile of local police force area
• Overview of current approach to child protection
• Brief overview of involvement in / awareness of any other current child protection inspections / inspections with child protection elements

3. Knowledge of NCPI programme

• Understanding of NCPI programme
• Expectations of inspection
• Awareness of NCPI programme
• Awareness of HMICFRS learning events

4. Experience of being inspected/ re-inspected

• Timing and outcome of inspection
• Overview of inspection process
• Views on inspection process – what worked well/ less well
• Views on HMICFRS inspection team
• Experience of being re-inspected (if relevant)

5. Perceived outcomes and impacts

• Recommendations and findings from inspection/ reinspection
• Specific outcomes of inspection/ reinspection
• Other outcomes of inspection/ reinspection

6. Reflections

• What works well/ less well and recommendations
• Perceived added value of NCPI programme

7. Next steps and close

Survey questionnaire

The final questionnaire specification for the survey is provided here.

Evaluating the HMICFRS National Child Protection Inspections programme

Questionnaire spec

Questionnaire conventions Section name – does not appear in programmed survey {Question routing} – who is asked the question Variable name Question text [Response type] – Question response options

All questions are {Ask all} unless otherwise stated.

All are [single responses] unless otherwise stated.

Intro

Thank you for your interest in taking part in the evaluation of Her Majesty’s Inspectorate of Constabulary and Fire & Rescue Services’ (HMICFRS) National Child Protection Inspections (NCPI) programme. We would like to hear about a range of experiences and views, regardless of whether your force has been inspected.

What is the research about?

NatCen Social Research (NatCen) is an independent not-for-profit social research organisation that carries out research on a wide range of social issues. You can find out more about NatCen on our website.

HMICFRS has commissioned NatCen to carry out a research project to explore the impact of the NCPI on policing practice. This includes forces’ experiences of the inspections and wider views of the inspection programme among forces that have not yet been inspected themselves.

To explore these areas, we are conducting an online survey with police officers who work within the Neighbourhood Policing, Custody and Child Protection units across the 43 police force areas. Alongside this survey we are also carrying out focus groups and interviews with police force child protection leads, safeguarding partners and members of the inspection team at HMICFRS.

Findings from this research will be used to understand the impact of the NCPI programme and refine the inspection approach.

Who can take part?

We would like a range of police officers who work within Neighbourhood Policing, Custody and Child Protection units to take part in the survey. Within these units, all roles and ranks are invited to take part. Please note that we are interested in hearing about a range of experiences and views, regardless of your level of awareness or understanding about the inspection programme and whether or not your force has been inspected.

Completing this survey is your opportunity to give feedback on the NCPI, whether your force has previously been inspected or is yet to be inspected.

Please note that the survey should not be shared outside the units identified above or outside the police forces.

What will the survey involve?

The survey will take around 10 minutes to complete. This needs to be done in a single sitting.

The survey includes some background questions about you, such as your age, gender, and about your working role. We will not be asking any questions that would make you identifiable, as the survey is completely anonymous. This will be followed by several questions about your awareness of the NCPI programme, your views on the work of the inspection team and reflections on your own practice in relation to working with children and safeguarding.

Do I have to take part?

No. You do not have to take part in the research – it is completely voluntary, and you can change your mind at any time. Your police force will not know whether or not you take part or what you say, and it will not affect your job in any way. You can choose to answer as many questions as you wish, with as much detail as you would like to provide.

If you would like to withdraw from the survey, please make a note of the unique serial number provided at the end of the questionnaire. You will need to contact the NatCen research team about your intention to withdraw, quoting this number, before 25th March 2019. Contact information for the NatCen research team can be found below and will be given again at the end of the questionnaire.

Is it confidential?

Yes. What you tell us will be confidential. No one will know you have taken part, including us. We do not ask for any information that could identify you such as your name, address or telephone number. Feedback from all completed questionnaires will be systematically analysed and combined thematically in a report. Any quotes will be used in such a way that the person who provided them will not be identifiable.

If participating in the research raises concerns around professional standards or misconduct, or you would like to make a complaint about a police force, you can contact the professional standards department for your force via your force’s complaints website. If you would like to make a complaint about HMICFRS staff, please visit /contact-us.

What will happen to my feedback?

We will use the feedback you and others provide to write a summary report for HMICFRS and the findings from this evaluation will be published online. Remember that no identifiable information will be mentioned in this report.

How will you keep my information safe?

Your feedback will be given a code and stored safely on a secure computer server at NatCen. We will securely destroy your feedback after the research report has been published. An anonymised dataset of survey findings will be given to HMICFRS for further analysis. Please see the Privacy Notice (PDF document) for more information on how we process data from everyone who takes part in this research.

I have questions about the research. Who do I contact?

• Please contact Jane Kerr, the NatCen research team lead, on 020 7549 7121 or at kerr@natcen.ac.uk; or
• Please contact Rosanna Edey from the HMICFRS team: HMICVulnerabilityAnalysis@hmicfrs.gov.uk

Stop page

You have stopped the survey for now. Your responses will be saved.

If you have any questions or concerns about the survey, or wish to withdraw your data, please contact the NatCen research team lead, Jane Kerr, on 020 7549 7121 or jane.kerr@natcen.ac.uk and quote your unique serial number <b>1001-{#iom.info.serial}</b> by Monday 25th March, 2019.

If you’d like to contact HMICFRS about the survey, please email Rosanna Edey at HMICVulnerabilityAnlaysis@hmicfrs.gov.uk

If participating in the research has raised any concerns around professional standards or misconduct, or you would like to make a complaint about a police force, you can contact the professional standards department via your force’s complaints website. If you would like to make a complaint about HMICFRS staff, please visit www.justiceinspectorates.gov.uk///contact-us.

NatCen Social Research is an independent research organisation. To find out more about us, please visit our website.

Police staff questionnaire V3

Demographic/background questions

RAge

What is your age?

1. 18-24
2. 25-34
3. 35-44
4. 45-54
5. 55-64
6. 65+
7. Prefer not to say

RSex

Which of the following best describes how you think of yourself?

1. Female
2. Male
3. Other
4. Prefer not to say

REth

What is your ethnic group?

Choose one option that best describes your ethnic group or background.

1. White – British
2. White – Other
3. Asian – British
4. Asian – Other
5. Black – British
6. Black – Other
7. Mixed
8. Other ethnic minority background (please describe)
9. Prefer not to say

RForc

Which police force do you work for?

1. Avon and Somerset
2. Bedfordshire
3. Cambridgeshire
4. Cheshire
5. Cleveland
6. Cumbria
7. Derbyshire
8. Devon & Cornwall
9. Dorset
10. Durham
11. Dyfed-Powys
12. Essex
13. Gloucestershire
14. Greater Manchester
15. Gwent
16. Hampshire
17. Hertfordshire
18. Humberside
19. Kent
20. Lancashire
21. Leicestershire
22. Lincolnshire
23. City of London
24. Merseyside
25. MPS
26. Norfolk
27. Northamptonshire
28. Northumbria
29. North Wales
30. North Yorkshire
31. Nottinghamshire
32. South Wales
33. South Yorkshire
34. Staffordshire
35. Suffolk
36. Surrey
37. Sussex
38. Thames Valley
39. Warwickshire
40. West Mercia
41. West Midlands
42. West Yorkshire
43. Wiltshire
44. Prefer not to say

RServ

How long have you worked for the police?

1. Up to one year
2. Over one, up to two years
3. Over two, up to five years
4. Over five, up to 10 years
5. Over 10 years
6. Prefer not to say

RRank

What is your role/ rank within the police?

1. Police staff
2. Police Community Support Officer
3. Constable
4. Sergeant
5. Inspector
6. Chief inspector
7. Superintendent
8. Chief superintendent
9. Chief officer
10. Other (please specify)
11. Prefer not to say

RArea

Which area do you work in?

1. Custody
2. Child Protection
3. Neighbourhood Policing
4. Other (please specify)
5. Prefer not to say

Progintro

[DISPLAY PAGE]

The NCPI programme

We would now like to ask you about your awareness of, and views on, the work of the HMICFRS National Child Protection Inspection team and the child protection policies and practices within your police force.

NCPIInsp

Has your police force been inspected as part of the National Child Protection Inspection (NCPI) programme?

1. Yes – definitely
2. Maybe – The force has been inspected on its child protection policies and practices, but I’m not sure if it was the NCPI
3. No
4. Don’t know
5. Prefer not to say

NCPIAware

How did you first become aware of the NCPI programme?

1. I was informed by my manager/ another senior leader
2. I was informed by another colleague
3. I read about it via a communication channel such as the intranet or a newsletter
4. I read about it in an internal formal policy or other internal document
5. I read about it in HMICFRS communications (e.g. inspection reports or summary reports)
6. I was unaware of the NCPI before this survey
7. Other (please specify)
8. Don’t know
9. Prefer not to say

{If NCPIAware = ‘I was unaware of the NCPI before this survey’, go to TrainFre}

NCPI4

[DISPLAY PAGE]

The four key aims of the NCPI programme are:

• Assessing how effectively police forces safeguard children at risk
• Making recommendations to police forces for improving child protection practice
• Highlighting effective practice in child protection work
• Driving improvements in forces’ child protection practice

NCPIAims

How would you rate how informed or uninformed you are about the aims of the NCPI programme?

1. Very informed
2. Somewhat informed
3. Neither informed nor uninformed
4. Not very informed
5. Not at all informed
6. Don’t know
7. Prefer not to say

TrainFre

Approximately when did you last receive training about the following:

	1: In the last year	2: Between one and two years ago	3: More than two years ago	4: I have not received training in this area	5: Don’t know	6: Prefer not to say
Making decisions in the best interests of a child to ensure they are safe (i.e. making child-centred decisions)
Listening and speaking to children when dealing with incidents
Identifying children who are at risk
Referral pathways for children and their families

TrainHelp

How helpful or unhelpful do you think the training provided by your force is in supporting you to successfully safeguard children?

1. Very helpful
2. Somewhat helpful
3. Neither helpful nor unhelpful
4. Not very helpful
5. Not at all helpful
6. Don’t know
7. Prefer not to say

Info

Do you feel you have the right information and knowledge to effectively safeguard children?

1. I have all the information I need
2. I have some information but would like to know more about specific areas
3. I don’t have the information I need
4. Don’t know
5. Prefer not to say

{If Info = ‘I have all the information I need’, go to DecConf}

InfoAdd

Please identify any specific areas in which you would like to improve your knowledge of safeguarding children:

[OPEN TEXT]

1. Don’t know
2. Prefer not to say

DecConf

How confident or unconfident do you feel about making ‘child-centred decisions’ – that is, decisions that are in the best interests of a child to ensure they are safe?

1. Very confident
2. Somewhat confident
3. Neither confident nor unconfident
4. Not very confident
5. Not confident at all
6. Don’t know
7. Prefer not to say

DecSupp

How supported or unsupported by the management and leadership team in your force do you feel in the decisions you make about the needs of vulnerable or at-risk children?

1. Very supported
2. Somewhat supported
3. Neither supported nor unsupported
4. Not very supported
5. Not supported at all
6. Don’t know
7. Prefer not to say

FEff

To what extent do you feel the management and leadership team in your force has fostered an environment which effectively safeguards children?

1. Very much so
2. Mostly
3. Somewhat
4. Not at all
5. Don’t know
6. Prefer not to say

TrainEqu

To what extent do you agree or disagree that the training you have been provided with by your police force has equipped you to deal with the following:

	1: Strongly agree	2: Agree	3: Neither agree nor disagree	4: Disagree	5: Strongly disagree	6: Don’t know	7: Prefer not to say
Identifying children at risk of harm
Protecting children at risk of harm (e.g. referring or signposting to other support)

FSyst

To what extent do you agree or disagree with the following statements about the systems and processes in place in your force to support good practice across key areas:

	1: Strongly agree	2: Agree	3: Neither agree nor disagree	4: Disagree	5: Strongly disagree	6: Don’t know	7: Prefer not to say
I am able to efficiently record all the relevant information about incidents relating to children in my force’s system
I can access all the relevant information I need about incidents relating to children in a timely way
The information held on my force’s systems about incidents relating to children is consistently of a high quality and contains the necessary level of detail
Detailed information about incidents relating to children is effectively shared with partner agencies

FLis

Overall, to what extent do you agree or disagree that there is a culture of listening to children in your police force?

1. Strongly agree
2. Agree
3. Neither agree nor disagree
4. Disagree
5. Strongly disagree
6. Don’t know
7. Prefer not to say

LeEv

Have you heard about or attended an NCPI learning event before now?

(Learning events are held by the NCPI team to disseminate best practice and discuss common areas of challenge in child protection policy and practice. They are typically communicated via the National Police Chiefs’ Council and through HMICFRS’s national stakeholder newsletters.)

1. I have heard about and attended a learning event
2. I have heard about NCPI learning events before but have not attended
3. I had not heard about the NCPI learning events until now
4. I don’t know if I have heard about the NCPI learning events before
5. Prefer not to say

{If ‘I have heard about and attended a learning event’ ask LeEvHelp; otherwise skip to LeEvInt}

LeEvHelp

How helpful or unhelpful was the event you attended in learning about how to improve how you work with children?

1. Very helpful
2. Somewhat helpful
3. Neither helpful nor unhelpful
4. Not very helpful
5. Not at all helpful
6. Don’t know
7. Prefer not to say

LeEvInt

Would you be interested in attending an NCPI learning event in the future?

1. Yes
2. No
3. Don’t know
4. Prefer not to say

NCPIimp

Overall, what impact do you think the NCPI programme has had on the force’s approach to child protection?

1. The force’s approach to child protection has improved
2. The force’s approach to child protection has not changed
3. The force’s approach to child protection has worsened
4. I don’t know whether the force’s approach to child protection has changed or not
5. Prefer not to say

FBPrac

Please provide brief details of up to three areas of best practice for your force’s approach to child protection and safeguarding children.

[OPEN TEXT]

1. Don’t know
2. Prefer not to say

FImp

Please provide brief details of any areas of your force’s approach to child protection that you think could be improved.

[OPEN TEXT]

1. Don’t know
2. Prefer not to say

AOB

Please use the space here for any additional comments about the NCPI programme.

[OPEN TEXT]

1. Don’t know
2. Prefer not to say

END OF QUESTIONNAIRE

End

Thank you for your interest in this research. We will use the feedback you and others provide to write a summary report for HMICFRS and the findings from this evaluation will be published online. No identifiable information will be mentioned in this report.

If you would like to withdraw from the survey, please contact the NatCen research team on the details below by Monday 25th March, 2019 and quote your unique serial number <b>1001-{#iom.info.serial}</b>.

If you have any further questions or concerns about the survey, please contact:

• the NatCen research team lead, Jane Kerr, on 020 7549 7121 or at jane.kerr@natcen.ac.uk
• or Rosanna Edey from the HMICFRS team at HMICVulnerabilityAnalysis@hmicfrs.gov.uk

If participating in the research has raised any concerns around professional standards or misconduct, or you would like to make a complaint about a police force, you can contact the professional standards department for your force via your force’s complaints website. If you would like to make a complaint about HMICFRS staff, please visit www.justiceinspectorates.gov.uk///contact-us.

NatCen Social Research is an independent research organisation. To find out more about us, please visit our website.

Appendix E. Overview of the NCPI methods

Coverage of inspection areas

Specific areas of practice investigated by the NCPI programme include:

• domestic abuse;
• incidents where police officers and staff identify children in need of help and protection, e.g. neglect;
• information sharing regarding children potentially at risk of harm;
• registered sex offender management;
• management of missing children;
• child sexual exploitation (CSE);
• children in police custody; and
• the exercise of powers of police protection under section 46 of the Children Act 1989 (taking children into a place of safety) and section 47 (completion of enquiries related to a child ‘in need,’ including those of a criminal and a non-criminal nature).

Notification timeframes

HMICFRS notifies groups of three forces at a time, two months in advance that they will be inspected within a four-month window. Forces selected for an inspection are sent a letter and instructions about the preparatory activities and timescales within which these should be completed. The forces do not know when within this four-month window the inspection will begin and will only be told about the inspection two working days before it is due to commence. This is intended to allow HMICFRS to get a better sense of the reality of child protection and safeguarding work undertaken within the force and among local partners.

Methods of inspection

Inspectors generally spend about two weeks at each force, and use multiple modes of assessment, detailed below. Assessment criteria and indicators used to examine forces are outlined in an ‘Assessment Criteria’ document used by the inspection team.[28] The inspection methodology includes:

Self-assessment

Forces are provided with a template to self-assess a sample of cases, which are then completed and submitted to HMICFRS.[29] The same cases are also assessed by inspectors. The template includes indicators of good practice and guidelines on the types of evidence forces might wish to consider. Strengths and weaknesses identified from these cases form the basis for discussions between the force and HMICFRS while also establishing a baseline against which to measure progress.

Case audits

On average, between 80-100 cases are audited for each force. Each case is rated for adequacy of how well they were managed (e.g. good, needs improvement). Common themes across cases in the same area of practice are then identified (e.g. police had a good understanding of risk across cases involving management of registered sex offenders).

Discussions with police staff and representatives from partner agencies

Including interviews with strategic leads, department heads, and Directors of Children’s Services (DCS) and members of the local safeguarding partnership, as well as focus groups with police officers and staff.

Examination of reports

Such as significant case reviews or other serious cases.

Examination of service statistics, reports, policies, and other relevant written materials

Such as performance data, corporate governance information, partnership data, and child consultation materials e.g. youth panels, surveys.

Reality testing and observations

Inspection staff observe various teams and locations, e.g. control room, safeguarding hubs, daily briefings, daily management meetings, management of sexual or violent offenders (MOSOVO) teams, response teams, investigative units and custody suites.

Changes to the inspection methodology (pre/post 2017)

In 2017 HMICFRS changed the way in which NCPI and JTAI inspections were carried out. While the assessment criteria and reporting remained broadly consistent, the way in which forces were engaged and how the fieldwork was conducted changed in significant ways.

Previously forces were notified of an inspection but there was little direct engagement until the fieldwork began. Once the fieldwork started, updates and contact between the force and HMICFRS was sporadic and often little dialogue occurred until the final ‘hot debrief’. After the fieldwork, contact was limited to the factual accuracy response the force is asked to provide when it received a draft version of the report. Often the factual accuracy response was many, many pages in length, did not reflect a full (or shared) understanding of the evidence and reflected an unnecessarily defensive relationship between the force and HMICFRS. This seemed like a missed opportunity to support improvements in practice and at odds with the stated aims of the NCPI and JTAI programme.

Since 2017 each force to be inspected is contacted by the lead inspector and offered a pre-fieldwork visit. This visit sets out the aims of the programme and provides details about how the inspection will be conducted as well as guidance on how to conduct the self-assessment. During the fieldwork, regular ‘keep in touch meetings’ are held to share findings on a regular basis. Challenge is invited and discussion of evidence encouraged at these meetings resulting in professional and supportive debate and a shared understanding of the evidence. Following the fieldwork, further visits are offered to support the force to develop and focus their improvement activity. HMICFRS also offers wider learning events for staff across the organisation. This is not ‘resource light’ but has been warmly welcomed by forces and reduced challenges around inspection findings.

Appendix F. Glossary

Director of Children’s Services (DCS)

The DCS is responsible for securing the provision of services which address the needs of all children and young people, including the most disadvantaged and vulnerable, and their families and carers.

Logic model

A logic model is a graphic which represents the theory of how an intervention produces its outcomes. It represents, in a simplified way, a hypothesis or ‘theory of change’ about how an intervention works.

References

[1] Children Act 2004, chapter 31 (PDF document). Relevant sections are 11 and 28 for England and Wales respectively.

[2] HMICFRS (2018). National child protection inspections. HMICFRS.

[3] HMICFRS. Child protection inspection methodology (PDF document). HMICFRS.

[4] Formerly this would have been the chair of the Local Safeguarding Children’s Board (LSCB).

[5] If a difference or change is ‘statistically significant’ at 95%, we can be 95% confident that these are real differences between different respondent groups’ views, rather than differences relating to chance.

[6] Any numerical inference is likely to be misleading or inaccurate as qualitative samples are not designed for this purpose, but instead to capture range and diversity of views and experiences.

[7] More information on dissemination of inspection reports and levels of awareness around findings is included in chapter 4.

[8] No differences were evident in relation to the other two questions about systems (ability to record all relevant information efficiently and about their forces’ information-sharing with partners).

[9] Unintended consequences of changes are described in section 4.

[10] Follow-up activities vary in detail depending on the inspection outcome, and the decision for the type of inspection activity is HMICFRS’s decision. Full re-inspections will for example involve more thorough evidence collection than a review or re-visit. See ‘Overview of NCPI methodology’ for full details on follow-up activities undertaken as part of the inspection programme.

[11] For more information on changes to the inspection methodology that happened after 2017, please see appendix E.

[12] More information on the timescales communicated to forces around recommendations in inspection reports is outlined in appendix E.

[13] Counterfactual impact evaluation is a method of comparison which involves comparing the outcomes of those having received an intervention, policy or programme with those of a group similar in all respects, the only difference being that the comparison/control group has not been exposed to the policy or programme.

[14] If a difference or change is ‘statistically significant’ at 95%, we can be 95% confident that these are real differences between different respondent groups’ views, rather than differences relating to chance.

[15] The total percentage for this question adds up to less than 100 percent due to rounding.

[16] There was no evidence of any significant variation by inspection status in when training was delivered on making decisions in the best interests of children and on referral pathways for children and families.

[17] The total percentage adds up to more than 100 percent in some answer categories due to rounding.

[18] As outlined in the logic model in appendix B.

[19] Training refers to any of the four strands of training discussed earlier in this appendix.

[20] An additional 1% of those who had worked in the police for up to five years and 1% of those who had worked for the police for over five years said that they did not know how confident they felt in making child-centred decisions.

[21] More information on programme theory and the rationale for the logic model approach can be found in the methodological appendix – appendix C.

[22] W. K. Kellogg Foundation, (2004). Using logic models to bring together planning, evaluation, and action: Logic model development guide (PDF document). Purdue University.

[23] Spencer, L., Ritchie, J., O’Connor, W., Morrell, G. and Ormston, R. ‘Analysis in Practice’ in Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (Eds.) (2014) Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage.

[24] This wide range could partly be explained by the different approaches taken to distributing the survey across the police forces. In some cases, the HMICFRS force contact targeted specific individuals and sent the survey link to them directly, whereas other HMICFRS force contacts distributed the survey link more widely, for example on the force intranet.

[25] Software package for statistical analysis.

[26] Comparisons between inspected and uninspected forces have only been noted where there are statistically significant differences in the data.

[27] Any numerical inference is likely to be misleading or inaccurate as qualitative samples are not designed for this purpose, but instead to capture range and diversity of views and experiences.

[28] HMICFRS. Assessment criteria (PDF document). HMICFRS.

[29] HMICFRS. Self-assessment template (PDF document). HMICFRS.

Back to publication

National Child Protection Inspection programme evaluation