Borrowing Our Neighbours’ Methods — Estimating the effect of preventative services on child welfare intervention rates under austerity

# Estimating the effect of preventative services on child welfare intervention rates under austerity
#### Borrowing Our Neighbours' Methods Symposium EUSARF 2023

<br>

**Dr. Calum Webb**<br>
Sheffield Methods Institute, the University of Sheffield<br>
[c.j.webb@sheffield.ac.uk](mailto:c.j.webb@sheffield.ac.uk)

]

*Postdoctoral Fellowship*  
*PF21\210024*

]
]

<div>
<style type="text/css">.xaringan-extra-logo {
width: 180px;
height: 128px;
z-index: 0;
background-image: url(header/smi-logo-white.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:2em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---

## Two analyses of the same data

---

## Two analyses of the same data

---

In both cases, our dataset is 13 years of observations of ~150 local authorities in England .footnote[[1] [Bell, A., & Jones, K. (2015)](https://www.cambridge.org/core/journals/political-science-research-and-methods/article/explaining-fixed-effects-random-effects-modeling-of-timeseries-crosssectional-and-panel-data/0334A27557D15848549120FE8ECD8D63)]

## Analysis 1:

`$$Y_i = \beta_0 + \beta_1X + e_{i}\\$$`

]

## Analysis 2:

`$$Y_{ij} = \beta_0 + \beta_1\bar{X}_{j} + \beta_2(X_{ij}-\bar{X_j}) + U_{0j} + e_{i}\\
U_{0j} \sim N(0, \sigma_{u_0})$$`

]

---

## Analysis 1:

`$$Y_i = \beta_0 + \beta_1X + e_{i}\\$$`

* Ignores the structural nature of the data (multiple time points refer to the same local authorities) - .turquoise_dark[autocorrelation - dependence between residuals]

]

## Analysis 2:

`$$Y_{ij} = \beta_0 + \beta_1\bar{X}_{j} + \beta_2(X_{ij}-\bar{X_j}) + U_{0j} + e_{i}\\
U_{0j} \sim N(0, \sigma_{u_0})$$`

In `$Y$`:

* `$U_{0j}$` adjusts for the average rate of CLA within the local authority over time (e.g. unmeasured LA-level effects)

In `$X$`:

* `$\beta_1\bar{X_j}$` measures the effect of the .turquoise_dark[local authority average spend]
* `$\beta_2(X_{ij}-\bar{X_j})$` measures the effect of .turquoise_dark[changes in that spend over time within local authorities]

]

---
class: inverse, middle

# The equations and technical reasons for a specific model aren't important: the differences in the causal structure at the between and within level does!

---

]
]

# .turquoise_dark[Our theoretical model]

Poverty causes an increase in child welfare interventions (via other mediators not shown, *family stress*, *family investment*).<sup>2</sup>

Spending on family support and other preventative services causes decreases in child welfare interventions (via more complex mediators not shown, *respite*, *ordinary help*, *therapeutic or parenting interventions*, *childcare*, *socialisation/education*)

Spending on family support and other preventative services can alleviate poverty (via more complex mediators, *additional childcare*), which can indirectly reduce rates of intervention.

]

.footnote[[2] [Skinner, G. C., Bywaters, P. W., & Kennedy, E. (2023).](https://doi.org/10.1002/car.2795) ]

---

# .turquoise_dark[But isn't this an equally plausible model..?]

Child welfare interventions cause an increase in spending on family support, preventative, and other child welfare services more generally as national and local government direct more resources to local authorities with the greatest needs. Alternatively, places with lower child welfare interventions have more money to spend on family support.

Poverty is used in funding formulas to determine the funding that a local authority receives for child welfare services, in this way, poverty can be argued to *cause* higher spending.

]

]
]

---
class: inverse, middle

# How can both of these causal models be true?

---

### Within Local Authorities over time

*'In years when a local authority is able to spend more than their average, are CWI rates higher or lower within that local authority?'* `$\beta_2(X_{ij}-\bar{X}_j)$`

]
]

### Between Local Authorities

*'When local authorities have higher spending than other local authorities, are their CWI rates higher or lower than other local authorities?'* `$\beta_1\bar{X}_j$`

]
]

---
class: inverse, middle

# In a within-between model (or comparable model, e.g. RI-CLPM<sup>3</sup>), we can separate these conflicting causal models.

.footnote[[3] [Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. (2015).](https://doi.org/10.1037/a0038889) ]

---
class: middle
background-color: white

# Findings

A **£100 increase in family support & preventative services per child sustained over two years** was associated with a **decrease of around 3.52** (89%CrI: -3.98, -3.06) children looked after in out-of-home care per 10,000 children (around 3,900 children nationally, or a 5% reduction in an average local authority CLA rate) **and a 1.65 decrease in children on a child protection plan per 10,000** (89%CrI: -2.09, -1.21) (around 1,800 children nationally, or a 3.6% decrease).

]

]

---
class: middle
background-color: white

# Findings

The **concurrent** or **immediate** same-year effect of a £100 increase in family support and preventative services spending per child was a decrease of around **1 CLA per 10,000**.

]

]

---
class: middle
background-color: white

# Findings

The **concurrent** or **immediate** same-year effect of a £100 increase in family support and preventative services spending per child was a decrease of around **1 CLA per 10,000**.

The **lagged** effect of an increase of £100 in family support and preventative services was a decrease of around **2 CLA per 10,000**.

]

]

---
class: middle
background-color: white

# Findings

The **concurrent** or **immediate** same-year effect of a £100 increase in family support and preventative services spending per child was a decrease of around **1 CLA per 10,000**.

The **lagged** effect of an increase of £100 in family support and preventative services was a decrease of around **2 CLA per 10,000**.

The remainder of the effect (~**0.5**) was an **indirect effect due to reduced rates of children on child protection plans and children in need associated with the hypothesised spending increase**.

]

]

---
class: middle
background-color: white

]

]

---
class: middle
background-color: white

---

## Variation between years and local authorities...

The model can also be extended to estimate variation **between years** and **across different local authorities**, e.g.:

`$$Y_{ij} = \beta_0 + \beta_1\bar{X}_{j} + \beta_{2}(X_{ij}-\bar{X_j}) + U_{0j} + U_{1j}(X_{ij}-\bar{X_j}) + e_{i}\\
U_{0j} \sim N(0, \sigma_{u_0})\\
U_{1j} \sim N(0, \sigma_{u_1})$$`

In the above example, we are stating: *there is a general population effect of spending on child welfare intervention rates within local authorities* ( `$\beta_{2}(X_{ij}-\bar{X_j})$` ), *but also that this effect varies depending on the local authority* ( `$U_{1j}(X_{ij}-\bar{X_j})$` ).

]

]

---
class: middle
background-color: white

]

---

## In years where the effectiveness of preventative services was worse, were preventative services also worse at reducing child poverty?

It's difficult to say, but this does not seem to be the case. However, it's important to highlight that the **effectiveness of family support and preventative services was greater in the earliest time points where such an analysis is possible** (UK LA child poverty levels between 2009-10 and 2014-15, and between 2015-16 and 2021-22 are not directly comparable).

]

#### The lagged effect of greater expenditure on family support and preventative services on rates of child poverty within local authorities

]

---

# Conclusions

* We have a core, probably under-recognised problem in a lot of child welfare research where different causal structures operate at different levels, especially where 'supply-side' resources are concerned where central governance interacts with local governance.

* Luckily, methods have been developed and are already widely used to help us untangle effects between-units and within-units; such methods also further extend to segregating stable 'between' differences, trends, and deviations from trends where a causal model suggests they may differ (perhaps due to an unmeasured external factor, e.g. CIN<sup>4</sup>)

* Untangling this causality strongly suggests that spending on family support and other preventative children's services decrease rates of children entering care or being placed on child protection plans, but this varies quite dramatically over time and across different places, creating opportunities for collaboration and learning. Further analysis currently being finalised supports existing studies<sup>5</sup> that find that the effectiveness of support also differs markedly between different populations, with older children benefitting most on average.

.footnote[<br>[5] [Bennett, D. L., et al. (2021).](https://doi.org/10.1016/j.childyouth.2021.106289) ]

---

]

# Thank you for listening!

<br>

### Dr. Calum Webb
**Sheffield Methods Institute**<br>The University of Sheffield<br>The Wave, 2 Whitham Road<br>Sheffield<br>S10 2AH

#### [c.j.webb@sheffield.ac.uk](mailto:c.j.webb@sheffield.ac.uk)

]