Construct CUSUM charts for hospital performance

Lena Hubig

2019-03-19

Overview

This vignette describes CUSUM charts based on a simulated false-signal-probability for hospital performance data in the R package cusum. This is a practical guide to constructing and evaluating non-risk-adjusted and risk-adjusted CUSUM charts following Steiner et al. (Biostatistics 1.4 (2000), pp. 441-52).

The cusum packages takes different factors into account that influence the signal rate of CUSUM charts. Some are given by the process to be monitored; these factors are:

The primary control input when constructing a CUSUM chart is the control limit. The control limit signals performance deterioration once crossed by the cumulated sum.

The control limit depends on a number of variables:

Motivating example

To illustrate how cusum can be used for monitoring, we employ a simple and artificial data set generated to closely follow the performance data of German hospitals for one non-risk-adjusted performance indicator and one risk-adjusted performance indicator in 2016 and 2017.

risk-adj. Indicator Description Further explanation (in German)
NO Ratio of observed to expected cases of severe stroke or death under open carotid stenosis surgery pdf (p4)
YES Preoperative stay more than 24 hours for patients with proximal femur fracture pdf (p23)

Non-risk-adjusted performance indicator

data("cusum_example_data", package = "cusum")

head(cusum_example_data)
#>   t     y year
#> 1 1 FALSE 2016
#> 2 2 FALSE 2016
#> 3 3 FALSE 2016
#> 4 4 FALSE 2016
#> 5 5 FALSE 2016
#> 6 6 FALSE 2016

Risk-adjusted performance indicator

data("racusum_example_data", package = "cusum")

head(racusum_example_data)
#>   t     y   score year
#> 1 1 FALSE 0.00237 2016
#> 2 2 FALSE 0.00237 2016
#> 3 3 FALSE 0.02412 2016
#> 4 4 FALSE 0.01893 2016
#> 5 5 FALSE 0.00725 2016
#> 6 6 FALSE 0.00810 2016

First, CUSUM charts are constructed on performance data from 2016 (Phase I), and then applied and evaluated on performance data from 2017 (Phase II).

cusum_example_p1 <- cusum_example_data[cusum_example_data$year == 2016, ]
cusum_example_p2 <- cusum_example_data[cusum_example_data$year == 2017, ]

racusum_example_p1 <- racusum_example_data[racusum_example_data$year == 2016, ]
racusum_example_p2 <- racusum_example_data[racusum_example_data$year == 2017, ]

Non-risk-adjusted CUSUM chart

Simulation of CUSUM Control Limits

We get the control limit of our CUSUM chart by simulating for a false signal probability depending on sample size and accepted failure probability.

We can estimate the accepted failure probability by taking the average of Phase I. Alternatively, we could also define an accepted failure probability politically

Then, control limits can be simulated using cusum_limit_sim.

Applying CUSUM Charts

Monitoring via CUSUM charts is applied on performance data from 2017 (Phase II) and the control limit cusum_limit. It can be calculated using cusum.

Performance is as expected during the first half of monitoring, and then deteriorates. We get a signal at t=547. If reset==TRUE, the CUSUM resets after each signal.

Evaluating CUSUM charts

The false signal probability of a CUSUM chart can be simulated using cusum_alpha_sim given a predefined control limit.

We see that cusum_alpha equals our previously defined false signal probability of 0.05.

Risk-adjusted CUSUM chart

Simulation of RA-CUSUM Control Limits

Control limits of RA-CUSUM charts are simulated for a false signal probability depending on sample size and risk distribution.

RA-CUSUM Control limits can be simulated using racusum_limit_alpha.

Applying RA-CUSUM charts

Monitoring via RA-CUSUM chart is applied on performance data from 2017 (Phase II) and the control limit racusum_limit. It can be calculated using racusum.

Performance is as expected during the first half of monitoring, and then deteriorates. We get a signal at t=865. If reset==TRUE, the CUSUM resets after each signal.

Evaluating RA-CUSUM charts

The false signal probability of a CUSUM chart can be simulated using cusum_alpha_sim.

We see that racusum_alpha is similar to our previously defined false signal probability of 0.05. Deviation is possible due to a slight change in risk population.