Objectives

To explore: bedside professional reported (BPR) perceptions of safety in intensive care staff; and the relationships between BPR safety, staffing, patient and work environment characteristics.

Design An exploratory study of self-recorded staff perceptions of shift safety and routinely collected data.

Setting A large teaching hospital comprising 70 critical care beds.

Participants All clinical staff working in adult critical care.

Interventions Staff recorded whether their shift felt “safe, unsafe or very unsafe” for 29 consecutive days. We explored these perceptions and relationships between these and routine data on staffing, patient and environmental characteristics.

Outcome measures Relationships between BPR safety and staffing, patient and work environment characteristics.

Results 2836 BPR scores were recorded over 29 consecutive days (response rate 57.7%). Perceptions of safety varied between staff, including within the same shift. There was no correlation between perceptions of safety and two measures of staffing; care hours per patient day (r= 0.13 p=0.108) and Safecare Allocate (r= -0.19 p=0.013). We found a significant, positive relationship between perceptions of safety and the percentage of Level 3 (most severely ill) patients (r=0.32, p=0.0001). There was a significant inverse relationship between perceptions of safety and the percentage of Level 1 patients on a shift (r= -0.42, p=<0.0001). Perceptions of safety correlated negatively with increased numbers of patients (r= -0.44, p=0.0006) and higher percentage of patients located side rooms (r=0.63, p<0.0001). We found a significant relationship between perceptions of safety and the percentage of staff with a specialist critical care course (r=0.42. p=0.0001).

Conclusion Existing staffing models, which are primarily influenced by staff:patient ratios may not be sensitive to patient need. Other factors may be important drivers of staff perceptions of safety and should be explored further.

Trial registration- UK Health Research Authority approval was obtained (ID249248).

Data were collected over 29 consecutive days during October and November 2018. During the study period each site was visited or contacted daily by one of the investigators (CLV), to collect and verify data. 

Data collection and measures

  Bedside Professional Rating of Safety (BPR).

We asked all ICU staff to rate each shift as “safe, unsafe, or very unsafe” using a coloured sticker (green, amber or red). This question was generated from previous work into staff experiences of safety.  During staff briefings and daily visits to sites, participants were encouraged to interpret the term “safe” as their own perception of safe in its normal conversational sense in ICU.

Staff were asked to choose one sticker to rate each shift and put this onto a card labelled with the relevant date and shift (e.g. October 4^th Night shift). Participants posted the cards in data collection boxes, allowing them to complete the process anonymously, thus we were not able to identify participants by profession.

To allow us to look for relationships between perceptions of safety and other variables, including staffing, patient and work environment factors, a BPR shift score was created for each shift using the following method; we counted the responses and allocated a score of 1 for each red response, 2 for each amber response and 3 for each green response. We summed the total score and divided this by the number of responses for that shift. This created a mean shift safety score between 1 (very unsafe) and 3 (safe), which reflected the diversity and weighting of responses and is simple to reproduce. This method was discussed with senior nursing staff to confirm that they felt it reasonably summarized perceptions of safety on a given shift.

Routinely available data on staffing, patient and work environment

Staffing and workload data.

We recorded the number of staff (including nurses, doctors and allied health professionals) working clinically in the ICU during the study period. Staffing data were extracted from an electronic rostering system and verified with the nurse in charge.  Where professional groups did not use electronic rostering e.g. physiotherapists, the data were collected daily and confirmed weekly with the relevant manager. Staff who worked clinically for part of a 12-hour shift were represented as a proportion of a whole-time equivalent (e.g. a physiotherapist working for 6 hours on ICU was counted as 0.5).

We calculated workload intensity scores by summing the total number of organs in failure for each day and dividing this by the number of staff working that day. We looked at this by profession and by all professions combined. We used data from the critical care minimum data set (CCMDS)  to calculate the number of organs each patient had in failure, this was only available by 24-hour period, and therefore, data were collapsed into a 24-hour period.

We recorded the care hours per patient day (CHPPD) from our electronic roster. CHPPD is a measure that can be used to assess productivity and facilitates benchmarking. Werecorded staffing utilization data from SafeCare in the commercial rostering system provided by Allocate (AllocateSoftware Richmond 2017). This uses the composite acuity/dependency scoring system from the Safer Nursing Care Tool to provide nursing utilisation reports to support real-time deployment decisions. This is done by calculating required nursing hours (based on acuity and dependency data entered by senior nursing staff) and dividing this figure by the number of rostered nursing hours to produce a percentage of nursing hours utilization (e.g. required hours 126.4, rostered hours 115.00 = 109.9 % utilisation). 

This figure is reported in the following (partially) colour coded categories:  <90% Green, 90.1% -105.1% %, 105% Amber, 105.1%- 110%, 110.1% Red. There is meaning attached to these five categories, so that green is reported as underutilisation whilst all figures falling into amber and above are reported as overutilisation. In our analysis we reflected these categories by allocating a number to each so that green = 5 through to red =1. 

During the study if we found within-shift changes in staff numbers, patient numbers or patient acuity we recorded the status at approximately 5:00 and 17:00 (day shifts). This was a pragmatic decision that reflected the timings of local processes to manage staffing.

Patient characteristics data.

Data regarding the number of Level 1,2 and 3 patients were recorded by visiting or contacting each site daily. In ICU, patients are broadly categorised by the number of organs in failure, with Level 3 patients being the most severely ill.

Work environment data.

We recorded the total number of patients and the number of patients in side rooms per shift and confirmed this daily with a senior nurse.

Data regarding nurse training.

We recorded the percentage of nursing staff on each shift who had completed a post registration critical care course (CCC), the accuracy of the data were confirmed with local clinical nurse educators. We did not collect staff characteristic data for other professional groups as this is not routinely available, and the smaller numbers of the group sizes would impact on our ability to perform statistical analysis

Analysis

Data were collated into an Excel spreadsheet (Microsoft, Redmond, Washington State, USA).  We used descriptive statistics to analyse frequency of responses. Data were tested for normality using the D’Agostino-Pearson Omnibus test and non-parametric tests were used where appropriate. We used Spearman’s rank correlation to evaluate strength and direction of relationships; regression lines and 95% confidence intervals were plotted for illustrative purposes. Prism (v8.01 for Windows, GraphPad Software, La Jolla California USA) was used throughout.

Although the shift BPR was primarily analysed as a mean for simplicity – we further explored this using both median and a weighted approach (whereby a weighting of 5 was given to very unsafe, 3 to unsafe and 1 to safe). The purpose of this was to allow us to explore the sensitivity of the BPR tool.

The raw data is collated on an Excel spreadsheet.

Sheets-1-3 called Site 1,2,and 3 display for each site; the number of responses of each colour sticker by shift .The response rate for the number of nurses on shift and for the total number of healthcare staff on shift. 

Shift rating by different scoring methods, these are all described in the methods section above.

BPR score - This is described in the methdos section : a score of 1 per red sticker, 2 per amber sticker and 3 per green sticker was calculated and the sum divided the number  of responses.

Median - this method  calculated the score as above but used a median

SJB ,this method calculated a weigthed score so that 5 was given to very unsafe, 3 to unsafe and 1 to safe .

Workload scores for each shift, these are the care hours per patient day (CHPPD)  and the safe care scores- these are described in the methods section .

The number of patients and the percentage of side room occupancy.

Subset of responses from staff identifying as being in a side room

There are 3 further sheets intensity site 1,2 and 3 

These show the data for each shift of the number of staff in total and broken down by professional group . The total number of organs in failure  and an intensity score calculated as described in the methods section (number of organs dvivded by number of staff )

The final 3 sheets CCC site 1,2,3 show the percenatge of nursing staff on each shift who have a critical care course 

There are some shifts where no responses were returned , these values are missing and the box left empty.