If you work in a hospital these days, you've probably received the invitation: Take a survey about how well you, your team and your hospital do at protecting patients from harm, and how empowered you feel to do the right thing.

In fact, you've probably received many invitations and reminders to take the survey, as your hospital tries to get employees to respond. It's all in the name of gauging something called "patient safety culture."

But a new study questions whether such surveys actually measure how well a hospital is doing at keeping patients safe.

A team of researchers based mainly at the University of Michigan Medical School and the VA Ann Arbor Healthcare System did the study using data from hundreds of hospitals. They published their findings in BMJ: Quality & Safety.

What they found surprised even them. Overall, hospital units' patient safety culture scores didn't match up with how well the units did on a key patient safety goal: reducing two types of infections that patients can catch during their hospital stay.

Big disconnect, big implications

This disconnect — between safety culture survey results and actual safety improvement — happened in hospital units throughout the country that had signed on to a national patient safety project.

All the units were working to reduce central line associated bloodstream infection (CLABSI), from devices used to deliver medicine into the bloodstream, and catheter-associated urinary tract infection (CAUTI), from devices used to collect urine.

The hospitals gave staff technical help — tools, training, new procedures and other support — to reduce incidence of these infections. They also fostered cultural changes aimed at improving teamwork and encouraging staff to speak up to stop an unsafe situation. And they surveyed employees about patient safety at the start and toward the end.

"We hypothesized that those that did better on survey measures of safety culture would achieve better infection rates, especially given that there had been so much effort put into trying to improve safety culture in these collaboratives," says lead author and assistant professor Jennifer Meddings, M.D.

But, she explains, "in the data from both collaboratives, there was no connection between the safety culture scores derived from the surveys and the actual decline in infection rates on the units. We think this indicates it's much more difficult to detect and measure safety culture than has been thought." She also hopes that new or revised surveys will be tested for their ability to measure real change on a small scale — a process called validation — before they're launched on a large scale.

Survey says …

Many hospitals pour resources into trying to get employees to take the surveys that form the basis for the safety culture scores.

Even so, less than half of the hospital staff on the units in these projects took them. The length of the survey — 42 questions for each staff member — may have contributed to this. But with survey results from only a portion of employees, the score that comes from those results may be inaccurate.

A previous study that looked at baseline data from the CLABSI-reduction project found that performance on a measure dubbed "safety climate profile," which researchers elsewhere generated using some combinations of the individual measures, did correspond with reductions in infections.

But Meddings and her colleagues wanted to use the safety culture data that hospitals actually receive once their employees have taken the survey, as recommended by the survey tool instructions, without creating any new types of composite scores. They looked at how well hospital units' scores on each one of the 42 questions in the Hospital Survey on Patient Safety Culture (HSOPS) survey corresponded with rates of CLABSI and CAUTI in those units.

Though the hospital units in the study managed to reduce CLABSI rates by 47 percent and CAUTI rates by 23 percent overall, the changes in HSOPS scores were minor. For some, hospital safety scores worsened despite improvements in infection rates.

Meddings says the findings should give the patient safety community reason to evaluate how surveys are used as part of larger safety-focused efforts, which are often routine components for participation in collaboratives.

"Unexpectedly, the data from these collaboratives showed no association of better infection rates for hospitals with better safety culture scores," she says. "These data suggest that either the infection rates were improved by improving other aspects of care than safety culture such as improving skills in catheter use, or that these surveys simply do not capture safety culture well."

She adds, "We hope these analyses inform future collaboratives to be designed in way to better assess which components of multi-component interventions are most important to reduce infection and to reduce the routine use of culture surveys for busy clinicians until there are clearer links between the surveyed measures and culture observed on the units."