Main Menu Browse by Day or Program Author Keywords

305 Interobserver Variability in Bloodstream Infection Determinations Using National Healthcare Safety Network Definitions

Saturday, April 2, 2011: 2:45 PM
Cortez Ballroom (Hilton Anatole)
Paul Malpiedi, MPH , Centers for Disease Control and Prevention, Atlanta, GA
Bala Hota, MD, MPH , Stroger Hospital of Cook County/Rush University Medical Center, Chicago, IL
Shelley Magill, MD, PhD , Centers for Disease Control and Prevention, Atlanta, GA
William Trick, MD , John H. Stroger, Jr. Hospital of Cook County, Chicago, IL
Teresa Horan, MPH, CIC , Centers for Disease Control and Prevention, Atlanta, GA
Rosie Lyles, MD, MS , John H. Stroger, Jr. Hospital of Cook County, Chicago, IL
John Martin, MPH , Premier, Inc., Philadelphia, PA
Chris Craver, MA , Premier, Inc., Charlotte, NC
Scott K. Fridkin, MD , Centers for Disease Control and Prevention, Atlanta, GA
Background: CDC’s National Healthcare Safety Network (NHSN) provides standard surveillance definitions for healthcare-associated bloodstream infections (BSI).  Variations in interpretation and understanding of definition criteria can reduce interobserver agreement and the accuracy of inter-facility comparisons.

Objective: To determine the agreement between 2 experienced infection preventionists (IPs) reviewing medical records of patients with positive blood cultures to identify NHSN-defined BSI.

Methods: NHSN hospitals using Premier Inc.’s SafetySurveillorTM software system were recruited to participate.  All blood culture results from 1/2008–6/2009 were extracted from the SafetySurveillorTM data warehouse.  A positive blood culture episode eligible for IP review was defined as a patient’s first positive blood culture in a 30 day period obtained > 2 days after admission, excluding single positive cultures for common skin contaminants.  Two experienced IPs trained in use of NHSN definitions reviewed medical records of a random sample of patients with eligible positive blood culture episodes and independently categorized episodes into 1 of 5 groups: contaminant, community-onset (CO) BSI, primary non-central line associated BSI, central line-associated BSI (CLABSI), or healthcare-associated infection with secondary BSI.  We calculated descriptive statistics, compared proportions of IP determination categories using McNemar’s test, and assessed interobserver agreement using the kappa (κ) statistic.

Results:  14 hospitals participated, yielding a total of 5,929 positive blood culture episodes eligible for sampling.  1,140 (19%) episodes were sampled.  Of these, 1,009 (89%) were categorized by both reviewers.  Overall agreement was 64% (κ 0.35, 95% CI 0.30, 0.39).  Although IPs categorized similar proportions of episodes as CLABSI (~9%), the proportion categorized as contaminants or CO BSI differed between IPs (Table).  Agreement was fair to moderate in each category.  One or both IPs categorized 155 episodes as CLABSI; 127 (82%) of these episodes had a discrepancy in IP categorization, where one reviewer categorized the episode as secondary BSI (71, 56%), CO BSI (33, 26%), or contaminant (21, 17%).  

 

Category

Reviewer 1 Determinations (N, %)

Reviewer 2 Determinations (N, %)

p-value

Kappa

Secondary BSI

657 (65.1)

640 (63.4)

0.31

0.39

Community-onset BSI

225 (22.3)

143 (14.2)

0.01

0.41

CLABSI

90 (8.9)

93 (9.2)

0.79

0.23

Contaminant

22 (2.2)

112 (11.1)

0.01

0.21

Primary non-central line associated BSI

15 (1.5)

21 (2.0)

0.25

0.21

Conclusions: We found evidence of limited agreement between experienced IPs in applying the NHSN BSI definition.  Continued efforts to understand reasons for limited agreement are needed.  One potential solution is to apply algorithms that only use objective criteria, which could dramatically improve agreement.