Comparison of Automated and Manual Methods of Estimating Methicillin-Resistant Staphylococcus aureus (MRSA) Healthcare-Associated Infection (HAI) Rates

Background: It is important for HAI surveillance systems to reliably detect real changes in HAI rates.  Manual surveillance (MAN) is routinely performed by Infection Preventionists and incorporates objective and subjectivefindings with individual interpretation.  Automated algorithms (AA) using only objective, electronic data have been developed to measure HAI more objectively and reliably, but both MAN and AA may be influenced by screening practices such as active MRSA surveillance. 

Objective: Compare three AA for HAI surveillance with each other and with MAN, and analyze the rates generated by each approach using a time-series analysis.

Methods: For MAN, we collected MRSA HAI surveillance data at the VA Salt Lake City Health Care System from fiscal years 2004 – 2008.  For AA, we electronically measured MRSA HAI rates using (1) the NHSN MDRO proxy measure for Overall MRSA Infection/Colonization Incidence Rate (NHSN); (2) the Nosocomial Infection Marker developed by MedMined and Evanston Northwestern Healthcare (NIM); and (3) an electronic algorithm we locally developed (SLC).  Patient admissions with known historic or newly discovered MRSA colonization (COL) were tabulated by month.  We tabulated the monthly incidence (per 100 admissions) of MRSA-HAI applying Hanning's linear smoother three times before measuring correlation with Pearson's r.  Pairwise correlations were calculated in six month blocks.  Unsmoothed rates were used to perform change-point analyses.  This was performed with Change-point Analyzer (Taylor Enterprises) which utilizes cumulative sum charts and a 1000 iteration bootstrap procedure.

Results: Pairwise correlation coefficients among AA for the entire period were between 0.69 and 0.79.  Correlations between MAN and AA were 0.50 (NIM), 0.65 (SLC) and 0.90 (NHSN).  When six month blocks were analyzed, correlations were comparable between MAN and NHSN and among AA until late 2007.  When all modalities were compared with COL, the greatest correlations were for RS (-0.81) and NHSN (-0.65).  Changepoint analysis of MAN found significant decreases in HAI rates in 7/05, 8/07 and 2/08.  All AA had significant drops in 7/05, but NHSN and NIM had an additional drop in 4/07 and SLC a drop in 2/08.

Conclusions: HAI metrics should not be influenced by changes in screening practice.  AA are reliable, but because NHSN subtracts out COL, NHSN rate changes are biased by increases in screening.  It is still unclear if MAN's correlation with COL reflects influence from a VA MRSA initiative incorporating active surveillance which started around 10/07.  Although decreases in HAI rates were observed around the institution of this VA initiative, we found variable results.  The interpretation of HAI rate measurements should be approached with care.  Further study is underway to compare AA and MAN to a reference standard.