Objective: To compare 3 methods of HAI identification: traditional NHSN surveillance, a commercial data-mining approach, and International Classification of Diseases 9th Revision (ICD-9) coding.
Methods: We monitored catheter-associated bloodstream infections (CA-BSIs) for 3-month periods in 2009 in each of 6 intensive care units (ICUs) of a large university hospital. NHSN surveillance was completed by experienced infection control practitioners. A Nosocomial Infection Marker report was generated by data-mining using the MedMined™ Data Mining Surveillance service (CareFusion Corporation). ICD-9 coding was carried out by veteran billing personnel. The reports were derived independently and then compared. Cases classified differently by the 3 approaches were reviewed by 2 infectious diseases physicians to determine their appropriate designation as either true or false positive CA-BSIs. We calculated the sensitivity and positive predictive value of each of the 3 methodologies.
Results: A total of 56 CA-BSI were identified in the 6 ICUs (bone marrow transplant 20, cardiac 3, medical 13, neurosurgical 6, surgical 11, high-risk neonatal 3) during 7,154 catheter-days. The overall CA-BSI rate was 7.9 per 1000 catheter-days. NHSN surveillance detected 46 cases, and there were no false-positives. Data-mining identified 48 cases but also 44 false-positives. ICD-9 coding documented 7 cases and 5 false-positives. Overall sensitivities of the 3 metrics were: NHSN surveillance 0.82, data-mining 0.86, and ICD-9 coding 0.13. Positive predictive values of the 3 measures were: NHSN surveillance 1.0, data-mining 0.52, and ICD-9 coding 0.58.
Conclusions: While comparable in regards to sensitivity, there was an important difference in the positive predictive value of traditional NHSN surveillance versus data-mining. Any consideration of implementing an automated surveillance system must take into account the expertise and time necessary to review cases and eliminate false-positives. Otherwise, data-mining was too inaccurate for use in providing feedback to ICUs or for public reporting. The ICD-9 coding scheme lacked both sensitivity and predictive power as a metric for identifying CA-BSIs. The NHSN approach has imperfect discriminatory power and inter-rater reliability, but the adoption of either surrogate metric does not enhance HAI reporting performance from an accuracy or efficiency standpoint.