Vol. 20 • Issue 7 • Page 56
Editor’s note: This is part 1 of a 2-part article.
Clinical laboratorians are well accustomed to the tried and true Levy-Jennings (LJ) chart for recording and reviewing quality control (QC) data. At the simplest level the review of the LJ chart allows the laboratorian to decide if an assay is in control prior to and following an analytic run. In the hands of an experienced user the LJ chart can reveal a high level of detail about an assay and may demonstrate long-term shifts or trends in QC data.
Despite their usefulness, the LJ chart at best is only monitoring QC materials and assay performance at single points in time. If an assay develops an unacceptable analytic error, several hundred patient samples may be affected prior to the next QC run and detection of the out-of-control condition. To minimize the number of potentially affected samples one could assay QC materials more frequently, but in the era of autoverification this strategy still only looks at snapshots of the assay performance.
Another strategy proposed has been the use of patient data to continuously monitor assay performance. This concept of Average of Normals (AON) or Moving Averages (MA) is not new to the clinical laboratory. Proposed by Hoffman and Waid in 1965,1 its implementation into the clinical laboratory has been slow while it has found a home in the manufacturing industry as part of their quality assurance (QA) programs.
The MA concept has not yet achieved wide use in the clinical chemistry laboratory, but it has had some integration into QA programs, especially with Bull’s Algorithm in the hematology laboratory to monitor Red Blood Cell indices (MCV, MCH and MCHC). Perhaps part of the reason why MA has thus far failed to become a routine part of the laboratory’s QA program is that the necessary programs in real-time have not been widely available. Some in vitro diagnostics manufacturers have recently made real-time MA programs available as part of their laboratory information systems (e.g., Abbott Diagnostics Instrument Manager, Beckman Coulter Remisol Advance, Roche Middleware Solutions, Siemens Advia Centralink and Siemens EasyLink). It appears that MA programs and protocols are on the verge of wider acceptance in the clinical laboratory.
To enhance the QA program in our clinical chemistry laboratory at Dartmouth-Hitchcock Medical Center, we have started the process of establishing a MA program from our middleware vendor. This MA program is a Roche Diagnostics provided solution from Data Innovations and allows the laboratorian to configure user defined protocols to monitor patient means to uncover systematic errors in advance of detection by a QC event. While this program is highly flexible and applicable to many assays, it is not an out-of-the-box solution. For this reason we are highlighting some of the steps we’ve undertaken to facilitate adoption, and likely enhancement, by others in the clinical laboratory profession.
To effectively use any MA program the user first needs to determine how many patient samples will be averaged for each mean data point. Detailed guidance can be found in several articles.2-4 It is not sufficient to simply choose the same number of patient samples for each assay that you wish to monitor, as analytes with a low interindividual index of variability will require fewer points to detect an analytic shift in assay performance while those with a large interindividual index will require more samples.
The laboratory’s patient population mean and standard deviation (SD) as well as the analytical SD for each assay for which a protocol is to be developed must initially be determined. While the mean and SD should approximate the assay reference interval it is still important to determine the mean and SD, as slight differences will greatly affect the MA protocol performance. The patient mean and SD can be determined in a number of ways, and some LISs are capable of calculating these values from historical data. While it would be ideal to have the population means and SD prior to beginning a MA program, it is not necessary as we were able to determine the means and SD by collecting several hundred data points with the aid of the MA program.
In establishing a patient population mean and SD it is also important to understand the demographics of the patient population. If, for instance, the patient population is composed of both ambulatory outpatients and hospitalized inpatients, filters will need to be applied to exclude from the statistical analysis certain populations such as emergency room and dialysis patients and those admitted to intensive care units. While these patients are few in number their laboratory values often differ greatly from the ambulatory population and inclusion of the outlying values would unduly influence the overall mean.
Once the above information is collected the number of samples (N) to be averaged for each assay can be determined by calculating the ratio of the population SD (Sp) and assay SD (Sa). This ratio can then be applied to previously published nomograms such as the one in the Cembrowski et al article.2 If the N is determined using the nomogram from the aforementioned article, it is important to note that this nomogram was designed to detect an analytic shift of 2 SD with a probability of 0.5 (or a 50% chance). If the user wishes to detect a larger or smaller analytical shift changing the N used in the MA calculation will become necessary to maximize the probability of detection while minimizing the probability of false detection.
Finally, it is important to realize that there are assays that will not lend themselves to the MA procedures. Infrequently ordered assays in which the N required for optimal monitoring is close to or exceeds the total number of daily patient samples run will not generate sufficient data for monitoring. In addition, assays such as creatinine kinase and lactate dehydrogenase that demonstrate very large interindividual variability are not likely candidates for MA protocols.
Continuing Process Improvement
From the steps outlined above we are monitoring a number of assays with straight SD limits and establishing protocols that use the Standard Error of the Mean (SEM or SD in the Data Innovations’ MA program). The SEM, calculated as the SD divided by the square root of N (SD/VN), results in warning and error limits that are significantly smaller than the SD rules alone. The use of SEM limits requires establishment of appropriate filters or truncation limits to effectively eliminate extreme laboratory values and minimize the probability of false rejection of the analytical run.
While we have established several working MA protocols, we continue to improve upon them and create new ones to enhance our repertoire. Establishment of an MA program will require a thorough analysis of your patient population and assay performance prior to activating protocols. Failure to tailor the program to your patient population will likely result in the detection of false analytical shifts due to poorly established means or overly generous truncation/exclusion limits. Initiating an MA program represents a significant up-front investment in time, but if the program is set up correctly it will result in an analytical tool to monitor both reagent and instrument stability.
The implementation of an MA program, such as the one we are evaluating, does not supplant a laboratory’s current QC/QA program; rather, it is intended to add another layer of sophistication at little or no cost.
Dr. Cervinksi is director of Clinical Chemistry, Dartmouth-Hitchcock Medical Center, Lebanon, NH. Frank Polito is supervisor of Chemistry.
1. Hoffmann RG, Waid ME. The “average of normals” method of quality control. Am J Clin Pathol 1965;43:134-41.
2. Cembrowski GS, Chandler EP, Westgard JO. Assessment of “average of normals” quality control procedures and guidelines for implementation. Am J Clin Pathol 1984; 81: 492-9.
3. Ye JJ, Ingels SC, Parvin CA. Performance evaluation and planning for patient-based quality control procedures. Am J Clin Pathol 2000;113:240-8.
4. Westgard JO, Smith FA, Mountain PJ, Boss S. Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control. Clin Chem 1996;42:1683-8.