TROUBLESHOOTING FAILED QC

Download Troubleshooting Failed QC. Dr Douglas Chesher. Clinical Biochemistry, NSW Health Pathology, Royal. North Shore Hospital. Northern Clinical,...

1 downloads 682 Views 1MB Size
Troubleshooting Failed QC Dr Douglas Chesher Clinical Biochemistry, NSW Health Pathology, Royal North Shore Hospital Northern Clinical, School University of Sydney

Outline • • • •

Types of errors Approach to troubleshooting Problem fixed, now what? Lot number changes

Experimental Errors • Systematic Error • Random Error • Blunders

Systematic Errors • Due to identified causes and can, in principle, be eliminated. Errors of this type result in measured values that are consistently too high or consistently too low. • Types – Instrumental – Observational. For example, parallax in reading a meter scale. – Environmental – Theoretical. Due to simplification of the model system or approximations in the equations describing it. http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html

Random Errors • Random errors are positive and negative fluctuations that cause about one-half of the measurements to be too high and one-half to be too low. – Sources of random errors cannot always be identified.

• Types of random error – Observational. For example, errors in judgment of an observer when reading the scale of a measuring device to the smallest division. – Environmental. For example, unpredictable fluctuations in line voltage, temperature, or mechanical vibrations of equipment.

http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html

Blunders • An outright mistake. • Should stick out like sore thumbs if we make multiple measurements or if one person checks the work of another.

http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html

“My QC Failed”

Response to a failed QC • Stop • Do not just rerun the QC – Unless part of your rule (eg. 1 2s rerun ‘once’) – Regression towards the mean

Response to failed QC • Do not just recalibrate as a routine. – Calibrations add noise and increase imprecision.

150 145 140 135 130 125 120 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 148 146 144 142 140 138 136 134 132 130 128 126 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59

Points of failure • Instrument Failure: – Check for error messages (printed or displayed). Refer to the troubleshooting section of the instrument manual.

• Reagent, Quality Control Materials, Calibrators: – In correct reconstitution, i.e. wrong or contaminated diluent or use of the wrong volume of diluent. – In correct storage, e.g. left at room temperature for excessive amount of time. – Prepare fresh (allow enough time to dissolve) – NOTE: Bulk liquid reagents may be contaminated.

• Human: – Reread the method - Check whether the correct sample volumes or reagents were used. Make sure that a step in the procedure was not missed. – Were the correct parameters entered?

Have other tests also failed? • Blunders – Swapped QC levels

• Short sampled

Are there any instrument flags?

What rule has failed? • Systematic error – 1 3s, 2 2s, 4 1s, 10x etc

• Random error – 1 3s, R4s

Review your QC Chart 148 147 146 145 144 143 142

Systematic error Serum X -3SD -2SD -1SD Mean 1SD 2SD 3SD

Has anything changed? • Just calibrated • New bottle of reagent • New lot of reagent

Quarterly maintenance

New reagent lot

Review points of failure • Reagents – Correct reagent?, sufficient volume, shelf expiry date, on-board expiry date

• Quality Control Material – Material, lot no. & assigned value, preparation, shelf expiry date, in-use expiry date, storage conditions.

• Calibrators – Correct material, lot no. & assigned values, preparation, storage, expiry date. Inspect calibration trace of last calibration.

Review points of failure • Instrument – Is maintenance up to date? – Recheck flags, probes, lamps, cuvettes, water bath.

QC Failure

Corrective action required immediately ?

Monitor QC performance and correct non-urgent faults as appropriate

Y Instrument flags?

N Correct instrument fault

N Maintenanc e up to date?

Y

Perform all outstanding maintenance

Y

Reagents OK?

N Correct reagent fault

Y N

Make up and run correct QC

N

Make-up and run correct calibrator

N

Fix identified instrument fault

QC OK

Y Calibrators OK?

Y

Call for Help!

Instrument OK?

Rerun QC to confirm problem fixed

Problem Solved!

Problem solved, now what? • Run QC for evidence that problem has been solved. • Document what you have done. Troubleshooting logs, QC annotation. • Address patient results from previous good QC to when QC failure occurred. – Repeat testing

• Exclude / Inactivate failed QC from data analysis if cause of the outlier is clearly identified.

Reagent lot changes • CLSI EP26-A User Evaluation of BetweenReagent Lot Variation; Approved Guideline. • “The protocol attempts to balance the need to reliably detect clinically significant change in reagent performance that may affect patient results with the recognition that reagent lot verification is a relatively frequent task that puts demands on the laboratory’s limited resource”

QC material is not always commutable • Shift in QC may not reflect a similar shift in patient results • Just because QC does not show a shift, does not mean the patients will not show a shift.

• Verify all new lots before they are put into use. – Need only perform once for group of labs if using same lot QC and reagent.

Test patient samples with current and candidate reagent lots

Estimate average difference between lots

Average diff < CD?

No Investigate lot difference. Do not report patient results with this lot

Yes

Was QC acceptabl e with new lot?

Yes

Candidate lot acceptable for patient testing

No Update QC targets

CD: Critical Difference

Defining the critical difference • Evaluation of comparability based on clinical outcomes. • Evaluation of comparability based on clinical decisions – derived from either biological variation or – from data based on clinicians’ opinions.

• Published professional recommendations. • Performance goals set by regulatory bodies or organizers of External Quality Assessment Schemes. • Goals based on the current state of the art as demonstrated by data from EQA or from current publications on methodology.

Determine the statistical power

Determine the number of target concentrations

Determine the number of samples to be tested and the rejection limit

Determine the number of replicates per sample

• Alpha (Type I error, false positive) • Probability of detecting an error when there is no error. • Typically set at 0.05 • Beta (Type II error, false negative) • Probability of not detecting a difference when there is a difference • Typically set at 0.20 to 0.05 • Statistical Power = 1 – Beta. Generally measure at clinical decision points, often similar to the QC levels. Statistical tables may suggest that only one patient is required but it is recommended that at least three samples are used to avoid random errors. If difficult to get sufficient samples, replicate analysis may provide the necessary additional statistical power. However, under normal circumstances measure in singlicate

Conclusions • Do not automatically re-run QC or recalibrate an assay. • Type of QC failure and Quality Control Charts may give a clue to the nature of the problem. • Have a structured approach to troubleshooting. • Have a process for managing your patient results • Prevention is better than cure – Manage introduction of new lots to the laboratory