Introducing new testing methods into the laboratory is an exciting event but it comes with additional quality process requirements. While the system manufacturer offers a readymade solution, laboratories are often required to verify that the system is performing according to the manufacturer’s specifications in their own laboratory environment. Verification ensures that the system functions correctly and that laboratory personnel can produce accurate and reproducible results.
System verification and validation are often interchanged – so what is the difference? According to Clinical and Laboratory Standards Institute (CLSI) a validation consists of collecting a wide set of performance data, often generated by multiple operators and study sites with a specific type and number of samples, that can be used to seek FDA clearance or IVD/CE marking 1,2. Validation work is done by the method/system manufacturer. If a lab is developing an IVD testing method of their own or is using a commercial product in a way that deviates from the manufacturers specified use, a validation study would need to be performed2.
Verification is required by various guidelines, such as ISO 15189 standard3, EUCAST4 and regional recommendations5, when a system is introduced into the laboratory for the first time to either measure an analyte previously unknown to the laboratory or to compare the new system to an existing system already in use. CLSI document M521 describes a detailed protocol on Verification of commercial microbial identification and antimicrobial susceptibility testing systems.
What is required to be tested for an antimicrobial susceptibility testing (AST) system verification? 1,2 Accuracy and precision are the main performance specifications:
|Accuracy||Agreement ≥ 90% of test results, with < 3% very major/major discrepancies or errors|
Agreement ≥ 95% of test results.
≥ 95% of QC strain results within QC specifications.
Comprehensive and limited verification are performed by testing accuracy and reproducibility but the number of isolates tested varies; comprehensive verification would be performed in the event of a new system or change in method while limited verification is sufficient when introducing a new antimicrobial agent within an existing method, for example.
|Type of change||Accuracy||Reproducibility|
|Minimum of 30 isolates||Test 5 isolates x 3 (QC strains or clinical isolates)|
|Minimum of 10 isolates||Test QC strains 3 x for 5 days or 1 x day for 20 days|
|Minimum of 30 isolates||Test QC strains 1 x for 5 days|
Reference method for laboratory verification can be selected from one of three options2:
Samples tested should cover clinical strains including strains with relevant resistance mechanisms; the selection of organisms for testing should be representative of those clinically indicated for the antimicrobial agents that will need to be verified1. Proficiency testing isolates and strains from other laboratories are also often utilized. In the US the CDC-FDA Antimicrobial Resistance Isolate Bank (AR Bank) offers various isolates that can be used for system verification6 and in Europe EUCAST has started to build recommendations for verification isolate sets7. In addition to testing for accuracy and reproducibility, quality control testing is recommended to be performed every day of testing with a set of various QC strains. See a comprehensive selection of QC strains here: View the full portfolio ›