At the heart of Method Verification (MV) lies a simple but crucial principle: a laboratory should never operate under assumption. Regulatory bodies expect laboratories to present objective evidence that a method performs as intended through evaluation of performance characteristics such as accuracy, precision and linearity.
In order for a laboratory to generate reliable results, MV studies must be carefully planned. One of the most critical steps in planning includes defining pre-established standards known as acceptance criteria.
In this blogpost, let’s take a closer look at acceptance criteria and why they are fundamental in creating a confident method verification plan.
What is an Acceptance Criteria?
Acceptance criteria are pre-defined numerical standards that guide laboratory testing. This ranges from sample requirements to performance metrics and overall quality standards. These criteria act as the reference point to determine whether results are reliable.
While the concept is universal, each laboratory develops their own set of acceptance criteria tailored to their specific needs, which can vary depending on the performance characteristics are under evaluation.
Acceptance Criteria may include but is not limited to the following:
- Type of sample to be used
- Number of samples to be used
- Number of replicates
- Number of days tested
- Maximum Allowable Error (Total Allowable Error, CV%, SD)
All of these are essential in a method verification study as they provide objective, statistical evidence that a method functions as intended.
Who sets the Acceptance Criteria?
Each laboratory is allowed to establish their own regulations. The laboratory director should ultimately decide and approve the regulations according to its limits. As James O. Westgard said in his article, “Truth be told, the laboratory is free to do whatever it wants! However, it would be best that the laboratory director or clinical consultant sign-off on whatever limits are set.”
Interested in learning more about Acceptance Criteria for MV Experiments? Check out Cualia’s MV eBook for a comprehensive guide on criteria!
What is a Total Allowable Error?
Total Allowable Error (TEa) defines the maximum allowable deviation of an experimental result to the true value. Measuring this error provides the most direct insight into a method’s reliability, making it one of the most crucial steps in an MV study.
TEa is either expressed as a percentage or whole of the measured value because at low levels even a small difference (1.2 vs 1.4) can represent a large percent difference in ratio. This incorporate both systematic and random errors, which may be observed throughout laboratory examinations.
Why are TEas Important?
TEa’s are important since they establish the amount of error tolerated for any laboratory test. In other words, this serves as the baseline to determine whether a method can be relied for routine testing. By setting a maximum acceptable deviation from the true value of a results, TEa also help identify sources of error and improve quality of results obtained.
Errors can originate from varying sources including intrumental errors, experimental methodologies, environmental influences, and human involvements. Identifying the impact and source of these are is essential as it allows laboratories to address and minimize them.
How are TEas used in MV?
TEas serve as the benchmark to determine whether a performance characteristic passes or fails. In principle, a method’s observed values must fall within the pre-established TEa to be considered acceptable. This ensure that the method’s error - regardless of source - remains within tolerable limits.
Who established a TEa?
A laboratory establishes their preferred TEa based on the clinical significance of the test, statistical considerations like method variability, regulatory guidelines, and historical lab performance data. The determined TEa must be practical, ensuring test results are reliable for clinical decisions and achievable with the lab's current methods and equipment.
In the United States, organizations such as the Clinical Laboratory Improvement Amendments (CLIA), the College of American Pathologists (CAP), and the Food and Drug Administration (FDA) provides sources of TEas. Similarly, in Europe, the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) establishes standards that may be adopted by a laboratory.
Add in link to Cualia’s TEa database
How are Acceptance Criteria and TEa’s Used in MV?
While acceptance criteria and TEa serve different functions within MV, they share a common purpose: setting clear and measurable quality standards. In practice, acceptance criteria provides broad limits on performance. This covers a broader scope which applies to samples, metrics and replicates. While TEa determines the maximum error acceptable which determines whether a method’s performance characteristic passes or fails.
For instance in a Replication Experiment where a Glucose’s precision is being measured, the most common acceptance criteria is as follows:
- At least 2 samples representing clinical decision points or reference limits. Common practice is to use quality control samples for cost-effectivity
- 5 replicates per run
- 5 days
- At least 25 replicates per sample across 5 days
- TEa of ±8% or 6 mg/dL
Head on over to www.cualia.io and customize your acceptance criteria tailored to your own laboratory needs!
References
Westgard, James. (2020). Basic Method Validation, 4th Edition. Wisconsin, Westgard QC, Inc.
Westgard, James. (2016). Calibration Verification: Defining Criteria for Acceptable Performance. Basic Method Validation. Retrieved from: https://www.westgard.com/cal-verification-criteria.htm
Matters, Bench. (2021, December). Total Allowable Error (TEa): How Much Error Can Your Laboratory Allow?. Clinical Laboratory News.
Griffin, Karen. (2021, November). Total Allowable Error (TEa): Struggling to set quality specifications?. Maine Standards Blog. Retrieved from: https://blog.mainestandards.com/tea
https://asm.org/articles/2022/january/planning-a-method-verification-study-in-clinical-m