AUTHOR VALIDATION COMMITTEE DOCUMENT NUMBER EQU5104 EFFECTIVE (OR POST)

1. Plot the data in EP Evaluator Two Instrument Comparison Module or a similar regression analysis program.

1. Plot the reference method on the X axis and the method being validated on the Y axis.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table. (Appendix 2)

3. Enter the results for the reference (X) and the new (Y) methods.

4. Evaluate the statistics by clicking on Print Preview or on the statistics tab.

5. The correlation coefficient (R) must be >0.975

If	Then
R <0.975	Data does not extend over acceptable range. More data must be evaluated over larger range.
R >0.975	Proceed with step B below to evaluate acceptability.

2. Using this method, 95% of the data points from the comparative method must be within Total Allowable Error limit of the reference method (refer to SMILE Chemistry TE Limits table, Appendix 2). Use one of the methods below to evaluate.

1. Using a regression analysis program or spreadsheet (such as the SMILE Accuracy Pack, Appendix 1) calculate the “Error Index” to measure the difference between the two methods as a ratio of the Total Allowable Error.

   1. The Error Index can be calculated by subtracting the reference method data point (X) from the method being validated data point (Y) and dividing by the Total Allowable Error (TEa). The equation is: (Y-X)/TEa.

   2. Use the percent TEa or the minimum detectable difference (absolute TEa), whichever is greater.

   3. The absolute TEa value is calculated by converting the TEa percentage to a decimal and multiplying by the reference method data point.

   4. The Error Index is measured for each X-Y pair, and must fall within -1 and 1. If more than 5% of the specimens have an Error Index of less than -1 or greater than 1, the accuracy experiment fails.

1. The EP Evaluator program uses a statistic called the “Error Index” to measure the difference between the two methods as a ratio of the Total Allowable Error.

   1. The Error Index can be calculated by subtracting the reference method data point (X) from the method being validated data point (Y) and dividing by the Total Allowable Error (TEa). The equation is: (Y-X)/TEa.

   2. EP Evaluator uses the percent TEa or the minimum detectable difference (absolute TEa), whichever is greater.

   3. The absolute TEa value is calculated by converting the TEa percentage to a decimal and multiplying by the reference method data point.

   4. The Error Index is measured for each X-Y pair, and must fall within -1 and 1. If more than 5% of the specimens have an Error Index of less than -1 or greater than 1, the accuracy experiment fails. (See Appendix 3 for an example EP report).

2. Evaluate results using the table below:

If	Then
95% of Error Indices are acceptable	Accuracy is acceptable Proceed with Linearity experiments
<95% of Error Indices are acceptable	Proceed with step D below.

2. Plot the data in a regression analysis program such as the SMILE Accuracy Pack (Appendix 1) or EP Evaluator Alternate (Quantitative) Method Comparison module. Deming Regression is preferred, however if not available regular regression is acceptable provided that the Correlation Coefficient (R) is >0.975.

1. Plot the reference method on the X axis and the method being validated on the Y axis.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table (Appendix 2).

3. Enter Medical Decision Points

   1. A Medical Decision Point (MDP) is the concentration of the analyte at which a medical decision is triggered. You may enter up to five values. These values are plotted on the graph, and the program computes a predicted Y value and confidence interval at each X decision point.

   2. For many analytes, the medical decision points correspond to the lower and upper limits of the normal range. It is advisable to also include medical decision points at the DAIDS Toxicity limits and/or laboratory established critical values.

4. Evaluate using one of the methods described below:

1. If using a regression analysis program or spreadsheet calculate and evaluate the 95% confidence limits and error indices. The SMILE Accuracy Pack, (Appendix 1) performs these calculations automatically.

1. Subtract the X Method MDP from the low and the high 95% confidence limits for each Medical Decision Point.

2. Calculate Error Indices for each end of the 95% confidence limits for each Medical Decision Point as follows: Divide the difference (from step i. above) by the SMILE TE Limit absolute value.

1. Use the percent TEa or the minimum detectable difference, whichever is greater.

2. To calculate the absolute TEa value, convert the TEa percentage to a decimal and multiply by the reference method data point.

1. All error indices must fall within -1 and 1. An Excel spreadsheet is provided to calculate Error Indices (See Appendix 5 SMILE Error Index Calculator).

1. If using EP Evaluator program create the report by selecting the Print Preview button.

2. Review the Medical Decision Point Analysis data on page 1 of the report. (See Appendix 4 for an example EP report)

3. Subtract the X Method MDP from the low and the high 95% confidence limits for each Medical Decision Point.

4. Calculate Error Indices for each end of the 95% confidence limits for each Medical Decision Point as follows: Divide the difference (from step c above) by the SMILE TE Limit absolute value.

1. Use the percent TEa or the minimum detectable difference, whichever is greater.

2. To calculate the absolute TEa value, convert the TEa percentage to a decimal and multiply by the reference method data point.

3. All error indices must fall within -1 and 1. An Excel spreadsheet is provided to calculate Error Indices (See Appendix 5 SMILE Error Index Calculator).

5. Evaluate acceptability using the table below:

If	Then
All error indices fall between -1 and 1	Proceed with step 6 below.
Any error indices fall outside -1 and 1	Consider possible causes of inaccuracy. Troubleshoot before repeating accuracy studies. Consult SMILE for assistance.

6. Six Sigma method acceptability evaluation

1. Plot the data in a Six Sigma analysis program such as the SMILE Accuracy Pack (Appendix 1) or EP Evaluator Six Sigma Metrics module.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table.

3. Enter Medical Decision Points.

4. Enter the slope and intercept calculated in step D above.

1. If using EP Evaluator, use the Deming Regression slope and intercept calculated by the program.

2. If EP Evaluator is not used, it is acceptable to use slope and intercept calculated using regular regression.

5. Enter the Means and %CVs from your Between Day Precision experiments.

   1. If using EP Evaluator create the report by selecting the Print Preview button. (See Appendix 6 for an example EP report).

   2. The SMILE Accuracy Pack and EP Evaluator calculate a Sigma metric for each Medical Decision point.

   3. Sigma metric=(TEa-Systematic Error)/Random Error

6. Evaluate acceptability at each MDP using the table below:

If	Then
The Sigma metric less than 2.0	The method has unacceptable performance and does not meet your requirement for quality, even when the method is working properly. It is not acceptable for routine operation.
The Sigma metric is between 2.0-3.0	The method has marginal performance and provides the necessary quality when everything is working correctly. This method will require: 4-8 controls per run well-trained operators reduced rotation of personnel more aggressive preventive maintenance careful monitoring of patient test results continual efforts to improve method performance
If the Sigma metric is between 3.0-4.0	The method has fair performance and meets your requirement for quality and can be well-managed in routine operation. This method will require 2 control measurements per run using standard Westgard QC rules.
If the Sigma metric is between 4.0-6.0	The method has good performance and is clearly acceptable and can be well-managed in routine operation with only 2 control measurements per run using standard Westgard QC rules.
If the Sigma metric is >6.0	The method has Six Sigma performance and is even easier to manage and control.

4. References

   1. GCLP Guidelines, Standard for Performance Specifications.

   2. Clinical and Laboratory Standards Institute (CLSI). User Verification of Performance for Precision and Trueness: Approved Guideline-Second Edition. CLSI document EP15-A2 (ISBN 1-56238-574-7). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

   3. NCCLS. (Currently CLSI) Evaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline—Second Edition. NCCLS document EP5-A2 (ISBN 1-56238-542-9). NCCLS, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA.

   4. Clinical and Laboratory Standards Institute (CLSI). Method Comparison and Bias Estimation Using Patient Samples: approved Guidelines- Second Edition. CLSI document EP9-A2 (ISBN 1-56238-472-4). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

   5. Clinical and Laboratory Standards Institute (CLSI).Preliminary Evaluation of Quantitative Clinical Laboratory Measurement Procedure: Approved Guidline – Third Edition. CLSI document EP10-A3 (ISBN 1-56238-622-0). ). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

   6. Clinical and Laboratory Standards Institute (CLSI).Estimation of Total analytical Error for Clinical Laboratory Methods. Approved 2003. CLSI document EP21-A (ISBN 1-56238-502-xX. Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

   7. EP Evaluator Release 8, David G. Rhoads Associates Inc., www.dgrhoads.com.

   8. James O. Westgard, Online Validation Training, Westgard QC, Inc. www.westgard.com, Sections 11-Determining Bias,12- Estimating Trueness, and 13- Judging Method Acceptability.

617206.doc 3 of 8 30-Apr-09