

39.01  Performance Measures  
Overview: The usefulness of a test
is often judged in how well it makes the diagnosis for the presence or
absence of a disease.
A person with the disease who has a "positive" test is termed a true positive, whereas a person with the disease but a "negative" test result is termed a false negative. A person without disease who has a "positive" result is termed a false positive, while a person without disease having a "negative" result is termed a true negative. In real life things are not always clear cut; the distinction between positive and negative in a test result is sometimes artificial while it is not always possible to say if a person does or does not have a disease. 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.01  Incidence Rate  


incidence rate = 
((A) / (a + b + c + d)) / T)  
where: 



TOP  References:  
Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983., page 51 

39.01.02 
Prevalence 

Overview: Prevalence is all patients with disease divided by all patients tested. This is also termed the "prior probability." 

prevalence = 
(a + c) / (a + b + c + d)  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210


39.01.03  Sensitivity 

Overview: Sensitivity is the truepositive test results divided by all patients with the disease. 

sensitivity = 
(a / (a + c))  
where: 



Comments 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.04 
Specificity 

Overview: The specificity of a test is the truenegative test results divided by all patients without the disease. 

specificity = 
(d / (b + d))  
where: 



Comments


TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.05 
FalseNegative
Rate 

Overview: The false negative rate for a test is the falsenegative test results divided by all patients with the disease. 

falsenegative rate = 
(c / (a + c))  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.06 
FalsePositive
Rate 

Overview: The false positive rate for a test is the falsepositive test results divided by all patients without the disease. 

falsepositive rate = 
(b / (b + d))  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.07 
Positive
Predictive Value 

Overview: The positive predictive value is truepositive test results divided by all positive test results. This is also referred to as the predictive value of a positive test. This is equivelent to Bayes's formula for posttest probability given a positive result. 

positive predictive value = 
(a / (a + b))  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.08 
Negative
Predictive Value 

Overview: The negative predictive value is the truenegative test results divided by all patients with negative results. This is also referred to as the predictive value of a negative test. This is equivelent to Bayes's formula for posttest probability given a negative result. 

negative predictive value = 
(d / (c + d))  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Panzer RJ, Black ER, Griner PF. Interpretation of diagnostic tests and strategies for their use in quantitative decision making. pages 1728. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.09 
Overall
Accuracy, or Diagnostic Efficiency 

Overview: The overall accuracy of a test is the measure of "true" findings (truepositive + truenegative results) divided by all test results. This is also termed "the efficiency" of the test. 

overall accuracy = 
((a+d) / (a + b + c + d))  
where: 



TOP  References  
Braunwald E, Isselbacher KJ, et al (editors). Harrison's Principles of Internal Medicine, 11th edition. McGrawHill Book Publishers. 1987. page 7 Goldman L. Chapter 10: Quantitative aspects of clinical reasoning. pages 4348. IN: Isselbacher KJ, Braunwald E, et al. Harrison's Principles of Internal Medicine, Thirteenth Edition. McGrawHill. 1994. Clave P, Guillaumes S, Blanco I, et al. Amylase, Lipase, Pancreatic Isoamylase, and Phospholipase A in Diagnosis of Acute Pancreatitis. Clin Chem. 1995; 41:11291134. <with ROC> Speicher C, Smith JW Jr.. Choosing Effective Laboratory Tests. WB Saunders. 1983. pages 5051, and 210 

39.01.10 
Deriving
Missing Performance Measures When Only Some Are Known 

Overview: If some performance measures for a test are known but others are not, it is often possible to calculate the missing values from those that are known. 

Key for equations 


(1)sensitivity = 
(( 1 + (((NPV^(1))  1) * (((SP^(1))  1)^(1)) * ((PPV^(1))  1)))^(1))  
(2) specificity = 
(( 1 + (((PPV^(1))  1) * (((SE^(1))  1)^(1)) * ((NPV^(1))  1)))^(1))  
(3) positive predictive value = 
(( 1 + (((SP^(1))  1) * (((NPV^(1))  1)^(1)) * ((SE^(1))  1)))^(1))  
(4) negative predictive value = 
(( 1 + (((SE^(1))  1) * (((PPV^(1))  1)^(1)) * ((SP^(1))  1)))^(1))  
(5) accuracy = 
((1 + (((((PPV ^ (1))  1) ^ (1) ) + (((SP ^ (1))  1) ^ (1)))^(1)) + (((((SE ^ (1))  1) ^ (1) ) + (((NPV ^ (1) )  1) ^ (1)))^(1))) ^ (1))  
(6) positive predictive value = 
((1 + (((SE ^ (1))  (ACC ^ (1))) * (((((ACC ^ (1))  1) * (((SP ^ (1))  1) ^ (1)))  1) ^(1)))) ^ (1))  
where:  


(7) sensitivity = 
((1 + (((PPV ^ (1))  (ACC ^ (1))) * (((((ACC ^ (1))  1) * (((NPV ^ (1))  1) ^ (1)))  1) ^(1)))) ^ (1))  
where: 



(8) specificity = 
(( 1 + ((((SE ^ (1)) + (PPV ^ (1))  (ACC ^ (1))  1) ^ (1)) * ((ACC ^ (1))  1) * ((PPV ^ (1)) 1))) ^ (1))  
(9) positive predictive value = 
(( 1 + ((((SP ^ (1)) + (NPV ^ (1))  (ACC ^ (1))  1) ^ (1)) * ((ACC ^ (1))  1) * ((SP ^ (1)) 1))) ^ (1))  
(10) specificity = 
((((((ACC ^ (1))  1) * (((SE ^ (1))  1) ^ (1)) * ((NPV ^ (1))  1)) + (ACC ^ (1))  (NPV ^ (1)) + 1)) ^ (1))  
(11) sensitivity = 
((((((ACC ^ (1))  1) * (((SP ^ (1))  1) ^ (1)) * ((PPV ^ (1))  1)) + (ACC ^ (1))  (PPV ^ (1)) + 1)) ^ (1))  


Implementation Notes  


TOP  References:  
Einstein AJ, Bodian CA, Gil J. The relationship among performance measures in the selection of diagnostic tests. Arch Pathol Lab Med. 1997; 121: 110117.  
39.01.11 
Youden's Index 

Overview: Youden's index is one way to attempt summarizing test accuracy into a single numeric value. 

Youden's index = 
1  ((false positive rate) + (false negative rate))  
= 
1  ((1  (sensitivity)) + (1  (specificity)))  
= 
(sensitivity) + (specificity)  1  
It may also be expressed as:  
Youden's index = 
( a / (a + b)) + (d / (c + d))  1 =  
((a * d)  (b * c)) / ((a + b) * (c + d))  
where: 
• a + b = people with disease • c + d = people without disease • a = people with disease identified by test (true positive) • b = people with disease not identified by test (false negatives) • c = people without disease identified by test (false positives) • d = people without disease not identified by test (true negatives) 

Interpretation 
• minimum index: 1 • maximum index: +1 • A perfect test would have a Youden index of +1. 

Limitation 
• The index by itself would not identify problems in sensitivity or specificity.  
TOP  References:  
Hausen H. Caries prediction  state of the art. Community Dentistry and Oral Epidemiology. 1997; 25: 8796. Hilden J, Glasziou P. Regret graphs, diagnostic uncertainty and Youden's index. Statistics in Medicine. 1996; 15: 969986. Youden WJ. Index for rating diagnostic tests. Cancer. 1950; 3: 3235. 

39.02  Bayes's Theorem
and Modifications 

39.02.01 
Bayes's
Theorem 

Overview: Bayes's theorem gives the probability of disease in a patient being tested based on disease prevalence and test performance. 

posttest probability disease present given a positive test result =  
= ((pretest probability that disease present) * (probability test positive if disease present)) / (((pretest probability that disease present) * (probability test positive if disease present)) + ((pretest probability that disease absent) * (probability test positive if disease absent))) 

posttest probability disease present given a negative test result =  
= ((pretest probability that disease present) * (probability test negative if disease present)) / (((pretest probability that disease absent) * (probability disease absent when test negative)) + ((pretest probability that disease present) * (probability test negative if disease present)))  
Variable 
Alternative
Statement 

pretest probability that disease present  prevalence  
probability test positive if disease present  sensitivity  
pretest probability that disease absent  (1  (prevalence))  
probability test positive if disease absent 
false positive rate = (1  (specificity)) 

probability test negative if disease present  false negative rate = (1  (sensitivity))  
probability disease absent when test negative  specificity  
Bayes's formula can also be expressed in the positive and negative predictive values:  
posttest probability given a positive result = = positive predictive value = = (true positives) / (all positives) = = (true positives) / ((true positives) + (false positives)) 

posttest probability given a negative result = = negative predictive value = = (false negatives) / (all negatives) = = (false negatives) / ((true negatives) + (false negatives)) 

Limitations of Bayes's theorem • Bayes's theorem assumes test independence, which may not occur if multiple tests are used for diagnosis 

TOP  References  


39.02.02  Odds and
Likelihood Ratios 

Overview: One form of Bayes's theorem is to calculate the posttest odds for a disorder from the pretest odds and performance characteristics for the test. 

odds ratio = 
(probability of disease) / (1  (probability of disease))  
likelihood ratio = 
(probability of a test result in a person with the disease) / (probability of a test result in a person without the disease)  
posttest odds = 
(pretest odds) * (likelihood ratio)  
where: • disease prevalence in the population can be used as the pretest odds • likelihood ratios can be expressed in terms of the sensitivity and specificity of the test for the diagnosis • positive likelihood ratio is the likelihood ratio for a positive test result; it is the truepositive rate divided by the falsepositive rate, or (sensitivity) / (1  (specificity)) • negative likelihood ratio is the likelihood ratio for a negative test result; it is the false negative rate divided by the true negative rate, or (1  (sensitivity)) / (specificity) 

posttest odds that the person has the disease if there is a positive test result = 
(pretest odds) * (positive likelihood ratio)  
posttest odds that the person has the disease if there is a negative test result = 
(pretest odds) * (negative likelihood ratio)  
TOP  Calculating PostTest Odds  
Step 1: Calculate the positive and negative likelihood ratios for the test • positive likelihood ratio = = (sensitivity) / (1  (specificity)) • negative likelihood ratio = = (1  (sensitivity)) / (specificity) 

Step 2: Convert the prior probability to prior odds: ((prior probability) * 10) : ((1  (prior probability)) * 10) 

Step 3: Multiply the prior odds by the likelihood ratios to obtain the posttest odds • ((positive likelihood ratio) * (prior probability) * 10) : ((1  (prior probability)) * 10) • ((negative likelihood ratio) * (prior probability) * 10) : ((1  (prior probability)) * 10) 

Step 4: Convert the posttest odds to posttest probabilities • positive posttest probability = = ((positive likelihood ratio) * (prior probability) * 10) / (((positive likelihood ratio) * (prior probability) * 10) + ((1  (prior probability)) * 10)) • negative posttest probability = = ((negative likelihood ratio) * (prior probability) * 10) / (((negative likelihood ratio) * (prior probability) * 10) + ((1  (prior probability)) * 10)) 

TOP  References:  
Einstein AJ, Bodian CA, Gil J. The relationship among performance measures in the selection of diagnostic tests. Arch Pathol Lab Med. 1997; 121: 110117. Noe DA. Chapter 3: Diagnostic Classification. pages 2743. IN: Noe DA, Rock RC (Editors). Laboratory Medicine. Williams and Wilkins. 1994. Scott TE. Chapter 2: Decision making in pediatric trauma. pages 2040. IN: Ford EG, Andrassy RJ. Pediatric Trauma  Initial Assessment and Management. W.B. Saunders. 1994 Suchman AL, Dolan JG. Odds and likelihood ratios. pages 2934. IN: Panzer RJ, Black ER, et al. Diagnostic Strategies for Common Medical Problems. American College of Physicians. 1991. Weissler AM. Chapter 11: Assessment and use of cardovascular tests in clinical prediction. pages 400421. IN: Giuliani ER, Gersh BJ, et al. Mayo Clinic Practice of Cardiology, Third Edition. Mosby. 1996 

39.02.03  Odds and
Likelihood Ratios for Sequential Testing 

Overview: If more than one test or finding is used for diagnosis, the final posttest probability can be calculated by combining the likelihood ratio for each test. 

posttest odds = 
(pretest odds) * (likelihood ratio for test 1) * (likelihood ratio for test 2) * .... * (likelihood ratio for test n)  
Limitation • For valid results, tests must be conditionally independent of each other, where conditionally independent indicates that the results of the tests are not associated with each other. • If conditionally dependent tests are used, then the calculated posttest probability will be overestimated. 

TOP  References:  


39.03  Risk Sensitivity
and Risk Specificity 





risk sensitivity in percent = 
(mortality for high risk subgroup in percent) * (percent of population identified as high risk) / (cumulative mortality in percent for the whole population)  
risk specificity in percent = 
(survival for low risk subgroup in percent) * (percent of population identified as low risk) / (cumulative survival in percent for the whole population)  
percent of population in high risk group = 
100  (percent of population in low risk group)  
cumulative survival of high risk group = 
100  (cumulative mortality of high risk group)  
cumulative survival of low risk group = 
100  (cumulative mortality of low risk group)  
cumulative survival of population = 
100  (cumulative mortality of population)  
TOP  References:  
Weissler AM. Chapter 11: Assessment and use of cardovascular tests in clinical prediction. pages 400421. IN: Giuliani ER, Gersh BJ, et al. Mayo Clinic Practice of Cardiology, Third Edition. Mosby. 1996  
39.04  Statistics for
the Normal Distribution and Use in Quality Control 

39.04.01 
Mean of Values
in a Normal Distribution 



mean of values = 
(sum of all values) / (number of values)  
TOP  References:  
Woo J, Henry JB. Chapter 6: Quality management. pages 125136 (128). IN: Henry JB (editorinchief). Clinical Diagnosis and Management by Laboratory Methods, 19th edition. WB Saunders.1996.  
39.04.02  Standard
Deviation (SD) 



standard deviation = 
absolute value [square root of the variance]  


TOP  References:  


39.04.03 
Coefficient
of Variation (CV) 



CV (expressed as a percent) = 
((standard deviation) * 100 / (mean))  
TOP  References:  
Dharan, Murali. Total Quality Control in the Clinical Laboratory. C.V. Mosby Co. 1977. page 22  
39.04.04 
Standard
Deviation Interval (SDI) 



SDI = 
(((mean)  (average of all means)) / (standard deviation of all means))  




TOP  References:  
College of American Pathologists QAS Program  
39.04.05 
Coefficient
of Variation Interval (CVI) 



CVI = 
(CV for laboratory) / (pool CV)  
or 
(CV for laboratory for time period) / (peer group CV for time period)  


TOP  References:  
The Interlaboratory Quality Assurance Program. Coulter Diagnostics. 1988.  
39.05 
Total
Allowable Error 



Variables 

laboratory mean = 
mean at laboratory for period of stability in reagents & controls  
"true" mean = 
mean for all methods & laboratories  
laboratory standard deviation = 
standard deviation noted at laboratory  
method standard deviation = 
standard deviation reported by vendor  


Calculations 

calculated bias = 
laboratory's deviation (based on site and method) from mean of all sites =  
((laboratory mean)  (true mean))  
laboratory imprecision = 
(factor) * (laboratory standard deviation)  


Total allowable error = TEa =  ((CLIA limit) * (true mean))  
bias as percent of CLIA limit = 
((calculated bias) / ((CLIA limit) * (true mean))) =  
(((laboratory mean)  (true mean)) / (total allowable error)) =  
( ((laboratory mean)  (true mean)) / ((CLIA limit) * (true mean)))  
total error = 
calculated bias + imprecision =  
(((laboratory mean)  (true mean)) + (laboratory imprecision))  
( ((laboratory mean)  (true mean)) + ((factor) * (laboratory standard deviation)))  
assessment of performance = 
(total error) / (total allowable error) * 100 =  
(((laboratory mean)  (true mean))+ (laboratory imprecision)) / (((CLIA limit) * (true mean)))* 100 =  
(((laboratory mean)  (true mean))+ (1.96 * (laboratory standard deviation))) / (((CLIA limit) * (true mean))) * 100  
systemic error (critical) = SEc = 
( ( ( (total allowable error)  (calculated bias) ) / (laboratory standard deviation) )  1.65) =  
( ( ( ((CLIA limit) * (true mean))  ((laboratory mean)  (true mean))) / (laboratory standard deviation))  1.65)  
Use the systemic error for selection of QC control rules to use  
• standard deviation to use =  ((calculated standard deviation) * ((denominator of primary rule) / 2))  
Example: If using 1:3s rule, where the denominator = 3 

standard deviation to use = 
(laboratory standard deviation)
* (3 / 2) = 1.5 * (laboratory standard deviation) 

TOP  References  


39.06 
Chi Square 

39.06.01 
Outcome
Comparison of Two Groups 





This data shows 1 degree of freedom.  
chi square value using Yates correction for 1 degree of freedom = 
((total number) * ((ABS(((number of group A improved) * (number group B not improved))  ((number of group A not improved) * (number of group B improved)))  ((total number) / 2))^2)) / (((number of group A improved) + (number of group B improved)) * ((number of group A not improved) + (number of group B not improved)) * (total number of group A) * (total number of group B))  
From the chisquare value, it is the probability that a difference is due to chance can be calculated. The Excel function CHIDIST will give the probability of the difference being due to chance for the chisquare value. The probability that the difference is not due to chance is then (1  (probability due to chance)).  
TOP  References:  


39.06.02 
Comparison
of Two Observers 



For more than 2 observations:  
chi square = 
(summation from 1 to number of observations ( (((observer A value)  (observer B value)) ^ 2) / ((observer A value) + (observer B value)) ) )  
From this value, the probability that the differences between the 2 observers is due to chance can be calculated. The equation can be simplified from an integral depending on whether there is an even or odd degree of freedom.  
Even Degrees of Freedom  
For even degrees of freedom, this is relatively simple.  
probability due to chance (chisquare, degree of freedom) =  ((e) ^ ((1) * (chisquare) / 2)) * (summation of i from 0 to I ( (((chisquare) / 2) ^ (i)) / (factorial (i)))  


Odd Degrees of Freedom 

For odd degrees of freedom, this is quite complex, and it is easier to use the Excel function CHIDIST.  
probability due to chance (chisquare, degree of freedom) = 
1  ( (1 / (gamma function (I + 1))) * (summation from 0 to infinity (((1)^i) * (((chisquare) / 2) ^ (I + i + 1)) / ((factorial (i)) * (I + i + 1)))  


TOP  References:  


39.07 
Test Comparison Using Receiver Operating Characteristics (ROC) Plots 



Receiver Operating Curve 

To generate a receiver operating curve it is first necessary to determine the sensitivity and specificity for each test result in the diagnosis of the disorder in question.  




When the xaxis is the false positive rate (1  (specificity)), the curve starts at (0,0) and increases towards (1,1). When the xaxis is the true negative rate (specificity), the curve starts at (0, 1) and drops towards (1, 0). The endpoints for the curve will run to these points.  
Area under Curve 

One way of measuring the area under a curve is by measuring subcomponent trapezoids. Data points can be connected by straight lines defined by:  
y = ((slope) * x) + intercept 

The area under each line can be determined by integration of (y * dx) over the interval of x1 to x2:  
area = (((slope) / 2) * ((x2 ^ 2)  (x1 ^2))) + ((intercept) * (x2  x1))  
By summating the areas under each segment, an approximation of the area under the entire curve can be reached. However, the trapezoidal method tends to underestimate areas (Hanley, 1983), so that other techniques for measuring area should be used if greater accuracy is required.  
The maximum area under ROC curve is 1 and is seen with the ideal test. The closer the area under the ROC curve is to 1, the better (more accurate) the test.  
Comparison of Two Methods  
Two methods can be compared by the area under their respective ROC curves. The method with the larger area under the ROC curve is preferable over one with a smaller area, allowing for variability, as being more accurate.  
TOP  References:  


39.08 
Zscore 



Zscore = 
((patient value)  (mean for reference population)) / (standard deviation for reference population)  
This appears to be share features with the Standard Deviation Interval (SDI).  
TOP  References  
Withold W, Schulte U, Reinauer H. Methods for determination of bone alkaline phosphatase activity: analytical performance and clinical usefulness in patients with metabolic and malignant bone diseases. Clin Chem. 1996; 42: 210217.  
39.09 
Westgard Rules and the Multirule Shewhart Procedure 

39.09.01 
Westgard Control Rules 



Procedure
1. Starting with a stable testing system and stable control material, a control material is analyzed for at least 20 different days. This data is used to calculate a mean and standard deviation for the control material. 2. Usually 2 control materials are analyzed (one with a low value, one with a higher value in the analytical range). Sometimes 3 or more control materials may be used, and rarely only 1. 3. The controls are included with each analytical run of the test system. 4. A LeveyJennings control chart is prepared to graphically represent the data for each control relative to the mean and multiples of the standard deviation. 5. With each analytical run, the pattern of the current and previous control results are analyzed using all of the selected Westgard control rules. 6. If none of the rules fail, then the run is accepted. If one or more rules fail, then different responses may occur. This may include rejecting the run, adjusting the stated mean, and/or recalibrating the test. 





TOP  References:  
Lott JA. Chapter 18: Process control and method evaluation. pages 293325 (Figure 184, page 302). IN: Snyder JR, Wilkinson DS. Management in Laboratory Medicine, Third Edition. Lippincott. 1998. Westgard JO. Chapter 150: Planning statistical quality control procedures. pages 11911200. IN: Rose NR, de Macario EC, et al (editors). Manual of Clinical Laboratory Immunology, Fifth Edition. ASM Press. 1997. Westgard JO, Klee GG. Chapter 17: Quality management. pages 384418. IN: Burtis CA, Ashwood ER. Tietz Textbook of Clinical Chemistry, Third Edition. WB Saunders Company. 1999 (1998).


39.09.02 
Using a Series of Control Rules in the Multirule Shewhart Procedure 





TOP  References:  
Lott JA. Chapter 18: Process control and method evaluation. pages 293325 (Figure 184, page 302). IN: Snyder JR, Wilkinson DS. Management in Laboratory Medicine, Third Edition. Lippincott. 1998. Westgard JO, Barry PL, Hunt MR. A multirule Shewhart chart for quality control in clinical chemistry. Clin Chem. 1981; 27: 493501. 

39.10 
Evaluating
Reports in the Medical Literature 

39.10.01 
Criteria for Assessing the Methodologic Quality of Clinical Studies 







Interpretation • minimum score: 0 • maximum score: 14 • The higher the score, the higher the quality in the study design and implementation. 

TOP  References:  
Heyland DK, Cook D, et al. Maximizing oxygen delivery in critically ill patients: a methodologic appraisal of the evidence. Crit Care Med. 1996; 24: 517524. Heyland DK, MacDonald S, et al. Total parenteral nutrition in the critically ill patient. JAMA. 1998; 280: 20132019. 

39.11 
Measures of
the Consequences of Treatment 

39.11.01 
Number
Needed to Treat 



Variables: • number of people in control group • number of people in control group who develop condition of interest during time interval • number of people in active treatment group • number of people in active treatment group who develop condition of interest during time interval 

event rate in control group = 
(number of people in control group with condition) / (number of people in control group)  
event rate in active treatment group = 
(number of people in active treatment group with condition) / (number of people in active treatment group)  
relative risk reduction = 
((event rate in control group)  (event rate in active treatment group)) / (event rate in control group)  
absolute risk reduction = 
(event rate in control group)  (event rate in active treatment group)  
number needed to treat = 
1 / (absolute risk reduction) =  
1 / ((event rate in control group)  (event rate in active treatment group))  
Interpretation • The number needed to treat indicates the number of patients who need to be treated to prevent the condition of interest during the time interval. • The smaller the number needed to treat, the greater the benefit of the treatment to prevent the condition. • The number needed to treat should be considered together with other factors such as the seriousness of the condition to be prevented and the risk of adverse side effects from the treatment. 

TOP  References:  
Altman DG. Confidence intervals for the number needed to treat. BMJ. 1998; 317: 13091312. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995; 310: 452454. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988; 318: 17281733. 

39.12 
The Corrected
Risk Ratio and Estimating Relative Risk 



Odds ratio and relative risk (see Figure on page 1690 of Zhang and Yu) • If the incidence of an outcome in the study population is < 10%, then the odds ratio is close to the risk ratio. • As the incidence of the outcome increases, the odds ratio overestimates the relative risk if it is more than 1, or underestimates the relative risk is less than 1. 

Situations when desirable to perform correction • if the incidence of the outcome in the nonexposed population is more than 10%, AND • if the odds ratio is > 2.5 or < 0.5 

incidence of outcome in nonexposed group = N = 
(number with outcome in nonexposed group) / (number in nonexposed group)  
incidence of outcome in exposed group = E = 
(number with outcome in exposed group) / (number in exposed group)  
risk ratio = 
E / N  
odds ratio = 
(E / (1  E)) / (N / (1  N))  
E / N = 
(odds ratio) / [(1  N) + (N * (odds ratio))]  
corrected risk ratio = 
(odds ratio) / [(1  N) + (N * (odds ratio))]  
This equation can be used to correct the adjusted odds ratio obtained from logistic regression.  
TOP  References  
Wacholder S. Binomail regression in GLIM: Estimating risk ratios and risk differences. Am J Epidemiol. 1986; 123: 174184. Zhang J, Yu KF. What's the relative risk? A method for correcting the odds ratio in cohort studies of common outcomes. JAMA. 1998; 280: 16901691. 

39.13 
Confidence
Intervals 

39.13.01 
Confidence
Interval for a Single Mean 



Data assumptions: single mean, symmetrical distribution  
confidence interval = 
(mean) +/ ((onetailed value of Student's t distribution) * (standard deviation) / ((number of values) ^ (0.5))  
where: • for a 95% confidence interval, the onetailed value is for 2.5% (F 0.975, t 0.025) • degrees of freedom = (number of values)  1 • as the number of values increases, the closer the onetailed value for t=0.025 approaches 1.96; at 120 degrees of freedom it is 1.98 

TOP  References:  
Beyer WH. CRC Standard Mathematical Tables, 25th Edition. CRC Press. 1978. Section: Probability and Statistics. Percentage points, Student's tdistribution. page 536. Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318. 

39.13.02 
Confidence Interval for the Difference Between Two Means 



Data assumptions: 2 sets of data with symmetrical distribution  
confidence interval for the difference in the means between 2 sets of data = 
ABS((mean first group)  (mean second group)) +/ (factor)  
factor = 
(onesided value of Student's tdistribution) * (pooled standard deviation) * (((1 / (number in first set)) + (1 / (number in second set))) ^ (0.5))  
degrees of freedom = 
(number in first set) + (number in second set)  2  
pooled standard deviation = 
((A + B) / (degrees of freedom)) ^ (0.5)  
A = ((number in first set)  1) * ((standard deviation of first set) ^ 2)  
B = ((number in second set)  1) * ((standard deviation of second set) ^ 2)  
where: • for a 95% confidence interval, the onetailed value is for 2.5% (F 0.975, t 0.025) • as the number of values increases, the closer the onetailed value for t=0.025 approaches 1.96; at 120 degrees of freedom it is 1.98 

TOP  References:  
Beyer WH. CRC
Standard Mathematical Tables, 25th Edition. CRC Press. 1978. Section:
Probability and Statistics. Percentage points, Student's tdistribution. page
536.
Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318. 

39.13.03 
Confidence
Interval for a Single Proportion 



Variables • N observations • X events of interest 

Distribution used • F distribution, with F = 0.975 for the 95% confidence interval • uses m and n as degrees of freedom 

proportion of events = 
X / N  
lower limit for the 95% confidence interval = 
X / (X + ((N  X + 1) * (F distribution for m and n)))  
where • m = 2 * (N  X + 1) • n = 2 * X 

upper limit for the 95% confidence interval = 
((X + 1) * (F distribution for m and n)) / (N  X + ((X + 1) * (F distribution for m and n)))  
where • m = 2 * (X + 1) = n + 2 • n = 2 * (N  X) = m  2 

TOP  References  
Beyer WH. CRC Standard Mathematical Tables, 25th Edition. CRC Press. 1978. Section: Probability and Statistics. Fdistribution. page 540. Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318. 

39.13.04 
Confidence Interval When the Proportion in N Observations is 0 or 1 



X = 
1  ((confidence interval in percent) / 100)  
If 0 events occur in n observations • lower limit for the confidence interval: 0 • upper limit for the confidence interval: 1  ((X/2) ^ (1/n)) 

If n events occur in n observations • lower limit for the confidence interval: ((X/2)) ^ (1/n)) • upper limit for the confidence interval: 1 (100%) 

TOP  References  
Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318.  
39.13.05 
Confidence Interval for the Difference Between Two Proportions Based on the Odds
Ratio 





(Table page 316, Young 1997)  
odds for the event in group 1 = 
C / A  
odds for the event in group 2 
D / B  
odds ratio for group 2 relative to group 1 = 
(odds group 2) / (odds group 1) =  
(A * D) / (B * C)  
confidence interval for 95% = 
EXP( X +/ Y)  
X = 
LN ((A * D) / (B * C))  
Y = 
1.96 * SQRT((1/A) + (1/B) + (1/C) + (1/D))  
where: • 1.96 is the value for Z from the standard normal distribution with F(Z) = 0.975 

If the odds ratio is 1.0, then there is no difference between the two groups. If the 2 groups are comparing an intervention, then this is equivalent to a null hypothesis of no intervention difference.  
Small Sample Sizes 

If sample sizes are small (less than 10 or 20), then 0.5 is added to each of the factors.  
odds ratio = 
(odds group 2) / (odds group 1) =  
((A+0.5) * (D+0.5)) / ((B+0.5) * (C+0.5))  
confidence interval for 95% = 
EXP( X +/ Y)  
X = 
LN (((A+0.5) * (D+0.5)) / ((B+0.5) * (C+0.5)))  
Y = 
1.96 * SQRT((1/ (A+0.5)) + (1/ (B+0.5)) + (1/ (C+0.5)) + (1/ (D+0.5)))  
NOTE: I am using sample size as (A + B + C + D).  
TOP  References  
Beyer WH. CRC Standard Mathematical Tables, 25th Edition. CRC Press. 1978. Section: Probability and Statistics. Fdistribution. page 524. Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318. 

39.13.06 
Confidence Interval for the Difference Between Two Proportions Using the Normal
Approximation 



Requirements (1) events occur with a normal distributions (2) populations and events are sufficiently large (3) the proportions for the 2 populations are not too close to 0 or 1 



proportion responding in population 1 = P1 = 
(R1) / (N1)  
proportion responding in population 2 = P2 = 
(R2) / (N2)  
confidence interval = 
P1  P2 +/ ((one tailed value of the standard normal distribution) * (SQRT (((P1 * (1  P1)) / N1) + ((P2 * (1  P2)) / N2)))  






TOP  References  
Beyer WH. CRC Standard Mathematical Tables, 25th Edition. CRC Press. 1978. page 524. Young KD. Lewis RJ. What is confidence? Part 2: Detailed definition and determination of confidence intervals. Ann Emerg Med. 1997; 30: 311318. 

39.14 
Odds and
Percentages 



total population = 
(number affected) + (number unaffected)  
odds denominator = 
(total population) / (number affected) =  
1 + ((number unaffected) / (number affected))  
odds = 
1 in (odds denominator)  
(number affected) to (number unaffected)  
percent affected = 
(number affected) / (total population) * 100% =  
1 / (odds denominator)  
TOP  References  
Harper PS. Practical Genetic Counselling, Fifth Edition. Butterworth Heinemann. 1999. Table 1.1, page 10.  
39.15 
Benefit, Risk
and Threshold for an Action 

39.15.01 
BenefittoRisk Ratio and Treatment Threshold for Using a Treatment Strategy 



benefit for treatment = 
(risk of adverse outcome from the disease in those untreated)  (risk of adverse outcome from the disease with treatment)  
risk of treatment = 
(risk of significant adverse complication due to treatment)  
benefittorisk ratio = 
(benefit for treatment) / (risk for treatment)  
treatment threshold = 
1 / ((benefittorisk ratio) +1 ) =  
(risk) / ((benefit) + (risk))  
Interpretation • Treatment should be given when the risk of having the condition exceeds the treatment threshold. • Treatment should be withheld if the risk of having the condition is less than the treatment threshold. 

TOP  References  
Beers MH, Berkow R, et al (editors). The Merck Manual of Diagnosis and Therapy, Seventeenth Edition. Merck Research Laboratories. 1999. Chapter 295. Clinical Decision Making. page 2523.  
39.15.02 
Testing and Test Treatment Thresholds 



Test features • performance characteristics (sensitivity and specificity) for the condition are known • assume that the test has no direct adverse risk to the patient 

benefit for treatment = 
(risk of adverse outcome from the disease in those untreated)  (risk of adverse outcome from the disease with treatment)  
risk of treatment = 
(risk of significant adverse complication due to treatment)  
testing threshold = 
((1  (specificity of test)) * (risk of treatment)) / (((1  (specificity of test)) * (risk of treatment)) + ((sensitivity of test) * (benefit of test)))  
test treatment threshold = 
((specificity of test) * (risk of treatment)) / (((specificity of test) * (risk of treatment)) + ((1  (sensitivity of test)) * (benefit of test)))  
Interpretation • If the probability of disease is equal or more than the testing threshold and equal or less than the test treatment threshold, then the test should be done. • If the probability 

