Sensitivity and Specificity

---

Practical examples using sensitivity, specificity, gold (reference) standard, positive predictive value, and negative predictive value.

---

An ELISA is developed to diagnose HIV infections. Serum from 10,000 patients that were positive by Western Blot (the gold standard assay) were tested and 9990 were found to be positive by the new ELISA. The manufacturers then used the ELISA to test serum from 10,000 nuns who denied risk factors for HIV infection. 9990 were negative and the 10 positive results were negative by Western Blot.

		HIV Infected
		+	-
ELISA	+	9990 (TP)	10 (FP)
	-	10 (FN)	9990 (TN)
		10,000 TP+FN	10,000 FP+TN
		Sensitivity = TP/(TP+FN) 9990/(9990+10) =.999 or 99.9%	Specificity= TN/(FP+TN) 9990/(9990+10) =.999 or 99.9%

With a sensitivity of 99.9% and a specificity of 99.9%, the ELISA appears to be an excellent test. Let's apply this test (assuming the sensitivity and specificity remain the same) to a million people where 1% are infected with HIV. Of the million people, 10,000 would be infected with HIV. Since our ELISA is 99.9% sensitive, the test will detect 9,990 ( true positives -- TP) people who are actually infected and miss 10 (false negative -- FN). Looking at those numbers, we would think that our test is very good because we have detected 9990 out of 10,000 HIV infected people. But there is another side to the test. Of our original one million, 990,000 are not infected. If we look at the test results on the HIV negative population (remember the specificity of the assay is 99.9%), we find that 989,010 are found to be not infected by the ELISA (true negatives -- TN), but we have 990 individuals who are found to be positive by the ELISA (false positives -- FN). If you released these test results without confirmatory tests (our gold standard Western Blot), you would have told 990 people or approximately .1% of the population that they are HIV infected when in reality, they are not.

		Disease
		+	-
Test	+	9990  True Positive (TP)	990  False Positive (FP)	All with Positive Test TP+FP	Positive Predictive Value= TP/(TP+FP) 9990/(9990+990) =91%
	-	10  False Negative (FN)	989,010  True Negative (TN)	All with Negative Test FN+TN	Negative Predictive Value= TN/(FN+TN) 989,010/(10+989,010) =99.999%
		All with Disease 10,000	All without Disease 999,000	Everyone= TP+FP+FN+TN
		Sensitivity= TP/(TP+FN) 9990/(9990+10)	Specificity= TN/(FP+TN) 989,010/ (989,010+990)	Pre-Test Probability= (TP+FN)/(TP+FP+FN+TN) (in this case = prevalence) 10,000/1,000,000 = 1%

Unfortunately, understanding sensitivity and specificity is not the be-all, end-all because they do not address the problems of the prevalence of disease in different populations. For that, you need to understand positive and negative predictive values. Let's take our previous example with the HIV ELISA and use that test in two different populations.

(1) The real blood donor pool, which has already been screened for HIV risk factors before they are even allowed to donate blood, so that the HIV sero-prevalence in this population is closer to .1% instead of 1%. How does this change the numbers -- for every 1,000,000 blood donors, 1,000 are HIV positive. Again using a sensitivity of 99.9%, the ELISA would pick up 999 of those thousand, leaving one individual who was HIV sero-positive to slip through the cracks. Of the 999,000 individuals who were not infected, the test would call 998,001 individual sero-negative (true negatives). The ELISA would, however, falsely label 999 individuals as sero-positive (false-positives). Testing the blood donor pool results in as many false positive as true positive results.

		Disease
		+	-
Test	+	999 (TP)	999 (FP)	All with Positive Test TP+FP 1998	Positive Predictive Value= TP/(TP+FP) =50%
	-	1 (FN)	998,001 (TN)	All with Negative Test FN+TN	Negative Predictive Value= TN/(FN+TN) =99.999%
		All with Disease 1000	All without Disease 999,000	Everyone TP+FP+FN+TN
		Sensitivity 99.9%	Specificity 99.9%	Pre-Test Probability 0.1%

(2) The other example we want to look at is a drug rehabilitation units for I.V. drug users. Here, the prevalence of HIV is say, 10%. So if we had a million people in these clinics, 100,000 of these individuals would be HIV-infected and 900,000 would be HIV negative. Using our HIV ELISA, we find 99,900 true positives and 100 false negatives. Of the 900,000 HIV negative individuals, the ELISA will find 899,100 to be negative (TN) and a falsely label 900. as positive (FP).

		Disease
		+	-
Test	+	99,900 (TP)	900 (FP)	All with Positive Test 100,800	Positive Predictive Value= TP/(TP+FP) 99,900/100,800 =99%
	-	100 (FN)	899,100 (TN)	All with Negative Test 899,200	Negative Predictive Value= TN/(FN+TN) 899,100/899,200 =99.99%
		All with Disease 100,000	All without Disease 900,000	Everyone TP+FP+FN+TN
		Sensitivity 99.9%	Specificity 99.9%	Pre-Test Probability 10%

The sensitivity and specificity of the test has not changed. It is just that the predictive value of the test has changed depending on the population being tested. The positive predictive value is how many of the test-positives truly have the disease. In the first example with a 1% sero-positive rate, the ELISA has a positive predictive value of 0.91 (91%). When looking at the blood donor pool with a 0.1% sero-prevalence, the positive predictive value is only 0.5 (50%), whereas in the high- prevalence population of intravenous drug users, the positive predictive value is 0.99 (99%). Although the sensitivity of the ELISA does not change between populations, the positive predictive value changes drastically from only half the people that tested positive being truly positive in a low- incidence population to 99% of the people testing positive being truly positive in the high- prevalence population. The negative predictive value of the ELISA also changes depending on the prevalence of the disease.

Everything we discussed so far has assumed that the sensitivity and specificity do not change, as one deals with different groups of people. Sensitivity and specificity, however, CAN CHANGE if the population tested is dramatically different from the population you serve:

Another problem we have when using diagnostic tests is that they are originally tested in populations that are homogeneously positive or negative and/or designed to be used in patients with a high pretest probability of disease. When the tests are used in clinical practice, they sometime fail to be helpful. For example, the Anti-Nuclear Antibody test is useful in confirming the diagnosis of SLE (Systemic Lupus Erythematosus) in patients with multiple clinical manifestations of the disease. The test, however, is often applied indiscriminately in patients where the physician is using a shotgun approach to diagnosis. Here, the test is diagnostically almost useless as the number of false positive overwhelm the number of true positive results but it does keep the consulting Rheumatologist busy saying "no, the patient does not have SLE."

The sensitivity and specificity of a test can also change in patients with early manifestations of a disease -- just the patients that you need the test for.