Problem Set 4.2 Spring 2009
Cost–Benefit Analysis of an Auditory Screening Protocol

A.  Objectives

To apply probability theory to the analysis of imperfect test results. Students will use probability theory and simple Cost–benefit analyses to develop a defensible testing strategy for the screening of children at high risk for hearing loss.  Several problems in this problem set involve the simulation of actual test results using the Excel file IMPTEST.xls.  Students will need to download the IMPTEST.xls file to complete this problem s Set.  Student will not need to upload any results.  Simply use the Excel file as a tool to simulate populations and test results based on those populations.

B.  Problems

1. The table below represents a hypothetical administration of an Otoacoustic Emission (OAE) screening test to a high-risk group of 6–18-month-old toddlers. The results are deterministic because all probabilities are reproduced exactly in the table, unlike empirical results that will vary due to random processes at work in the testing.

   1. Knowing that all the numbers "work correctly," fill in the missing values.

      Disease Prevalence	Test Result	Impaired	Normal	Totals
      5%	Positive	45
      of population	Negative		760
      is impaired.	Totals			1000

   2. Based on the completed table, calculate the sensitivity and specificity of this test.
      
      
      
      
      
      
      
      
   3. Now run a probabilistic simulation of the situation described in the table above using the Excel program IMPTEST. Fill in the table below with the simulation results.
      

      Disease prevalence	Test result	Impaired	Normal	Total
      5%	Positive
      of population	Negative
      is impaired.	Total			1000

   4. Why are the tables different? What would you expect to happen if you ran the simulation program 50 times and averaged the results?

       

       

   5. Now run a probabilistic simulation (using IMPTEST) of the same test but on a low-risk population. Fill in the table below with the simulation results.

      Disease prevalence	Test result	Impaired	Normal	Total
      0.1%	Positive
      of population	Negative
      is impaired.	Totals			1000

       

       

2. An imperfect test for a specific disease has a sensitivity of 0.85 and a specificity of 0.76.  The test will be administered to a population that has a disease prevalence of 14%. 400 people will be tested.
1. Draw a tree diagram modeling this situation, labeling the branches and outcomes. Also draw a decision matrix representing the same information in a different form.
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   

2. Use your tree diagram to calculate the probability that a person who has the disease tests positive for the disease. This is the conditional probability P(+|D).]
   
   
   
   
   

3. Use your tree diagram to calculate the probability that a person who tests positive for the disease does in fact have the disease. This is the conditional probability P(D|+).
   
   
   
   

4. Run a probabilistic simulation (using IMPTEST.xls) of using this test on a population of 100. Put the results into the standard 2 by 2 decision matrix.

    

    

    

    

 

3. 
   
   1. Derive a formula that calculates the probability P that a positive test result is a true positive test result, given the following parameters: S = sensitivity, C = specificity, and R = prevalence rate. (You want a formula of the form: P = . . . .).  Hint: use a tree diagram labeled with variables and make a normal P(D|+) calculation.
      
      
      
      
      
      
      b. Using your formula, plot the relationship between P and R for values of R ranging from 0% to 100% (or 0 to 1 for decimal equivalents) for the four pairs of given S and C values. Describe the relationship between P and R. (R is the independent or "x" variable, and P is the dependent or "y" variable).

      Test	S: Sensitivity	C: Specificity
      I	0.70	0.55
      II	0.90	0.85
      III	0.90	0.98
      IV	0.70	0.85

      

       

      

4. Below is a tree diagram as a model of a two-stage testing protocol.

    

   




The first test costs $40 per patient to administer. The second test is only given to people who test positive on the first test and costs $100 per patient.

1. Complete the following chart:

Prevalence rate	Test 1 sensitivity	Test 1 specificity	Test 2 sensitivity	Test 2 specificity



2. Calculate P(~D| + on first and second tests):

 

 

3. Calculate P(D| + on first and second tests):

 

 

4. Calculate P(D| + on first test, and – on second test):

 

 

 

5. The protocol is used with 1000 patients. Calculate the expected total cost of the protocol.
   
   
   
   
   
6. Calculate the expected cost per detection of a patient with disorder "D".