45-733 PROBABILITY AND STATISTICS I Topic #1C

25 January 2000 and 27 January 2000

Bayes Theorem

1. Suppose we have a K-Fold partition (as in (15) of notes #1) of our sample space, S. Over the top of this partition we drop a set B so that B intersects the K partitioning sets (A₁, A₂, ... , A_k). That is:
   

   

   Clearly B can be written as:
   B = (A₁ Ç B) È (A₂ Ç B) È (A₃ Ç B) È etc.
   Now, using the identity: P(B|A_i) = P(B Ç A_i)/P(A_i), and rearranging:
   P(A_i Ç B) = P(A_i)P(B|A_i). So that:
   P(B) = P(A₁)P(B|A₁) + P(A₂)P(B|A₂) + P(A₃)P(B|A₃) + etc.
   
   This allows us to state the famous Bayes Theorem.
   
   Let A₁, A₂, ... , A_k be a partition of S such that P(A_i) > 0 for all i,
   and let B be an event such that P(B) > 0. Then:
   P(A_i|B) = [P(A_i)P(B|A_i)]/ [å_j=1,k P(A_j)P(B|A_j)] =
   P(A_i Ç B)/ [å_j=1,k P(A_j Ç B)] =
   P(A_i Ç B)/P(B)
   
   
2. Example: We have three chests, each with two drawers. (The chests are identical in every respect.) In one chest there is a silver coin in one drawer and a silver coin in the second drawer. In a second chest there is a gold coin in one drawer and a silver coin in the second drawer. And in the third chest there is a gold coin in each drawer.
   We randomly select a chest and open a drawer. What is the probability that the un-opened drawer contains a gold coin?
   Let A₁, A₂, A₃, represent the three chests, and let B be the event "gold coin in the drawer". Clearly:
   P(A₁) = P(A₂) = P(A₃) = 1/3; and:
   P(B|A₁) = 0, P(B|A₂) = 1/2, P(B|A₃) = 1.
   Only if we chose the 3rd chest could there be a second gold coin. Hence, we need to compute:
   P(A₃|B) = [P(A₃)P(B|A₃)]/ [å_j=1,3 P(A_j)P(B|A_j)] =
   = (1/3 x 1)/(1/3 x [0 + 1/2 + 1]) = 2/3.
   
   
3. Voting Example: In a particular city we have three kinds of voters: Republicans, Democrats, and Reformers, which we represent as the events R, D, and J, respectively. Let V be the event "voted in the last election". We are given that:
   P(R) = .4, P(D) = .35, and P(J) = .25; and:
   P(V|R) = .7, P(V|D) = .65, P(V|J) = .4.
   We randomly select a person and learn that she did not vote. What is the probability that she is a Democrat?
   By the rule of the complement:
   P(V^c|R) = .3, P(V^c|D) = .35, P(V^c|J) = .6.
   The probability we need to compute is:
   P(D|V^c) = [P(D)P(V^c|D)]/
   [P(D)P(V^c|D) + P(R)P(V^c|R)] + P(J)P(V^c|J)]
   = (.35 x .35)/[(.35 x .35) + (.4 x .3) + (.25 x .6)] = .312
   
   
4. Problem 2.83 p.63
   Let D be the event "Disease" and B be the event "Positive Test". We are given in the problem statement that:
   P(D) = .01 so that P(D^c) = .99;
   P(B|D) = .9 and P(B^c|D^c) = .9.
   We are asked to compute:
   P(D|B) = [P(D)P(B|D)]/[P(D)P(B|D) + P(D^c)P(B|D^c)] =
   = (.01 * .9)/[(.01 * .9) + (.99 * .1)] = .0833.
   
   Note that P(D|B) > P(D). That is, we start with a prior probability of .01 that any randomly chosen person has the disease. Then we are given information ("postive test"). This must raise the probability that the person has the disease. This probability is our posterior probability. In the jargon of Bayesian statistics, we have revised our priors in light of the information.
   
   Also note that the probability 1 - P(D|B) is the probability of a false positive. This is not a costly error in this context. Far more costly is a false negative which in the context of disease testing means that you tell a sick person that they are not sick and send them back into the population to spread the disease. This probability is:
   
   P(D|B^c) = [P(D)P(B^c|D)]/ [P(D)P(B^c|D) + P(D^c)P(B^c|D^c)] =
   = (.01 * .1)/[(.01 * .1) + (.99 * .9)] = .00112.
   
   Given how costly this error is the goal of disease testing is to drive this probability to zero while keeping P(D|B) as high as possible.
   
   
5. Problem 2.93 p.64
   We have five bowls, i=1, 2, 3, 4, 5, with i white and 5-i black balls in the ith bowl, respectively. The experiment consists of randomly choosing a bowl and randomly drawing two balls from the bowl without replacement. That is, we simply take two balls out of the bowl (in contrast, drawing with replacement means that we would take one ball out, note its color, drop it back into the bowl, and then take another ball out and note its color).
   
   Part (a): What is the probability that both balls are White?
   Let A₁, A₂, ..., A₅, represent the five bowls, and let B be the event "two White balls are drawn". Namely:
   

   

   Clearly:
   P(A₁) = P(A₂) = ... = P(A₅) = 1/5; and:
   P(B|A₁) = 0
   P(B|A₂) = {[2 choose 2][3 choose 0]}/[5 choose 2] = 1/10
   P(B|A₃) = {[3 choose 2][2 choose 0]}/10 = 3/10
   P(B|A₄) = {[4 choose 2][1 choose 0]}/10 = 6/10
   P(B|A₅) = [5 choose 2]/10 = 1
   Hence:
   P(B) =[å_j=1,5 P(A_j)P(B|A_j)] = (1/5)( 0 + 1/10 + 3/10 + 6/10 +1) = 2/5
   
   Part (b): Given that both balls are White, what is the probability that they were drawn from bowl 3?
   This is the probability:
   P(A₃|B) = P(A₃ Ç B)/P(B) = [P(A₃)P(B|A₃)]/P(B)] =
   (1/5 x 3/10)/(2/5) = 3/20