Probability of $B$ Given That $A$ Has Occurred

Our estimation of the likelihood of an event can change if we know that some other event has occurred. For example, the probability that a rolled die shows a $2$ is $1/6$ without any other information, but if someone looks at the die and tells you that is is an even number, the probability is now $1/3$ that it is a $2$. The notation $P(B|A)$ indicates a conditional probability, meaning it indicates the probability of one event under the condition that we know another event has happened. The bar "|" can be read as "given", so that $P(B|A)$ is read as "the probability of $B$ given that $A$ has occurred".

The conditional probability $\displaystyle P(B|A)$ of an event $B$, given an event $A$, is defined by:

$\displaystyle P(B|A)=\frac{P(A\cap B)}{P(A)}$ 

When $P(A) > 0$. Be sure to remember the distinct roles of $B$ and $A$ in this formula. The set after the bar is the one we are assuming has occurred, and its probability occurs in the denominator of the formula.  

Example

Suppose that a coin is flipped 3 times giving the sample space:

$S=\{HHH, HHT, HTH, THH, TTH, THT, HTT, TTT\}$ 

Each individual outcome has probability $1/8$. Suppose that $B$ is the event that at least one heads occurs and $A$ is the event that all $3$ coins are the same. Then the probability of $B$ given $A$ is $1/2$, since $A \cap B=\{HHH\}$ which has probability $1/8$ and $A=\{HHH,TTT\}$ which has probability $2/8$, and $\frac{1/8}{2/8}=\frac{1}{2}.$

Independence

The conditional probability $P(B|A)$ is not always equal to the unconditional probability $P(B)$. The reason behind this is that the occurrence of event $A$ may provide extra information that can change the probability that event $B$ occurs. If the knowledge that event $A$ occurs does not change the probability that event $B$ occurs, then $A$ and $B$ are independent events, and thus, $P(B|A) = P(B)$.

Bayes' Theorem

In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule) is a result that is of importance in the mathematical manipulation of conditional probabilities. It can be derived from the basic axioms of probability.

Mathematically, Bayes' theorem gives the relationship between the probabilities of $A$ and $B$, $P(A)$ and $P(B)$, and the conditional probabilities of $A$ given $B$ and $B$ given $A$. In its most common form, it is:

$\displaystyle P(A|B)=\frac{P(B|A)P(A)}{P(B)}$

This may be easier to remember in this alternate symmetric form: 

$\displaystyle \frac{P(A|B)}{P(B|A)}=\frac{P(A)}{P(B)}$

Example:

Suppose someone told you they had a nice conversation with someone on the train. Not knowing anything else about this conversation, the probability that they were speaking to a woman is $50\%$. Now suppose they also told you that this person had long hair. It is now more likely they were speaking to a woman, since women in in this city are more likely to have long hair than men. Bayes's theorem can be used to calculate the probability that the person is a woman.

To see how this is done, let $W$ represent the event that the conversation was held with a woman, and $L$ denote the event that the conversation was held with a long-haired person. It can be assumed that women constitute half the population for this example. So, not knowing anything else, the probability that $W$ occurs is $P(W) = 0.5$.

Suppose it is also known that $75\%$ of women in this city have long hair, which we denote as $P(L|W) = 0.75$. Likewise, suppose it is known that $25\%$ of men in this city have long hair, or $P(L|M) = 0.25$, where $M$ is the complementary event of $W$, i.e., the event that the conversation was held with a man (assuming that every human is either a man or a woman).

Our goal is to calculate the probability that the conversation was held with a woman, given the fact that the person had long hair, or, in our notation, $P(W|L)$. Using the formula for Bayes's theorem, we have:

$\displaystyle \begin{aligned} P(W|L) &= \frac{P(L|W)P(W)}{P(L)}\\ &= \frac{P(L|W)P(W)}{P(L|W)P(W)+P(L|M)P(M)}\\ &=\frac{0.75\cdot 0.5}{0.75\cdot 0.5+0.25\cdot 0.5}\\ &=0.75 \end{aligned}$