Total Probability Theorem and Bayes’ Theorem
Introduction
In probability theory, the Total Probability Theorem and Bayes’ Theorem are fundamental concepts used to compute probabilities based on known conditions or prior events. Both are particularly useful when dealing with conditional probabilities, allowing us to update beliefs based on new information. This article provides a detailed overview of both concepts, including definitions, formulas, and real-life examples.
—
Total Probability Theorem
Definition
The Total Probability Theorem is used to calculate the probability of an event by considering all possible ways that the event can occur. If a set of events \( B_1, B_2, \dots, B_n \) is mutually exclusive and exhaustive (covering all possibilities), then the probability of any event \( A \) can be calculated as:
\[
P(A) = \sum_{i=1}^{n} P(A \mid B_i) \cdot P(B_i)
\]
Where:
– \( P(A \mid B_i) \) is the conditional probability of \( A \) given \( B_i \).
– \( P(B_i) \) is the probability of each possible condition or scenario \( B_i \).
This theorem is helpful when event \( A \) can happen under several different conditions, represented by \( B_1, B_2, \dots, B_n \).
Real-Life Example of the Total Probability Theorem
Example: Product Defects Across Factories
Suppose a company has three factories (A, B, and C) producing identical products. Factory A produces 30% of the total products, factory B 50%, and factory C 20%. The probability of a product being defective from each factory is as follows:
– \( P(\text{Defective} \mid A) = 0.01 \)
– \( P(\text{Defective} \mid B) = 0.02 \)
– \( P(\text{Defective} \mid C) = 0.03 \)
To find the total probability that a randomly selected product is defective, we can use the Total Probability Theorem:
\[
P(\text{Defective}) = P(\text{Defective} \mid A) \cdot P(A) + P(\text{Defective} \mid B) \cdot P(B) + P(\text{Defective} \mid C) \cdot P(C)
\]
Substituting values:
\[
P(\text{Defective}) = (0.01 \cdot 0.30) + (0.02 \cdot 0.50) + (0.03 \cdot 0.20)
\]
\[
P(\text{Defective}) = 0.003 + 0.01 + 0.006 = 0.019
\]
So, the probability that a randomly selected product is defective is 0.019, or 1.9%.
—
Bayes’ Theorem
Definition
Bayes’ Theorem allows us to update the probability of an event based on new information. It is particularly useful in situations where we want to revise our belief about an event after obtaining additional data.
For two events \( A \) and \( B \), Bayes’ Theorem is given by:
\[
P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)}
\]
Where:
– \( P(A \mid B) \) is the updated (posterior) probability of \( A \) given \( B \).
– \( P(B \mid A) \) is the probability of observing \( B \) given that \( A \) is true.
– \( P(A) \) is the prior probability of \( A \) before knowing \( B \).
– \( P(B) \) is the total probability of \( B \), which can be calculated using the Total Probability Theorem if necessary.
Real-Life Example of Bayes’ Theorem
Example: Medical Testing
Suppose a disease affects 1% of a population (\( P(\text{Disease}) = 0.01 \)). A test for the disease has:
– 99% accuracy for people who have the disease (\( P(\text{Positive} \mid \text{Disease}) = 0.99 \)).
– 5% false positive rate for people who do not have the disease (\( P(\text{Positive} \mid \text{No Disease}) = 0.05 \)).
Given that a person tested positive, we want to find the probability that they actually have the disease, \( P(\text{Disease} \mid \text{Positive}) \).
Using Bayes’ Theorem:
\[
P(\text{Disease} \mid \text{Positive}) = \frac{P(\text{Positive} \mid \text{Disease}) \cdot P(\text{Disease})}{P(\text{Positive})}
\]
To find \( P(\text{Positive}) \), we use the Total Probability Theorem:
\[
P(\text{Positive}) = P(\text{Positive} \mid \text{Disease}) \cdot P(\text{Disease}) + P(\text{Positive} \mid \text{No Disease}) \cdot P(\text{No Disease})
\]
\[
P(\text{Positive}) = (0.99 \cdot 0.01) + (0.05 \cdot 0.99) = 0.0099 + 0.0495 = 0.0594
\]
Now, substitute into Bayes’ Theorem:
\[
P(\text{Disease} \mid \text{Positive}) = \frac{0.99 \cdot 0.01}{0.0594} = \frac{0.0099}{0.0594} \approx 0.1667
\]
Thus, even with a positive test result, there is only a 16.67% chance that the person actually has the disease. This example shows how Bayes’ Theorem adjusts the probability based on new evidence (the test result).
—
Summary
– Total Probability Theorem helps calculate the overall probability of an event by considering all possible ways it can happen.
– Bayes’ Theorem allows us to update probabilities based on new information, adjusting our beliefs when we learn something new.
These concepts are foundational in statistics, data analysis, and decision-making, providing tools to handle complex situations with conditional probabilities and updated information.
