Probability Concepts, Rules, and Bayes' Theorem: Structured Study Notes

Probability Concepts

Basic Probability Rules

Probability is a measure of how likely an event is to occur, ranging from 0 (impossible) to 1 (certain). It is used to quantify uncertainty in experiments and real-world scenarios.

• Experiment: A repeatable process with well-defined outcomes (e.g., rolling a die, flipping a coin).

• Outcome: The result of a single trial of an experiment.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: Any subset of the sample space.

Example: Rolling a six-sided die. Sample space: S = {1, 2, 3, 4, 5, 6}. Probability of rolling a 6: $P(6) = \frac{1}{6}$.

Events and Interpretations of Probability

Probability can be interpreted in two main ways: as the long-run relative frequency of an event (frequentist) or as a measure of belief (subjective).

• Relative Frequency: Probability estimated by the proportion of times an event occurs in repeated trials.

• Equally Likely Outcomes: If all outcomes are equally likely, $P(A) = \frac{\text{number of outcomes in } A}{\text{total number of outcomes}}$.

Example: Rolling a die 1,000 times, each value will occur about 1/6 of the time.

Law of Large Numbers

As the number of trials increases, the proportion of occurrences of any given outcome approaches its theoretical probability.

• Bernoulli's Law: The proportion of heads in coin flips approaches 0.5 as the number of flips increases.

Unions, Intersections, and the Addition Rule

Events can be combined using unions (either event occurs) and intersections (both events occur).

• Addition Rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• Mutually Exclusive Events: If events cannot occur together, $P(A \cap B) = 0$, so $P(A \cup B) = P(A) + P(B)$

Example: Probability a student likes either coffee or tea: $P(C \cup T) = P(C) + P(T) - P(C \cap T)$

Probability from Contingency Tables

Contingency tables display the frequency of outcomes for two categorical variables. Probabilities can be estimated by dividing the frequency of interest by the total number of observations.

Pulmonary embolism status	D-dimer positive	D-dimer negative	Total
Present	1000	1000	2000
Absent	7000	7000	14000
Total	8000	8000	16000

Example: Probability a patient has a pulmonary embolism and D-dimer positive: $P(E \cap D) = \frac{1000}{16000} = 0.0625$

Conditional Probability and Independence

Conditional Probability

Conditional probability quantifies the likelihood of an event given that another event has occurred.

• Formula: $P(A|B) = \frac{P(A \cap B)}{P(B)}$

Example: Probability of cancer given smoking status, using a contingency table.

Independent and Dependent Events

Events are independent if the occurrence of one does not affect the probability of the other.

• Independence: $P(A \cap B) = P(A)P(B)$

• Dependent Events: $P(A \cap B) = P(A|B)P(B)$

Example: Flipping two coins, the outcome of one does not affect the other.

Mutually Exclusive Events vs. Independent Events

Mutually exclusive events cannot occur together, while independent events do not affect each other's probabilities.

• Mutually Exclusive: $P(A \cap B) = 0$

• Independent: $P(A \cap B) = P(A)P(B)$

Conditional Probability from Contingency Tables

Conditional probabilities can be estimated from contingency tables by dividing the frequency of the joint event by the frequency of the conditioning event.

Pulmonary embolism status	D-dimer positive	D-dimer negative	Total
Present	700	300	1000
Absent	2100	6900	9000
Total	2800	7200	10000

Example: $P(E|D) = \frac{700}{2800} = 0.25$

Bayes' Rule

Bayes' Rule and Diagnostic Testing

Bayes' Rule allows us to update probabilities based on new evidence, such as test results in medical diagnostics.

• Formula: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

• Sensitivity: Probability a test correctly identifies a diseased individual as positive.

• Specificity: Probability a test correctly identifies a healthy individual as negative.

• Positive Predictive Value: $P(D|T)$, probability a person has the disease given a positive test result.

• Prevalence: Proportion of individuals in the population who truly have the disease.

Example: If a test has sensitivity 0.95, specificity 0.90, and prevalence 0.10, Bayes' Rule can be used to compute the probability a person with a positive test actually has the disease.

Understanding Bayes' Rule Through an Example

Suppose a disease has prevalence $P(D) = 0.05$, test sensitivity $P(T|D) = 0.90$, and specificity $P(T^c|D^c) = 0.95$.

• Probability of positive test: $P(T) = P(T|D)P(D) + P(T|D^c)P(D^c)$

• Probability of disease given positive test: $P(D|T) = \frac{P(T|D)P(D)}{P(T)}$

Recap Table

Keyword/Concept	Definition/Meaning
Probability	Numerical value between 0 and 1 measuring how likely an event is to occur
Experiment	Repeatable process with uncertain outcome
Sample space	Set of all possible outcomes
Event	Subset of sample space
Conditional probability	Probability of one event given another has occurred
Independence	Events do not affect each other's probabilities
Bayes' Rule	Formula to update probabilities in light of new evidence

Additional info:

• Examples and applications are drawn from medical testing, dice rolling, and coin flipping.

• Tables are used to illustrate contingency table calculations and probability estimation.

• JMP Pro 17 is referenced for computational probability tasks.