Probability and Conditional Probability: Study Notes for Introduction to Statistics (Lectures 7–13)

Probability Theory

Introduction to Probability

Probability forms the basis of inferential statistics, allowing us to make predictions about a population based on information from a sample. It quantifies the likelihood of events occurring in random experiments, such as flipping a coin or rolling a die.

• Random Process: A process whose outcome cannot be predicted with certainty.

• Random Experiment: The act of performing and observing the outcome of a random process.

• Outcome: One of the possible results of a random experiment.

• Sample Space (S): The set of all possible outcomes of a random experiment.

• Event: A collection of outcomes of a random experiment.

• Relative Frequency of an Event: The number of times an event occurs divided by the number of trials.

• Probability of an Event: The relative frequency of the event in the long run as the number of trials increases.

Example: Rolling a single die and observing which side is facing up is a random experiment. The sample space is {1, 2, 3, 4, 5, 6}.

What is Randomness?

Randomness refers to situations where outcomes cannot be predicted with certainty, even if the process is repeated under identical conditions. For example, flipping a fair coin or rolling a fair die are random processes.

Approaches to Probability

• Theoretical Probability: Based on mathematical reasoning and equally likely outcomes.

• Empirical Probability: Based on observations from actual experiments.

• Simulations: Use of computer models to estimate probabilities when experiments are impractical.

Example: To estimate the probability of flipping a coin and getting heads, you could flip the coin many times and record the results, or simulate the process using a computer.

Basic Probability Rules

Notation and Definitions

• Use capital letters (A, B, C, ...) to represent events.

• The complement of event A (not A) is denoted as Ac.

• The probability of event A is denoted as P(A).

Rules of Probability

• Rule 1: Probability values range from 0 to 1: $0 \leq P(A) \leq 1$

• Rule 2: The sum of probabilities of all possible outcomes in the sample space is 1.

• Rule 3 (Classical Probability): If all outcomes are equally likely, $P(A) = \frac{\text{Number of outcomes in } A}{\text{Number of all possible outcomes}}$

Example: If a die is rolled once, the probability of rolling a number greater than 4 (i.e., 5 or 6) is $P(A) = \frac{2}{6} = \frac{1}{3}$.

Compound Events and Combining Events

AND and OR Events

When analyzing more than one variable, we often combine events:

• AND Event (A AND B): Outcomes that are shared by both events A and B.

• OR Event (A OR B): Outcomes that are in A, in B, or in both.

Example: In a group of people, let event A be 'wearing glasses' and event B be 'raising hand'. The event A AND B is the set of people who are both wearing glasses and raising their hand. The event A OR B is the set of people who are either wearing glasses, raising their hand, or both.

Mutually Exclusive Events

Two events are mutually exclusive if they have no outcomes in common. In symbols, A and B are mutually exclusive if $A \cap B = \emptyset$.

Example: In a table of education level and marital status, 'single' and 'married' are mutually exclusive categories.

Probability Rules for Compound Events

• Rule 4a (General Addition Rule): $P(A \text{ OR } B) = P(A) + P(B) - P(A \text{ AND } B)$

• Rule 4b (Addition Rule for Mutually Exclusive Events): If A and B are mutually exclusive, $P(A \text{ OR } B) = P(A) + P(B)$

Example: If a die is rolled, the probability of getting an even number OR a number greater than 4 is $P(\text{even}) + P(>4) - P(\text{even AND } >4)$.

Probability Tables and Applications

Using Contingency Tables

Contingency tables summarize data for two categorical variables. Probabilities can be calculated by dividing the count of interest by the total number of observations.

Ed Level	Single	Married	Divorced	Widowed	Total
Less HS	17	70	10	28	125
HS	68	240	50	30	397
College	27	98	15	3	143
Total	112	408	84	61	665

Example Questions:

• What is the probability a person is married? $P(\text{Married}) = \frac{408}{665}$

• What is the probability a person has a college education? $P(\text{College}) = \frac{143}{665}$

• What is the probability a person is married AND has a college education? $P(\text{Married AND College}) = \frac{98}{665}$

• What is the probability a person is married OR has a college education? $P(\text{Married OR College}) = P(\text{Married}) + P(\text{College}) - P(\text{Married AND College})$

Conditional Probability

Definition and Calculation

Conditional probability is the probability of one event occurring given that another event has already occurred. It is denoted as $P(A|B)$, the probability of A given B.

• Rule 5a: $P(A|B) = \frac{P(A \text{ AND } B)}{P(B)}$

• Rule 5b: $P(A \text{ AND } B) = P(B) \cdot P(A|B)$

Example: In a table of political party and support for legalization, the probability that a randomly selected student is a Democrat AND supports legalization is $P(\text{Legalization AND Democrat}) = \frac{39}{126}$, and the conditional probability that a student supports legalization given they are a Democrat is $P(\text{Legalization} | \text{Democrat}) = \frac{39}{43}$.

Cannabis Opinion	Democrat	Republican	Independent	None of the Above	Total
Yes	39	25	12	20	96
No	4	15	5	6	30
Total	43	40	17	26	126

Independence and Association

Definition of Independence

Two events A and B are independent if the occurrence of one does not affect the probability of the other. Formally, A and B are independent if $P(A|B) = P(A)$.

If $P(A|B) \neq P(A)$, the events are associated (or dependent).

Example: In the context of rolling dice, the event 'roll is even' and 'roll is 4' are associated, since knowing the roll is 4 guarantees it is even.

Testing for Independence in Tables

To test if two categorical variables are independent, compare $P(A|B)$ and $P(A)$. If they are equal (or nearly so), the variables are independent.

Summary Table: Key Probability Rules

Rule	Formula	Description
Classical Probability	$P(A) = \frac{\text{Number of outcomes in } A}{\text{Number of all possible outcomes}}$	For equally likely outcomes
Addition Rule	$P(A \text{ OR } B) = P(A) + P(B) - P(A \text{ AND } B)$	For any two events
Mutually Exclusive Events	$P(A \text{ OR } B) = P(A) + P(B)$	If A and B cannot both occur
Conditional Probability	$P(A|B) = \frac{P(A \text{ AND } B)}{P(B)}$	Probability of A given B
Multiplication Rule	$P(A \text{ AND } B) = P(B) \cdot P(A|B)$	For dependent events
Independence	$P(A|B) = P(A)$	Events A and B are independent

Additional info:

• Some context and definitions were expanded for clarity and completeness.

• Examples and formulas were inferred and elaborated based on standard introductory statistics curriculum.