Fundamental Probability Concepts and Rules

Probability: Fundamental Concepts

Random Experiments and Sample Spaces

Probability theory begins with the study of random experiments—processes whose outcomes cannot be predicted with certainty in advance. The set of all possible outcomes of a random experiment is called the sample space, denoted by S.

• Random Experiment: An action or process that leads to one of several possible outcomes, where the outcome cannot be predicted with certainty.

• Sample Space (S): The set of all possible outcomes of a random experiment. Examples:

  • Tossing a coin: S = {Head, Tail}

  • Rolling a die: S = {1, 2, 3, 4, 5, 6}

• Event: Any subset of the sample space. An event may consist of one or more outcomes. Examples:

  • Tossing a head (single outcome)

  • Tossing two heads in two tosses (multiple outcomes)

Probability of Events

The probability of an event A, denoted by P(A), quantifies the likelihood that event A will occur. Probabilities are real numbers between 0 and 1, inclusive.

• Properties of Probability:

  • $0 \leq P(A) \leq 1$

  • $P(A) = 0$ means event A is impossible.

  • $P(A) = 1$ means event A is certain.

  • The larger the value of $P(A)$, the more likely event A is to occur.

• Sum of Probabilities: The sum of the probabilities of all non-overlapping (mutually exclusive) outcomes in the sample space is 1. Example: For a fair die, each face has probability $\frac{1}{6}$, and $\frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = 1$.

Set Operations and Event Relationships

Complement, Union, and Intersection

• Complement of an Event (Ac): The event that A does not occur. $A^c$ contains all outcomes in S that are not in A.

• Union (A or B): The event that either A, B, or both occur. Denoted $A \cup B$.

• Intersection (A and B): The event that both A and B occur. Denoted $A \cap B$.

Venn Diagrams are often used to visualize these relationships:

• Complement: The area outside A but within S.

• Union: The area covered by either A, B, or both.

• Intersection: The area where A and B overlap.

Disjointness and Independence

• Disjoint (Mutually Exclusive) Events: Two events A and B are disjoint if they cannot occur together (i.e., $A \cap B = \emptyset$). The occurrence of one prevents the occurrence of the other.

• Non-Disjoint Events: Events that can occur together (i.e., $A \cap B \neq \emptyset$).

• Independence of Events: Two events A and B are independent if the occurrence of A does not affect the probability of B, and vice versa. Formally, A and B are independent if $P(B|A) = P(B)$ or $P(A|B) = P(A)$.

• Independence of Trials: Repeated trials are independent if the outcome of one trial does not influence the outcome of another (e.g., repeated coin tosses).

Probability Rules

Complement Rule

The probability that event A does not occur is given by:

• $P(A^c) = 1 - P(A)$

Addition Rules

• Addition Rule for Disjoint Events: If A and B are disjoint, then $P(A \text{ or } B) = P(A) + P(B)$

• General Addition Rule: If A and B are not disjoint, then $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

Classical Probability (Equally Likely Outcomes)

If all outcomes in the sample space are equally likely, the probability of event A is:

• $P(A) = \frac{\text{number of outcomes in event A}}{\text{total number of outcomes in the sample space}}$

Conditional Probability and Independence

Conditional Probability

The conditional probability of event B given that event A has occurred is denoted $P(B|A)$ and is defined as:

• $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

Multiplication Rules

• Independent Events: If A and B are independent, then $P(A \text{ and } B) = P(A) \times P(B)$

• Dependent Events (General Multiplication Rule): If A and B are not independent, then $P(A \text{ and } B) = P(A) \times P(B|A)$ or $P(A \text{ and } B) = P(B) \times P(A|B)$

To determine if two events A and B are independent, check if any of the following hold:

• $P(A) = P(A|B)$

• $P(B) = P(B|A)$

• $P(A \text{ and } B) = P(A) \times P(B)$

Summary Table: Event Relationships

Relationship	Definition	Key Formula
Disjoint (Mutually Exclusive)	Events cannot occur together	$P(A \text{ and } B) = 0$
Independent	Occurrence of one does not affect the other	$P(A \text{ and } B) = P(A) \times P(B)$
General (Not Disjoint or Independent)	Events may overlap or be dependent	$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

Examples and Applications

• Example 1: Tossing a fair coin. S = {Head, Tail}. P(Head) = 0.5, P(Tail) = 0.5.

• Example 2: Rolling a fair die. S = {1,2,3,4,5,6}. P(rolling a 3) = $\frac{1}{6}$.

• Example 3: If A = rolling an even number, B = rolling an odd number, then A and B are disjoint.

• Example 4: If A = first child is a boy, B = second child is a boy (in a family with two children), then A and B are not disjoint and may be independent depending on context.

Additional info: These notes provide a foundation for understanding probability, including the structure of sample spaces, event relationships, and the main rules for calculating probabilities in both simple and compound events. Mastery of these concepts is essential for further study in statistics and data analysis.