4.1 Basic Probability Theory and Descriptive Statistics

Descriptive Statistics

Measures of Central Tendency and Spread

Descriptive statistics summarize and describe the main features of a data set. The two primary categories are measures of central tendency and measures of spread.

• Measures of Central Tendency: These indicate the center or typical value of a data set.

  • Mean: The arithmetic average of the data.

  • Median: The middle value when data are ordered.

  • Mode: The most frequently occurring value.

• Measures of Spread: These describe the variability or dispersion in the data.

  • Range: Difference between the largest and smallest values.

  • Variance: Average squared deviation from the mean.

  • Standard Deviation: Square root of the variance.

Summation Notation: Data can be represented as lists $\{x_i\}_{i=1}^n$. The capital-Sigma notation $\sum_{i=1}^n x_i$ is used to sum elements $x_i$ from $i=1$ to $n$.

Parameters vs. Estimators

Parameters are characteristics of a population, while estimators are calculated from a sample to estimate population parameters.

	Mean	Variance	Standard Deviation
Parameter	$\mu$	$\sigma^2$	$\sigma$
Estimator	$\bar{x}$	$s^2$	$s$

Formulas:

• If the data is a population:

  • $\mu = \frac{1}{n} \sum_{i=1}^n x_i$

  • $\sigma^2 = \frac{1}{n} \sum_{i=1}^n (x_i - \mu)^2$

• If the data is a sample:

  • $\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$

  • $s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2$

Variance: Properties and Calculation

Variance measures the average squared deviation from the mean. For sample variance:

• $s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2$

• Equivalent form for calculation: $s^2 = \frac{1}{n-1} \left( \sum_{i=1}^n x_i^2 - n\bar{x}^2 \right)$

• Important property: $s^2 \geq 0$ (variance is always nonnegative).

• Also, $\frac{1}{n} \sum_{i=1}^n x_i^2 \geq \bar{x}^2$

Basic Probability Theory

Motivation and Definition

Probability quantifies the likelihood of uncertain events. It helps distinguish likely from unlikely outcomes and guides decision-making in uncertain situations.

• Examples: Rolling a 3 with a fair die, electricity demand exceeding capacity, predicting a recession.

Sample Space, Events, and Set Theory

Probability theory begins with defining the sample space and events using set theory.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: A subset of the sample space.

• Set Theory Operations:

  • Empty set ($\varnothing$): Contains no objects.

  • Subset ($A \subset B$): All elements of $A$ are in $B$.

  • Intersection ($A \cap B$): Elements in both $A$ and $B$.

  • Union ($A \cup B$): Elements in $A$, $B$, or both.

  • Complement ($A^C$): Elements in $S$ not in $A$.

  • Difference ($A \setminus B$): Elements in $A$ but not in $B$.

Example (Coin Toss): $S = \{H, T\}$; possible events: $\varnothing$, $\{H\}$, $\{T\}$, $\{H, T\}$.

Example (Die Roll): $S = \{1,2,3,4,5,6\}$; event $A = \{1,2,3\}$ (outcome three or less), event $B = \{2,4,6\}$ (even outcome).

Probability Axioms and Properties

A probability function $P$ assigns a number in $[0,1]$ to each event, satisfying:

• For any event $A$, $0 \leq P(A) \leq 1$

• $P(S) = 1$

• For disjoint events $A_1$ and $A_2$, $P(A_1 \cup A_2) = P(A_1) + P(A_2)$

If these properties are not met, $P$ is not a valid probability function.

Probability Calculations: Examples

• Uniform Probability (Die): For $S = \{1,2,3,4,5,6\}$, $P(i) = \frac{1}{6}$ for each $i$.

• Event Probability: For $A = \{1,2,3\}$, $P(A) = \frac{3}{6} = 0.5$

• Intersection: $A \cap B = \{2\}$

• Union: $A \cup B = \{1,2,3,4,6\}$

Probability Properties

• Complement Rule: $P(A^C) = 1 - P(A)$

• Null Set: $P(\varnothing) = 0$

• Addition Rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• If $A$ and $B$ are disjoint, $P(A \cup B) = P(A) + P(B)$

Conditional Probability

Conditional probability updates the likelihood of an event based on new information.

• Definition: For events $A$ and $B$ with $P(B) > 0$,

  • $P(A|B) = \frac{P(A \cap B)}{P(B)}$

• Interpretation: Probability of $A$ given $B$ has occurred.

• Note: $P(A|B) \neq P(B|A)$

Example (Medical Test):

	Test Positive	Test Negative	Total
Has COVID	95	5	100
No COVID	90	810	900
Total	185	815	1000

• $P(\text{Has COVID}) = \frac{100}{1000} = 0.1$

• $P(\text{Has COVID} | \text{Test Pos}) = \frac{95}{185} \approx 0.514$

• $P(\text{Test Pos} | \text{Has COVID}) = \frac{95}{100} = 0.95$

Multiplication Rule

The multiplication rule relates joint probability to conditional probability:

• $P(A \cap B) = P(A|B)P(B)$

Example: If $P(D) = 0.02$ (defective), $P(A|D) = 0.05$ (pass inspection given defective), then $P(D \cap A) = 0.02 \times 0.05 = 0.001$

Independence

Two events $A$ and $B$ are independent if the occurrence of one does not affect the probability of the other.

• Definition: $P(A \cap B) = P(A)P(B)$

• If $P(A|B) = P(A)$ and $P(B|A) = P(B)$, then $A$ and $B$ are independent.

Example (Workforce Table):

Sex	Sales	Clerical	Production	Total
Female	1700	800	250	2750
Male	800	700	750	2250
Total	2500	1500	1000	5000

• $P(F) = \frac{2750}{5000} = 0.55$

• $P(C) = \frac{1500}{5000} = 0.3$

• $P(F \cap C) = \frac{800}{5000} = 0.16$

• $P(F)P(C) = 0.55 \times 0.3 = 0.165$ (not equal to $P(F \cap C)$, so not independent)

Law of Total Probability

The law of total probability expresses the probability of an event as the sum over a partition of the sample space:

• $P(A) = \sum_{i=1}^n P(A|B_i)P(B_i)$, where $B_1, ..., B_n$ partition $S$

Example: If $P(\text{Sunny}) = 0.6$, $P(\text{Cloudy}) = 0.4$, $P(\text{Rain} | \text{Sunny}) = 0.1$, $P(\text{Rain} | \text{Cloudy}) = 0.5$:

• $P(\text{Rain}) = 0.1 \times 0.6 + 0.5 \times 0.4 = 0.06 + 0.2 = 0.26$

Bayes' Rule

Bayes' rule allows us to update the probability of a cause given an observed effect:

• $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

Example (Entomology): Suppose 98% of rare beetles have a pattern, 5% of common beetles have it, and rare beetles are 0.1% of all beetles. What is the probability a beetle is rare given the pattern?

• $P(\text{Pattern} | R) = 0.98$

• $P(R) = 0.001$

• $P(\text{Pattern} | \text{Common}) = 0.05$

• $P(\text{Common}) = 0.999$

• $P(R | \text{Pattern}) = \frac{0.98 \times 0.001}{0.98 \times 0.001 + 0.05 \times 0.999} \approx 0.019$

References

• Telhammer, R. C. (2013). Mathematical Statistics for Economics and Business. Springer.