Probability, Random Variables, and Sampling Distributions: Structured Study Notes for Statistics

Probability Theory and Sets

Introduction to Probability

Probability is a numerical measure of the chance that an event will occur, ranging from 0 (impossibility) to 1 (certainty). It is used to quantify uncertainty in various phenomena.

• Probability: Any activity where the outcome is determined by chance and uncertainty.

• Set Theory: Foundation for probability, involving collections of objects called sets.

Concept of Set

Sets are collections of well-defined objects. Understanding sets is essential for probability theory.

• Listing method: e.g., A = {1, 2, 3, 4}

• Description method: e.g., A = {x | x is an even number less than 10}

• Empty set: A set with no elements, denoted by ∅.

• Subset: If every element of A is in B, then A is a subset of B, denoted A ⊆ B.

Basic Concepts and Terms

• Experiment: An activity with well-defined outcomes (e.g., tossing a coin).

• Sample space (S): The set of all possible outcomes.

• Event: A subset of the sample space.

• Elementary event: An event with a single outcome.

• Mutually exclusive events: Events that cannot occur simultaneously.

• Independent events: Occurrence of one does not affect the other.

Counting Techniques

Counting techniques help determine the number of possible ways to arrange or select objects.

• Addition Rule: If there are n ways to do one thing and m ways to do another, and the two cannot happen together, total ways = n + m.

• Multiplication Rule: If a procedure consists of k steps, with n1, n2, ..., nk ways for each step, total ways = n1 × n2 × ... × nk.

• Permutation: Arrangement of objects in a specific order. $P(n, r) = \frac{n!}{(n-r)!}$

• Combination: Selection of objects without regard to order. $C(n, r) = \binom{n}{r} = \frac{n!}{r!(n-r)!}$

Definitions of Probability Approaches

• Classical Probability: Based on equally likely outcomes. $P(E) = \frac{n(E)}{n(S)}$

• Empirical (Relative Frequency) Probability: Based on observed data. $P(E) = \frac{f}{n}$

• Subjective Probability: Based on personal judgment or experience.

Properties (Rules) of Probability

• Probability values lie between 0 and 1.

• The sum of probabilities of all outcomes in the sample space is 1.

• If two events are mutually exclusive, $P(E_1 \cup E_2) = P(E_1) + P(E_2)$.

• If not mutually exclusive, $P(E_1 \cup E_2) = P(E_1) + P(E_2) - P(E_1 \cap E_2)$.

Random Variables and Probability Distributions

Definition of Random Variable

A random variable is a variable whose value is determined by the outcome of a random experiment. It can be discrete (countable values) or continuous (any value in an interval).

• Discrete random variable: Takes countable values.

• Continuous random variable: Takes values in a continuous range.

Probability Distribution

A probability distribution lists all possible values of a random variable together with their probabilities.

• Discrete Probability Distribution: Probability mass function (pmf) $P(X=x)$.

• Continuous Probability Distribution: Probability density function (pdf) $f(x)$, with $\int_{-\infty}^{\infty} f(x) dx = 1$.

Expectation: Mean and Variance of a Random Variable

• Mean (Expected Value): $\mu = E(X) = \sum x_i p(x_i)$ for discrete, $\mu = \int x f(x) dx$ for continuous.

• Variance: $\sigma^2 = E[(X-\mu)^2] = \sum (x_i - \mu)^2 p(x_i)$ for discrete, $\sigma^2 = \int (x-\mu)^2 f(x) dx$ for continuous.

Common Discrete Probability Distributions

Binomial Distribution

• Models the number of successes in n independent Bernoulli trials.

• Probability mass function: $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$

• Mean: $\mu = np$

• Variance: $\sigma^2 = np(1-p)$

Poisson Distribution

• Models the number of events in a fixed interval of time/space.

• Probability mass function: $P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$

• Mean and variance: $\mu = \sigma^2 = \lambda$

Common Continuous Distributions

Normal Distribution

• Probability density function: $f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

• Mean: $\mu$

• Variance: $\sigma^2$

• Standardization: $Z = \frac{X - \mu}{\sigma}$

Sampling Distributions

Sampling Techniques

• Population: Entire group of interest.

• Sample: Subset of the population.

• Statistic: Characteristic of a sample.

• Parameter: Characteristic of a population.

Types of Sampling

• Probability Sampling: Each member has a known chance of selection (e.g., simple random, stratified, systematic, cluster sampling).

• Non-probability Sampling: Selection based on judgment or convenience.

Sampling Distribution of the Sample Mean

• The distribution of sample means from all possible samples of a given size.

• Mean of sampling distribution: $\mu_{\bar{x}} = \mu$

• Standard deviation (standard error): $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

• Finite population correction factor: $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} \sqrt{\frac{N-n}{N-1}}$

Sampling Distribution of the Sample Proportion

• Mean: $E(P) = p$

• Standard deviation: $\sigma_P = \sqrt{\frac{p(1-p)}{n}}$

• Finite population correction: $\sigma_P = \sqrt{\frac{p(1-p)}{n}} \sqrt{\frac{N-n}{N-1}}$

Statistical Estimation

Parameter Estimation

Statistical estimation involves using sample data to estimate population parameters such as the mean and standard deviation.

• Point estimation: Single value estimate of a parameter.

• Interval estimation: Range of values within which the parameter is expected to lie.

Tables

Example: Stratified Sampling Table

This table shows the allocation of sample sizes to different departments using proportional allocation.

Department	Total number of learners
Agricultural Economics	100
Animal science	80
Plant science	150
Dry land science	120
AGVM	50
EDAM	40
TOTAL	600

Sample size for Agricultural Economics: $n = \frac{100}{600} \times 202 \approx 34$

Example: Sampling Distribution Table

Sample Combinations	Sample mean
1,2	1.5
1,4	2.5
2,4	3.0

Mean of sample means: $\mu_{\bar{x}} = \frac{1.5 + 2.5 + 3.0}{3} = 2.33$

Example: Probability Distribution Table

x	p(x)
0	1/8
1	3/8
2	3/8
3	1/8

This table shows the probability distribution for the number of heads in three coin tosses.

Additional info:

• Some context and definitions were expanded for clarity and completeness.

• Examples and formulas were added to ensure the notes are self-contained and suitable for exam preparation.