Discrete and Continuous Random Variables: Probability Distributions, Expectation, and Applications

Discrete Random Variables and Probability Distributions

Definition and Properties

A random variable whose set of possible values is either finite or countably infinite is called discrete. For a discrete random variable X, the probability mass function (pmf) is defined as:

• Probability Mass Function: $p(x) = P\{X = x\}$

• Expected Value (Mean): $E[X] = \sum_{x: p(x) > 0} x p(x)$

• Variance: $Var(X) = E[(X - E[X])^2]$

• Useful Identity: $Var(X) = E[X^2] - (E[X])^2$

• Standard Deviation: $\sqrt{Var(X)}$

• Linearity of Expectation: $E\left[\sum_{i=1}^n X_i\right] = \sum_{i=1}^n E[X_i]$

• Expectation of a Function: $E[g(X)] = \sum_{x: p(x) > 0} g(x) p(x)$

Common Discrete Random Variables

• Binomial: Number of successes in n independent trials, each with probability p.

  • PMF: $p(i) = \binom{n}{i} p^i (1-p)^{n-i}$, $i = 0, ..., n$

  • Mean: $E[X] = np$

  • Variance: $Var(X) = np(1-p)$

• Poisson: Number of events in a fixed interval, parameter $\lambda$.

  • PMF: $p(i) = \frac{e^{-\lambda} \lambda^i}{i!}$, $i \geq 0$

  • Mean and Variance: $E[X] = Var(X) = \lambda$

• Geometric: Number of trials until first success, parameter $p$.

  • PMF: $p(i) = p(1-p)^{i-1}$, $i = 1, 2, ...$

  • Mean: $E[X] = \frac{1}{p}$

  • Variance: $Var(X) = \frac{1-p}{p^2}$

• Negative Binomial: Number of trials until r-th success, parameters $r$, $p$.

  • PMF: $p(i) = \binom{i-1}{r-1} p^r (1-p)^{i-r}$, $i \geq r$

  • Mean: $E[X] = \frac{r}{p}$

  • Variance: $Var(X) = \frac{r(1-p)}{p^2}$

• Hypergeometric: Number of white balls in n draws from N balls (m white).

  • PMF: $p(i) = \frac{\binom{m}{i} \binom{N-m}{n-i}}{\binom{N}{n}}$, $i = 0, ..., m$

  • Mean: $E[X] = np$, $p = \frac{m}{N}$

  • Variance: $Var(X) = \frac{N-n}{N-1} np(1-p)$

Examples and Applications

• Binomial Example: Flipping a fair coin n times and counting heads.

• Poisson Example: Number of phone calls received in an hour.

• Geometric Example: Number of coin tosses until first heads.

• Hypergeometric Example: Drawing balls from an urn without replacement.

Continuous Random Variables and Probability Distributions

Definition and Properties

A continuous random variable has an uncountable set of possible values. The probability density function (pdf) $f(x)$ satisfies:

• $P\{X \in B\} = \int_B f(x) dx$ for any set $B$

• $\int_{-\infty}^{\infty} f(x) dx = 1$

• $P\{X = a\} = 0$ for any fixed $a$

• Cumulative Distribution Function (CDF): $F(a) = P\{X \leq a\} = \int_{-\infty}^a f(x) dx$

• Relationship: $\frac{d}{da} F(a) = f(a)$

Expectation and Variance

• Expected Value: $E[X] = \int_{-\infty}^{\infty} x f(x) dx$

• Expectation of a Function: $E[g(X)] = \int_{-\infty}^{\infty} g(x) f(x) dx$

• Variance: $Var(X) = E[(X - \mu)^2] = E[X^2] - (E[X])^2$

• Linearity: $E[aX + b] = aE[X] + b$

• Variance Scaling: $Var(aX + b) = a^2 Var(X)$

Common Continuous Random Variables

• Uniform: $f(x) = \frac{1}{\beta - \alpha}$ for $\alpha < x < \beta$, 0 otherwise.

  • Mean: $E[X] = \frac{\alpha + \beta}{2}$

  • Variance: $Var(X) = \frac{(\beta - \alpha)^2}{12}$

  • Example: Waiting time uniformly distributed between 0 and 30 minutes.

• Normal (Gaussian): $f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ for $-\infty < x < \infty$

  • Mean: $\mu$

  • Variance: $\sigma^2$

  • Standard Normal: $Z = \frac{X - \mu}{\sigma}$

  • Normal Approximation to Binomial: For large $n$, $Binomial(n, p)$ is approximated by $Normal(np, np(1-p))$

  • Example: Heights, measurement errors, test scores.

• Exponential: $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$, 0 otherwise.

  • Mean: $E[X] = \frac{1}{\lambda}$

  • Variance: $Var(X) = \frac{1}{\lambda^2}$

  • Memoryless Property: $P\{X > s + t | X > t\} = P\{X > s\}$

  • Example: Time until failure, time between arrivals.

• Gamma: $f(x) = \frac{\lambda e^{-\lambda x} (\lambda x)^{\alpha - 1}}{\Gamma(\alpha)}$ for $x \geq 0$, 0 otherwise.

  • Mean: $E[X] = \frac{\alpha}{\lambda}$

  • Variance: $Var(X) = \frac{\alpha}{\lambda^2}$

  • Special Case: $\chi^2$ distribution when $\lambda = \frac{1}{2}$, $\alpha = \frac{n}{2}$

• Laplace: $f(x) = \frac{1}{2\lambda} e^{-\lambda |x|}$ for $-\infty < x < \infty$

  • Example: Channel noise in communications.

• Weibull: Used in reliability and life data analysis (details omitted here).

Hazard Rate Function

• Definition: $\lambda(t) = \frac{f(t)}{1 - F(t)}$

• For exponential distribution, $\lambda(t) = \lambda$ (constant)

• Hazard rate uniquely determines the distribution: $F(t) = 1 - \exp\left(-\int_0^t \lambda(s) ds\right)$

Applications and Problem Types

Sample Problems

• Calculating probabilities for discrete and continuous random variables (e.g., dice rolls, coin tosses, urn problems).

• Finding expected values and variances for various distributions.

• Conditional probabilities and independence.

• Using normal approximation for binomial probabilities (with continuity correction).

• Applying the memoryless property of the exponential distribution.

• Hazard rate interpretation in reliability engineering.

Table: Normal Distribution Function Values

The following table provides values of the cumulative distribution function $\Phi(x)$ for the standard normal distribution:

X	$\Phi(x)$
0.0	0.5000
1.0	0.8413
2.0	0.9772
3.0	0.9987
-1.0	0.1587
-2.0	0.0228
-3.0	0.0013

Additional info: The full table is available in most statistics textbooks and is used to compute probabilities for normal random variables.

Summary Table: Key Distributions

Distribution	PMF/PDF	Mean	Variance	Typical Application
Binomial	$\binom{n}{k} p^k (1-p)^{n-k}$	$np$	$np(1-p)$	Number of successes in n trials
Poisson	$\frac{e^{-\lambda} \lambda^k}{k!}$	$\lambda$	$\lambda$	Events in fixed interval
Normal	$\frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$	$\mu$	$\sigma^2$	Measurement errors, heights
Exponential	$\lambda e^{-\lambda x}$	$\frac{1}{\lambda}$	$\frac{1}{\lambda^2}$	Time until event
Uniform	$\frac{1}{b-a}$	$\frac{a+b}{2}$	$\frac{(b-a)^2}{12}$	Random selection in interval

Conclusion

This study guide summarizes the foundational concepts of discrete and continuous random variables, their probability distributions, expectation and variance, and key applications. Understanding these distributions and their properties is essential for solving problems in probability and statistics, including engineering, science, and data analysis contexts.