Introduction to Probability in Statistics

Probability Concepts

Introduction to Probability

Probability is a foundational concept in statistics, especially in inferential statistics, where we use data from a sample to make conclusions about a population. Because samples may not perfectly represent the population, probability allows us to quantify the uncertainty in our conclusions.

• Inferential statistics use probability to estimate how likely it is that a sample result reflects the true population.

• Probability helps us attach a numerical value to the likelihood that a conclusion is correct or incorrect.

• For example, if there is a 20% probability that a conclusion is wrong, we may want to collect more data; if there is only a 1% probability, we may be confident in our result.

Definition of Probability

A probability is a number between 0 and 1 (inclusive) that expresses the likelihood of an event occurring. It can also be written as a percentage between 0% and 100%.

• Probability of 0: The event is impossible.

• Probability of 1: The event is certain.

• All probabilities refer to events (e.g., the event that it rains tomorrow).

Example: The probability of rain tomorrow is 20%, or 0.20.

Probability Notation

• We use P(Event) to denote the probability of an event.

• For example, if R is the event "it rains tomorrow": $P(R) = 0.20$

• Probabilities are always between 0 and 1: $0 \leq P \leq 1$

Experiments, Outcomes, and Events

To determine probabilities, we perform an experiment and observe its outcomes. An event is a set of one or more outcomes.

• Outcome: A possible result of an experiment.

• Event: A set of one or more outcomes.

• Sample space: The set of all possible outcomes of an experiment.

Experiment	Sample Space (All Possible Outcomes)
Observe whether it rains tomorrow	It rains, it does not rain
Roll a fair die and count the spots	1, 2, 3, 4, 5, 6
Roll two fair dice and count the spots	2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
Find the mean salary in dollars of a random sample of 100 new computer science graduates	A number between 0 and 100,000

• Example (Die Roll):

  • Event A: The number is five (A = {5})

  • Event B: The number is greater than 4 (B = {5, 6})

  • Event C: The number is even (C = {2, 4, 6})

• Example (Two Dice):

  • Event D: 2 spots (D = {2})

  • Event E: 7 spots (E = {7})

• Example (Sample Mean):

  • Event F: mean = 60,000

  • Event G: mean > 50,000

  • Event H: mean < 60,000

Sample Space

The sample space is the set of all possible outcomes of an experiment. In the table above, the second column lists the sample spaces for each experiment.

Finding Probabilities: Equally Likely Outcomes

When all outcomes in the sample space are equally likely, the probability of an event is the ratio of the number of favorable outcomes to the total number of outcomes.

• Let f = number of ways the event can occur (frequency)

• Let N = total number of outcomes in the sample space

The probability of an event is:

$P = \frac{f}{N}$

• Example (Die Roll):

  • $P(A) = \frac{1}{6} = 0.1667$

  • $P(B) = \frac{2}{6} = 0.3333$

  • $P(C) = \frac{3}{6} = 0.5$

• Example (Classroom): There are 35 students, 19 are male. Probability a randomly chosen student is male:

  • $P = \frac{19}{35} = 0.5429$

Frequentist Interpretation of Probability

The frequentist interpretation defines probability as the long-run relative frequency of an event occurring in repeated independent trials of an experiment.

• If an experiment is repeated many times, the probability of an event is the proportion of times the event occurs.

• Example: Tossing a fair coin many times, the proportion of heads approaches 0.5 as the number of tosses increases.

• Example: If the probability of rain is 0.3, then in many similar days, rain would occur about 30% of the time.

• Example: If the probability a smoker dies of cancer is 0.75, then among many such smokers, about 75% would die of cancer.

Note: In practice, the observed frequency may not exactly match the theoretical probability, especially for a small number of trials. As the number of trials increases, the observed frequency tends to get closer to the probability.

Connection to Relative Frequency

The formula $P = \frac{f}{N}$ is the same as the formula for relative frequency from descriptive statistics. When all outcomes are equally likely, probability and relative frequency are conceptually the same.