Statistical and Critical Thinking: Foundations and Pitfalls in Data Collection

Statistical and Critical Thinking

Introduction

Statistical and critical thinking form the basis of sound data analysis in statistics. Understanding how data is collected, interpreted, and the potential pitfalls in analysis is essential for drawing valid conclusions.

Definitions and Key Concepts

Data, Statistics, Population, and Sample

• Data: Collections of observations, such as measurements, genders, or survey responses. A single data value is called a datum.

• Statistics: The science of planning studies and experiments, obtaining data, and organizing, summarizing, presenting, analyzing, and interpreting those data to draw conclusions.

• Population: The complete collection of all measurements or data being considered.

• Sample: A subcollection of members selected from a population.

• Census: The collection of data from every member of the population.

Example

In a survey of 410 human resource professionals, 148 said job candidates were disqualified because of information found on social media postings. Population: All human resource professionals Sample: The 410 human resource professionals who were surveyed

Collecting Sample Data

Sampling Methods

• Voluntary Response Sample (Self-Selected Sample): Respondents themselves decide whether to be included. Common examples include internet polls, mail-in polls, and telephone call-in polls.

• Potential Issues: Voluntary response samples often lead to bias, as those with strong opinions are more likely to respond.

Example

ABC television asked viewers to call with their opinion about the United Nations headquarters. 176,671 of 186,000 respondents said it should be moved. In a separate survey, 500 randomly selected respondents were surveyed, and only 38% agreed. The random sample is more reliable due to superior sampling methods.

Statistical vs. Practical Significance

Understanding Significance

• Statistical Significance: A result is very unlikely to occur by chance.

• Practical Significance: A result does not make enough of a difference to justify its use.

Example

Among 100,000 births, 50,300 females is statistically significant (less than 3% chance), but not practically significant.

Example

In a weight loss program, 21 subjects lost an average of 2.1 kg. This is statistically significant, but for many, a loss of only 2.1 kg is not worth the time, cost, and effort, so it is not practically significant.

Analyzing Data: Potential Pitfalls

Common Pitfalls in Data Analysis

• Misleading Conclusions: Statements should be justified by statistical analysis, not by assumptions or physical evidence alone.

• Reported vs. Measured Data: It is better to measure data directly rather than rely on self-reported values, which may be inaccurate due to rounding or bias.

• Loaded Questions: Survey questions can be worded to elicit a desired response, leading to biased results.

• Order of Questions: The sequence of survey questions can influence responses and lead to different results.

• Nonresponse: Occurs when someone refuses to respond or is unavailable, which can bias results.

• Low Response Rates: Surveys with low response rates are less reliable and may not represent the population accurately.

• Percentages: Misleading or unclear percentages can distort the interpretation of results. Percentages over 100% are often unjustified.

Classroom Example

In a survey of 1046 adults, 70% said they always wash their hands in public restrooms. a. Identify the sample and the population. b. Better results would be obtained by observing hand washing rather than asking about it, to avoid self-reporting bias.

Summary Table: Key Terms

Term	Definition
Population	Complete collection of all measurements/data being considered
Sample	Subcollection of members selected from a population
Statistic	Science of planning, collecting, and analyzing data
Voluntary Response Sample	Sample where respondents choose to participate
Statistical Significance	Result unlikely to occur by chance
Practical Significance	Result makes a meaningful difference in practice

Formulas

Basic Probability (for Statistical Significance)

Probability of an event occurring by chance:

$P(E) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$

Conclusion

Understanding the foundations of statistical and critical thinking is essential for collecting, analyzing, and interpreting data accurately. Awareness of potential pitfalls helps ensure the reliability and validity of statistical conclusions.