Preface
1 Introduction to Probability 1
1.1 Introduction: Why Study Probability? 1
1.2 The Different Kinds of Probability 2
Probability as Intuition 2
Probability as the Ratio of Favorable to Total Outcomes (Classical Theory) 3
Probability as a Measure of Frequency of Occurrence 4
Probability Based on an Axiomatic Theory 5
1.3 Misuses, Miscalculations, and Paradoxes in Probability 7
1.4 Sets, Fields, and Events 8
Examples of Sample Spaces 8
1.5 Axiomatic Definition of Probability 15
1.6 Joint, Conditional, and Total Probabilities; Independence 20
Compound Experiments 23
1.7 Bayes’ Theorem and Applications 35
1.8 Combinatorics 38
Occupancy Problems 42
Extensions and Applications 46
1.9 Bernoulli Trials–Binomial and Multinomial Probability Laws 48
Multinomial Probability Law 54
1.10 Asymptotic Behavior of the Binomial Law: The Poisson Law 57
1.11 Normal Approximation to the Binomial Law 63
Summary 65
Problems 66
References 77
2 Random Variables 79
2.1 Introduction 79
2.2 Definition of a Random Variable 80
2.3 Cumulative Distribution Function 83
Properties of FX(x) 84
Computation of FX(x) 85
2.4 Probability Density Function (pdf) 88
Four Other Common Density Functions 95
More Advanced Density Functions 97
2.5 Continuous, Discrete, and Mixed Random Variables 100
Some Common Discrete Random Variables 102
2.6 Conditional and Joint Distributions and Densities 107
Properties of Joint CDF FXY (x, y) 118
2.7 Failure Rates 137
Summary 141
Problems 141
References 149
Additional Reading 149
3 Functions of Random Variables 151
3.1 Introduction 151
Functions of a Random Variable (FRV): Several Views 154
3.2 Solving Problems of the Type Y = g(X) 155
General Formula of Determining the pdf of Y = g(X) 166
3.3 Solving Problems of the Type Z = g(X, Y ) 171
3.4 Solving Problems of the Type V = g(X, Y ), W = h(X, Y ) 193
Fundamental Problem 193
Obtaining fVW Directly from fXY 196
3.5 Additional Examples 200
Summary 205
Problems 206
References 214
Additional Reading 214
4 Expectation and Moments 215
4.1 Expected Value of a Random Variable 215
On the Validity of Equation 4.1-8 218
4.2 Conditional Expectations 232
Conditional Expectation as a Random Variable 239
4.3 Moments of Random Variables 242
Joint Moments 246
Properties of Uncorrelated Random Variables 248
Jointly Gaussian Random Variables 251
4.4 Chebyshev and Schwarz Inequalities 255
Markov Inequality 257
The Schwarz Inequality 258
4.5 Moment-Generating Functions 261
4.6 Chernoff Bound 264
4.7 Characteristic Functions 266
Joint Characteristic Functions 273
The Central Limit Theorem 276
4.8 Additional Examples 281
Summary 283
Problems 284
References 293
Additional Reading 294
5 Random Vectors 295
5.1 Joint Distribution and Densities 295
5.2 Multiple Transformation of Random Variables 299
5.3 Ordered Random Variables 302
Distribution of area random variables 305
5.4 Expectation Vectors and Covariance Matrices 311
5.5 Properties of Covariance Matrices 314
Whitening Transformation 318
5.6 The Multidimensional Gaussian (Normal) Law 319
5.7 Characteristic Functions of Random Vectors 328
Properties of CF of Random Vectors 330
The Characteristic
