Material for the Final

General Concepts and Methods

Chapter 1, Lab 1: Spaces, events, basic counting, exponential approximation
Chapter 2: Addition and multiplication rules; conditioning and updating
Chapter 5: Unions and intersections of several events; exact probabilities and bounds
Section 9.1: Probabilities by conditioning and recursion (discrete)
Section 20.2: Probabilities by conditioning on a continuous variable
Section 4.5, 20.2: Independence

Chapter 3: Intro; equality versus equality in distribution
Chapter 4, Lab 2: Joint, marginals, conditionals, independence (discrete case), total variation distance
Section 5.3, 5.4: Random permutations and symmetry
Section 15.1, 15.2, Lab 6: Density
Section 6.1, 15.1, 16.3, Lab 6: CDF and inverse CDF
Chapter 16, Section 23.3, Lab 6, Lab 8: Density of a transformation
Chapter 17: Joint, marginal, and conditional densities; independence
Chapter 14, Chapter 19: Distribution of sum; probability generating function
Chapter 14, Section 15.3, 19.3: Central Limit Theorem

Chapter 8, Lab 3B: The crucial properties (discrete case) including method of indicators, expectations of functions, tail sum formula (see also geometric distribution)
Section 12.3, 19.4: Tail bounds: Markov, Chebyshev, Chernoff
Section 9.2, 9.3: Expectation by conditioning
Section 15.3, 17.1, 20.2: Expectation using densities and joint densities, and by conditioning on a continuous variable
Section 19.2: Moment generating function

Chapter 12: Definition and basic properties; linear transformations
Chapter 13, Lab 5: Covariance; variance of a sum
Section 24.2, Homework 7, Lab 5: Correlation and its properties
Section 22.3, 22.4: Variance by conditioning, mixtures
Section 23.1: Mean vector and covariance matrix of a random vector; linear transformations

Section 8.4: Unbiased estimators
Section 14.5, 14.6: IID sample mean; confidence interval for population mean
Homework 7, Homework 14: Unbiased estimator of a population variance; independence of normal sample mean and sample variance
Section 20.1: Maximum likelihood estimate
Section 20.3: Posterior density, MAP estimate
Section 12.2, 22.1, 22.2: Expectation and conditional expectation as least squares predictors
Section 24.1, 25.4: Least squares linear predictor

Section 8.1, 12.1: Uniform on \(1, 2, ..., n\)
Section 8.2, 12.1, 13.4: Bernoulli (indicator)
Section 6.1, 6.2, 6.3, 6.5, Lab 2, Chapter 7, 8.5, 13.3, Lab 5, 14.3, 19.2, Chapter 21: Binomial and multinomial
Section 5.4, 6.4, 8.5, 13.4: Hypergeometric
Section 5.3, 6.6, Lab 2, Chapter 7, Section 8.2, 8.3, 12.1, 19.2, Lab 7: Poisson
Section 8.2, 9.3, 22.4: Geometric

Section 14.3, 14.4: CLT; Normal cdf and inverse cdf
Sections 14.6: Normal confidence intervals
Section 16.1: Normal densities
Section 18.1, 18.2, 18.4: Independent normal variables, linear combinations, squares, Rayleigh, chi-squared
Section 19.3: Normal MGF, sums, CLT
Section 24.2, 24.3, Lab 8: Bivariate normal, linear combinations, independence, regression
Chapter 23, Section 25.4: Multivariate normal, linear combinations, independence, regression

Section 15.4, 16.1, 16.2.3, 18.1: Exponential and scaling; square root and the Rayleigh
Homework 8: Gamma function, gamma density, mean, variance
Section 18.3, 18.4: Gamma and scaling; chi-squared
Section 19.2: Sums of independent gammas with the same rate
Lab 7: Waiting times of arrivals in a Poisson process
Homework 14: The chi-squared and the normal sample variance

Section 5.2 (general inclusion-exclusion formula)
Chapters 10, 11 (Markov Chains)
Section 12.4 (Heavy-tailed distributions)
Section 16.4.1 (Two-to-one function change of variable for densities)
Section 19.3.4 (“Proof” of the Central Limit Theorem)
Section 21.3 (Long-run proportion of heads for a random coin)
Sections 25.1, 25.2, 25.3 (general best linear predictor based on multiple predictors)