Probability Theory II - Fall 2025

Welcome to my course on Probability Theory for third-year students of Bachelor’s Programs and MSc students in Advanced Combinatorics at MIPT.

This is the second part of the course. We will devote this semester to the study of convergence of random variables, limit theorems, gaussian vectors and Linear regression models.

1. Convergence of random variables. Examples for each type. Alexandrov’s theorem. Theorem on the relation between different types of convergence.
   Counterexamples on the relation between different types of convergence.
2. Characteristic functions of random vectors. Properties of characteristic funtions. Uniqueness theorem.
   Examples for some distributions: Bernoulli, Binomial, Geometric, Poisson, Normal, Gamma, Exponential.
3. Theorem on independence of random variables using characteristic functions.Theorem on the series expansion of characteristic functions. Inversion formula. Levy’s continuity theorem.
4. Central Limit theorem. WLLN in Khinchin’s form. Kolmogorov’s SLLN. Law of Iterated logarithm.
5. Types of convergence of random vectors. Theorem of convergence of components of random vectors.
   Convergence of a function of a random vector. Slutsky’s theorem. CLT for vectors. Delta Method.
6. Empirical distribution and empirical distribution function. Justification of the main problem of mathematical statistics and the Glivenko–Cantelli theorem. Kolmogorov’s theorem (without proof).
7. Statistics and estimators. Examples of statistics: sample characteristics, order statistics. Main properties of estimators: unbiasedness, consistency, strong consistency, asymptotic normality. Examples.
8. Inheritance of consistency and strong consistency under continuous transformations. Lemma on inheritance of asymptotic normality. Sample quantiles. Asymptotic normality of sample quantiles and the sample median (without proof). Examples.
9. Method of moments, consistency and asymptotic normality (without proof) of the method of moments estimator.
10. Maximum likelihood method. Extremal property of the likelihood function. Consistency of the maximum likelihood estimator.
11. Confidence intervals. Central statistic method. Asymptotic confidence intervals. Construction of asymptotic confidence intervals using asymptotically normal estimators. Examples.
12. Linear regression model. Least squares estimator and its main properties. Theorem on the best linear estimator (without proof). Unbiased estimator for the measurement error variance $\sigma^2$.
13. Gaussian linear model. Sufficient statistics in the Gaussian linear model. Best unbiased estimators of parameters in the Gaussian linear model and their distributions.
14. Chi-squared, Student’s t, and Fisher’s F distributions and their properties. Theorem on orthogonal decompositions of a Gaussian vector (without proof). Confidence intervals for parameters of the Gaussian linear model.
15. Statistical hypothesis testing: general principles and basic concepts (critical region, significance level, alternatives, Type I and Type II errors, power function). Comparison of tests: most powerful and uniformly most powerful tests. Unbiasedness and consistency of a statistical test, p-value.
16. F-test for testing linear hypotheses in the Gaussian linear model. Example with two Gaussian samples differing in mean: testing the hypothesis of homogeneity.

At the end of this course, you will get a grade from 0 to 10 (you need at least 3 to pass) according to the following parameters:

• A:= Max of 1 point for class attendance (1 if you missed no more than 1 class. And o,5 if you missed no more than 2)
• P:= Max of 2 points for participation in class by solving homework on the board.
• T:= Max of 6 points for Test 1 + Test 2
• E:= Max of 4 points for final exam (зачет) on theory

Final grade = A+P+T+E-2

IMPORTANT: You must get at least 1.5 points in the final exam on theory in order to pass the course.

The theoretical exam and tests 1 and 2 are closed-book, that is, you are not allowed to use any material.

The number of points you get for each activity, is either an integer x or x+0.5.

If your final grade (after the final exam) is not an integer (z+0,5), you can solve an extra problem to raise your grade to z+1. Otherwise you get just z.

If you have <= 2 points for A+T, then you have a solve problems in the exam. You need a minimum number of points of the solution of these problems to get your theoretical questions and continue with the exam.

On the other hand, in the retake you have to solve problems independently from the number of points you have for T.

• Probability (Graduate Texts in Mathematics) 2nd Edition - Albert N. Shiryaev.
• Introduction to probability for Data Science - Stanley H. Shan. [download]
• Probability and Statistics for Data Science - Carlos Fernandez-Granda.
• Introduction To Probability - Joseph K. Blitzstein, Jessica Hwang.
• Basics of Modern Mathematical Statistics - Vladimir Spokoiny
• Мера и интеграл, Дьяченко М.И.
• Курс теории вероятностей и математической статистики, Севастьянов Б.A.
• Курс теории вероятностей, Чистяков В.П.

• Short lectures on measure theory: [playlist]
• Short lectures on Probability Theory [playlist]
• Lecture in Statistics from MIT [playlist]
• Probability theory course IMPA [playlist]
• Probability theory course Harvard University [playlist]
• Interactive videos on probability from 3Blue1Brown [video]
• Lectures in introduction to probability (in russian) [playlist]
• [link]