Mathematics for Artificial Intelligence - Fall 2023

Instructor:

Teaching Assistants:

References:

Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning. Cambridge University Press, 2020.
Goodfellow, Ian, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning. Vol. 1, no. 2. Cambridge: MIT press, 2016.
Strang, Gilbert. Linear algebra and learning from data. Cambridge: Wellesley-Cambridge Press, 2019.
Heath, Michael T. Scientific Computing: An Introductory Survey, Revised Second Edition. Society for Industrial and Applied Mathematics, 2018.

Schedule

Session	Topic	Videos	Slides	Labs	Assignments
PART I - Linear Algebra and Matrix Analysis
Session1	Introduction, Motivation, and Applications	Aparat Youtube	Math4AI - Lecture 1.pdf	Lab 0 Lab 1 Lab 1 - Files	Assignment 1
Session2	Vectors and Vector Spaces, Linear Combination, Span	Aparat Youtube	Math4AI - Lecture 2.pdf
Session3	Linear Independence, Basis Vectors, Coordinates, Linear Subspace, Column and Row Spaces, Linear Transformations	Aparat Youtube	Math4AI - Lecture 3.pdf	Lab 2 Lab 2 - Files
Session4	Matrix Multiplications, dot product, inner product space, Outer Product, Matrix Rank	Aparat Youtube	Math4AI - Lecture 4.pdf
Session5	Linear Equations, Singular and Non-singular matrices, the Inverse Matrix	Aparat Youtube	Math4AI - Lecture 5.pdf	Lab 3 Lab 3 - Files
Session6	LU decomposition, Null space, all solutions to General Linear Equations	Aparat Youtube	Math4AI - Lecture 6.pdf
Session7	Homogeneous Equations, Orthogonal Projection	Aparat Youtube	Math4AI - Lecture 7.pdf	Lab 4 Lab 4 - Files
Session8	Orthonormal Basis, Orthogonal Matrices, QR decomposition, noisy measurements, Least Squares	Aparat Youtube	Math4AI - Lecture 8.pdf
Session9	Determinant, Intro to Eigenvalues	Aparat Youtube	Math4AI - Lecture 9.pdf
Session10	Eigenvalues and Eigenvectors, Algebraic Multiplicity, Eigenspaces, Geometric Multiplicity	Aparat Youtube	Math4AI - Lecture 10.pdf		Assignment 2
Session11	Complex Matrices, Diagonalization, Eigendecomposition of Symmetric and Hermitian Matrices, Positive Definite Matrices	Aparat Youtube	Math4AI - Lecture 11.pdf
Session12	Properties of Positive Definite Matrices, Cholesky Decomposition, Introduction to Singular Value Decomposition, Skinny SVD, Compact SVD	Aparat Youtube	Math4AI - Lecture 12.pdf
Session13	Noisey Homogeneous Equations, Matrix Norms, Low-rank Approximation and Eckart–Young–Mirsky theorem	Aparat Youtube	Math4AI - Lecture 13.pdf	Lab 5 Lab 5 - Files
Session14	Eigen decomposition and optimization, SVD and optimization, Noisy Homogeneous Equations, Affine spaces, affine maps, nonlinear functions, linearization and derivatives	Aparat Youtube	Math4AI - Lecture 14.pdf
PART II - Multivariate Calculus
Session15	vector-valued and matix-valued functions of one variable, differentiability classes, multivariate functions, directional derivatives, the gradient vector, linearization of multivriate functions	Aparat Youtube	Math4AI - Lecture 15.pdf
Session16	Examples of calculating gradient, Definition of differentiability, calculate gradients with inner product trick	Aparat Youtube	Math4AI - Lecture 16.pdf		Assignment 3
Session17	Functions of matrices, linear regression, vector valued multivariate functions, Jacobian	Aparat Youtube	Math4AI - Lecture 17.pdf
Session18	Jacobian Matrix, Chain Rule, Automatic Differentiation, Backpropagation	Aparat Youtube	Math4AI - Lecture 18.pdf
Session19	Second Derivatives, Multilinear maps and Tensors, the Hessian Matrix, Quadratic Functions, Quadratic Approximation, Taylor Series	Aparat Youtube	Math4AI - Lecture 19.pdf		Assignment 4
PART III - Basic Probability and Statistics
Session20	Modeling Uncertainty, Random Variables, Probability Mass and Density Functions, Operations on Random Variables, Cumulative Distribution	Aparat Youtube	Math4AI - Lecture 20.pdf
Session21	Joint distribution, Marginal Distribution, Conditional Distribution, Probabilistic Modeling, Generative vs Discriminative Models	Aparat Youtube	Math4AI - Lecture 21.pdf
Session22	Probabilistic Independence, Conditional Independence,	Aparat Youtube	Math4AI - Lecture 22.pdf
Session23	Statistical Measures, mean, median, mode, variance, standard deviation, covariance matrix	Aparat Youtube	Math4AI - Lecture 23.pdf
Session24	Commom Probability Distributions, Bernoulli distribution, Binomial distribution, Poisson Distribution, Unifrom Distribution, Exponential Distribution, Gaussian Distribution, Multivariate Gaussian Distribution.	Aparat	Math4AI - Lecture 24.pdf
Session25	More on Gaussian Distribution, Introduction to sampling, Monte Carlo methods, Generating Samples, Pseudo Random Numbers, roulette wheel selection	Aparat Youtube
Session26	Sampling Continuous Distributions, Sampling Multivariable Distributions, Sampling from Multivariate Gaussian Distribution		Math4AI - Lecture 26.pdf
PART IV - Optimization
Session27	Statistical Estimation, Maximum Likelihood Solution, Introduction to Optimization		Math4AI - Lecture 27.pdf
Session28	Gradient Descent, SGD, Momentum, Quadratic Approximation, Newton's method		Math4AI - Lecture 28.pdf
Session29	Quasi-Newthon methods, Nonlinear Least Squares, Gauss-Newton, Levenberg-Marquardt, Constrained Optimization, Equality Constraints, Lagrange Multipliers		Math4AI - Lecture 29.pdf
Session30	Multiple Equality Constraints, Convex Sets, Convex Hull, Supporting Hyperplane		Math4AI - Lecture 30.pdf
Session31	Convex Functions and their properties		Math4AI - Lecture 31.pdf
Session32	Inequality Constraints, Primal and Dual Problems, Strong and Weak Convexity		Math4AI - Lecture 32.pdf
Session33	Convex Optimization, Duality examples, Suboptimality Check, KKT conditions		Math4AI - Lecture 33.pdf