Search

Analysis of various optimization methods and their application in training common machine learning models. Gradient descent, line search, Newton’s method, and their implementation in standard PyTorch optim interface. Specific topics include: quasi-Newton method, (stochastic) gradient descent, and momentum. Coding these methods using the optimizer class interface in PyTorch, and competing on Kaggle with your classmates.

This course studies modern methods of applied mathematics suitable for first-year graduate students in Mathematics and Applied Mathematics, such as Fourier analysis and the spectral theory of compact operators. These methods, which are often regarded as belonging to the realm of functional analysis, have been motivated most specifically in connection with the study of ordinary differential equations, partial differential equations, and integral equations.