CSC466-2305 Numerical Optimization

The first meeting of CSC466-2305 is Tuesday, January 9, 2024, 15:00-17:00, Room MP 134.

• Aims
• Outline
• References
• Prerequisites
• Schedule for Winter 2024
• Marking scheme for Winter 2024
• Announcements for current students

• Review the basic concepts in numerical optimization.
• Introduce numerical methods for solving continuous (mostly unconstrained) optimization problems.
• Evaluate numerical optimization methods with respect to their accuracy, convergence, time and memory complexities.
• Develop and practice computer skills in implementing numerical optimization methods efficiently on the computer, as well as skills to judge the correctness of numerical results.
• Use high level software for studying numerical optimization methods.

• Introduction, continuous vs discrete, unconstrained vs constrained, global and local minima, general considerations
• (1D optimization) Golden section search, Newton's
• Line search methods
• Trust region methods
• Conjugate gradient methods
• Quasi-Newton'smethods
• Approximating derivatives, the gradient, the Jacobian, the Hessian

References

Numerical Optimization
Jorge Nocedal and Steven Wright	Numerical Optimization	Springer NY, 2006
General Numerical Analysis
Michael Heath	Scientific Computing: an introductory survey	SIAM 2018 or McGraw-Hill Inc. 2002+

Prerequisites

• Courses: a numerical methods course (e.g. csc336), a multivariate calculus course (e.g. mat235, mat237), a linear algebra course (e.g. mat221, mat223, mat240).
• General: Ability to handle notation and to do algebraic manipulation. Fluency in matrix and vector manipulation.
• Calculus: Differentiation and integration of polynomial, trigonometric, exponential, logarithmic and rational functions, continuity, limits, graphs of functions, Taylor series, Rolle's theorem, mean-value theorem, de l' Hospital's rule, partial derivatives, gradient, multi-dimensional Taylor series
• Linear Algebra: Matrix and vector addition and multiplication, elementary row operations, linear (in)dependence, inverse matrix, banded and sparse matrices, properties of matrices, matrix norms, condition numbers, eigenvalues, eigenvectors, various decompositions, etc
• Programming: strong coding abilities in some programming language, such as MATLAB, C or FORTRAN.
• Other Mathematics: induction.

Schedule for Winter 2024

Lectures	Tuesday 3-5 PM	Room MP 134
Tutorial	Thursday 3-4 PM	Room MP 134
Office Hours	Wednesday 1-2 PM	Room BA 4226 or online

Tutorial times will be used for lectures so that we go at a slower pace.
We may not use all tutorial times.

Tentative marking scheme for Winter 2024

Problem set 1	20%
Problem set 2	22.5%
Problem set 3	22.5%
Term test 1	35%

The problem sets include substantial computer work.