ENGINEERING TRIPOS PART IIA –
2011-2012 (NB. not offered 2010-11)
Module 3M1 – Mathematical methods
|
Timing:
|
Lent
|
|
Prerequisites:
|
none
|
|
Structure:
|
16 Lectures + coursework
|
AIMS
This module aims to teach some mathematical techniques that have wide applicability to many areas of engineering. Linear Algebra provides important mathematical tools that are not only essential to solve many technical and computational problems, but also help in obtaining a deeper understanding of many areas of engineering. Optimization methods are routinely used in almost of every branch of engineering, especially in the context of design. Stochastic (random) processes are important in fields such as signal and image processing, data analysis etc.
Syllabus
1. Linear Algebra (6L)
- Least squares solution of Ax = b for an m x n matrix with n independent columns: Gram-Schmidt orthogonalization, QR decomposition
- Solution of Ax = ??x, eigenvectors and eigenvalues
- Singular Value Decomposition
2. Optimization (6L)
- Introduction: Formulation of optimization problems; conditions for local and global minimum in one, two and multi-dimensional problems
- Unconstrained Optimization: gradient search methods (Steepest Descent, Newton’s Method, Conjugate Gradient Method)
- Linear programming (Simplex Method)
- Constrained Optimization: Lagrange and Kuhn-Tucker multipliers; penalty and barrier functions
- Network optimization
- Principal component analysis
3. Stochastic Processes (4L)
- Definition of a stochastic process, Markov assumption (with examples), the Chapman-Kolmogorov (CK) equation, conversion of a particular CK integral equation into a differential equation (for the case of Brownian motion)
- The general Fokker-Planck equation with particular examples (Brownian motion, Ornstein-Uhlenbeck process)
- Numerical Bayesian methods, comparison of optimization methods with simulation-based approaches
- Gibbs sampler, Metropolis Hastings, Importance sampling with applications
OBJECTIVES
By the end of the course students should:
Linear Algebra
- Be able to calculate the least squares solution of a set of linear equations;
- Be able to orthogonalize a set of vectors, complete the QR factorisation of a matrix, and be able to use this to find the least squares solution of a set of linear equations;
- Be able to find the eigenvalues and eigenvectors of a matrix, and complete the A = SAS–1 or A = QNQT factorisations as appropriate;
- Be able to find the SVD of a matrix, and to understand how this can be used to calculate the rank of the matrix, and to provide a basis for the each of its fundamental subspaces
Optimization
- Understand the concepts of local and global minimum and the conditions for which a global minimum can be obtained;
- Understand the algorithms of the different gradient search methods;
- Be able to solve unconstrained problems using appropriate search methods;
- Be able to select an appropriate optimization method for a specific problem;
- Be able to solve a constrained linear and non-linear problem using an appropriately selected technique;
- Be able to solve network optimization problems;
- Be able to apply principal component analysis to reduce the dimensionality of an optimization problem and/or to improve the solution representation.
Stochastic Processes
Understand the definitions and application areas of Stochastic Processes;
Understand the principle of Markov Chains;
Be able to implement various sampling schemes to enable parameters of stochastic processes to be estimated.
REFERENCES
Please see the Booklist for Part IIA Courses for references for this module.
