ENGINEERING TRIPOS PART IIB 2012/2013

Module 4F10 - Statistical Pattern Processing

AIMS

This module aims to describe the basic concepts of statistical pattern processing and some of the current techniques used in pattern classification.

LECTURE SYLLABUS

• Introduction (1L)
  Statistical pattern proecessing, Bayesian decision theory, generalisation.
  
  
• Multivariate Gaussian Distributors and Decision Boundaries (1L)
  Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, classification cost, ROC curves.
  
  
• Gaussian Mixture Models (1L)
  Mixture models, parameter estimation, EM for discrete latent variables.
  
  
• Expectation Maximisation (1L)
  Latent variables both continuous and discrete, proof of EM, factor analysis.
  
  
• Linear Classifiers (1L)
  Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis, limitations.
  
  
• Multi-Layer Perceptrons (2L)
  Basic structure, posterior modelling, regression, error back propogation, learning rates, second order optimisation methods.
  
  
• Support Vector Machines (2L)
  Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions.
  
  
• Gaussian Processes (2L)
  Gaussian processes, covariance functions, non-linear regresion, Gaussian processes for classification.
  
  
• Classification and Regression Trees (1L)
  Decision trees, query selection, multivariate decision trees.
  
  
• Non-Parametric Techniques (1L)
  Parzen windows, K-nearest neighbours, nearest neighbour rule.
  
  
• Application: Speaker Verification and Identification (1L)
  Speaker recognition/verification task, GMMs and MAP adaptation, SVM-based verification.
  
  

OBJECTIVES

On completion of the module students should:

• Understand the basic principles of pattern classification;
• Understand Expectation-Maximisation as a general optimisation technique;
• Understand current classification schemes such as Support Vector Machines and Gaussian Processes;
• Be able to apply pattern processing techniques to practical applications

REFERENCES

Please see the Booklist for Group F Courses for references for this module.

Last updated: May 2012