Department of Engineering - Annual Report 1998/99

Information Engineering

CONTROL SYSTEMS
Robust Control Theory and Design
Predictive Control and Identification
Robust Control Applications
Control Robotics

Signal Processing and Communications Laboratory

SPEECH, VISION AND ROBOTICS
Speech Processing
Neural Networks
Medical Applications of Information Engineering
Three Dimensional Ultrasound Imaging
Computer Vision and Robotics

LABORATORY FOR COMMUNICATIONS ENGINEERING

Research in the Information Engineering Division concerns the generation, distribution, analysis and use of information in engineering systems, with the principal interests in system level aspects and processing of signals in contrast to innovations in devices. The information considered include measurements in a flight control system, speech input to machines, vision, music and image data, and data in ubiquitous communication systems. The generation and transmission of data is expanding at an enormous rate, as is the computer processing power to analyse it; however huge challenges remain to exploit the opportunities of increasingly complex systems.

Control Systems

The research work of the Control Group ranges from theoretical studies which focus on the fundamental limitations in feedback systems and methodologies for control system design, through to very applied work in collaboration with outside organisations.

Robust Control Theory and Design

An approach to the study of nonlinear systems was investigated by considering the class of systems with a single repeated nonlinearity as in, for example, certain neural networks or as an approximation to more general systems. For this class of systems bounds on the system gain, model reduction errors and controller design have been derived(K22,K23).

The robustness of linear periodically time-varying feedback systems to gap metric uncertainty has been studied(K17). Results which show that the gap metric captures the important differences between open-loop systems from the perspective of closed-loop performance have been established in an abstract setting for linear, possible infinite-dimensional and time-varying, systems(K16). The study of sampled-data control systems continues with on-going work in the direction of optimal discretisation.

Research effort has continued on input-output approaches to control systems. A generalisation of the gap metric for uncertainty studies in nonlinear feedback systems was further studied. A generalisation of H-infinity control for the class of piecewise linear systems was considered in the context of incremental gain analysis and the use of linear matrix inequalities(K131).

The n-gap metric, which can be used for the investigation of robustness properties of linear feedback systems, was extended to nonlinear systems(K158). In particular, numerical methods were found for measuring the distance between a nonlinear system and a linearisation of that system. If this distance is small then a feedback controller designed for the linearisation can be guaranteed to be also effective with the original nonlinear system. This captures the fact that a large class of nonlinear systems can be adequately controlled by linear feedback. In addition, techniques using the n-gap for the simplification of uncertain system representations were further developed(K157).

In previous work, methods of anti-windup compensation had been developed for stable linear systems with actuator constraints that are guaranteed to give global stability with any nominal stabilizing controller. Some of this was extended(K41) to give local stability results for unstable linear systems, which cannot be globally stabilized in the presence of actuator saturation.

Predictive Control and Identification

Research has continued on the use of predictive control methods to obtain fault-tolerant control. A simulation study has investigated fault-tolerant flight control in the presence of control surface failures(K79). A solution has been given to the `COSY benchmark' fault-tolerant ship control problem(K87). The tuning of predictive controllers using LQG/LTR theory has been investigated(K132). It has been shown that predictive controllers have a certain degree of inherent fault-tolerance in the presence of actuator failures, when there is some redundancy of actuation(K105).

Research on subspace methods has also continued. A new algorithm, which includes preliminary estimation of Markov parameters, was proposed, its benefit being reduced bias when the measured input signal is not white(K26). As with any system identification method, subspace methods need input signals to be sufficiently rich to excite all the system dynamics, in order to be able to identify it correctly. New conditions for `informative experiments', namely those rich enough for this to be possible, have been established(K24); these conditions are more precise than previously-known conditions. A novel geometric interpretation of subspace methods has been given(K25). All these results have been submitted for journal publication. A new research project, which aims to extend subspace methods to bilinear systems, has been started.

New techniques for the identification of linear systems for the purposes of control system design were developed(K43,K44,K142). These contributions are part of a continuing effort to develop a comprehensive framework for control oriented system identification which is tailored to the needs and capabilities of robust control design methods, in particular H¥.

Robust Control Applications

A very successful test flight of the DERA VAAC Harrier aircraft was achieved this year at Boscombe Down with a controller designed in the control group using new methods for robust controller design funded by DERA, Bedford. The method describes the aircraft dynamics by linear models that vary with parameters of aircraft speed and pitch angle and synthesises a control law that also varies with these parameters and has certain guaranteed robustness properties.

The expertise of the group in flight control has been recognised with a special session at the AIAA, Conference on Guidance and Control, based around techniques originating in the group(K18,K116,K117,K118).

There is increasing involvement in automotive engine control problems. This is in collaboration with the Internal Combustion Engines activity reported in section A and sponsored by EPSRC and the Ford Motor Company. On the control side a number of problems are being addressed principally motivated by emissions control under transient conditions. These experimental studies take place in the group's dynamic dynamometers taking advantage of the high speed emissions instrumentation described in section A. Idle speed control, air/fuel ratio control and catalyst modelling for control are all under current investigation. In addition a supporting dynamometer control study is reported(K109).

Control Robotics

Work continued in the general area of robotics and control(K92).

Signal Processing and Communications Laboratory

This year work has continued on a study of Non-Gaussian noise processes, alpha-stable in particular, with a view of how to incorporate such distributions within a Bayesian Inference framework. This has been approached using approximate representations of alpha-stable distributions using mixtures of Gaussian density functions(K91). New methods have been developed for inference in the presence of alpha-stable noise, making use of a conditionally Gaussian representation of the symmetric stable density(K62,K63,K65,K66). Application areas have included Spatial Beamforming for mobile communications and the removal of impulsive noise from audio. A project has started on the deconvolution of nuclear spectroscopic data, using reversible jump Monte-Carlo Markov Chain (MCMC) methods and the initial results appear very encouraging.

New theoretical results on the convergence of optimisation using simulated annealing have been reported(K4).

An important problem in statistical signal processing is that of choosing the most appropriate signal models from within a class of models; progress has been made in this area and the results reported(K7,K8,K9,K125).

Progress has been made in the investigation of the potential of both non-stationary(K78) and cyclo-stationary signal processing. A class of blind deconvolution problems can be expressed in a non-stationary environment and Bayesian methodology has been applied to blind identification of transfer functions such as room acoustics(K78). Novel Time-Frequency methods have been developed for the analysis of non-stationary signals(K37) and these methods have been applied to biological multi-channel data in order to quantify the degree of `coherence' between non-stationary signals. Work has continued with Bayesian approaches to cyclostationary processes both from a theoretical point of view and with applications to the demodulation of BPSK signals(K106,K107,K108).

Work in blind source separation has continued successfully, with further developments in the decorrelation based methods(K3), and new approaches based on model-based Bayesian techniques in a project sponsored by AT&T Labs Cambridge(K64). Work in Markov chain Monte Carlo (MCMC) methods has continued with the development of a novel technique for maximisation of marginal distributions(K128), methods for spectral analysis(K5,K7) blind deconvolution(K19,K49,K50) and neural networks(K71). In a one year EPSRC project, advances have been made in the correction of distortion, quantisation and clipping in digital audio signals using MCMC(K149,K150,K151).

Image segmentation for both black and white and coloured images has continued and the results have been published(K12,K86).

A focus of research in the laboratory this year has been sequential simulation methodology, with several PhD students working in the area and the commencement of a major new EPSRC project; methodological advances have been made. Techniques have been developed for fixed-lag simulation and digital communications applications(K32,K33). Other areas being addressed include the application of particle filters to demodulation of signals in fading communication channels, multi-sensor data fusion, tracking of radar signals(K51,K104) and denoising of non-stationary signals(K10). A 3 day workshop on sequential MCMC methods was organized in December 1999 by Group members and it attracted an international audience of 60 participants.

Further results of work on efficient multiplier and digital filter design were presented in two papers at the 1999 European Conference on Circuit Theory and Design(K45,K46).

A paper(K95) reporting results on variable rate adaptive channel coding was awarded the Hong Kong Institution of Engineers Transactions Prize in 1999.

Work is continuing on time-frequency analysis applied to angle-of-arrival tracking from antenna arrays(K36), and to musical audio processing. Current projects include the general musical note-time analysis problem, bass line tracking, and singing-voice to MIDI conversion.

Work on image coding has followed 3 main themes: lattice vector- quantised coding, complex wavelet coding and error-resilient coding. First, vector quantisation has been combined with the relatively new, but popular, set partitioning approaches. New methods of dividing lattices into classes have been developed and performance comparable with the best achievable coding schemes has been obtained. Secondly, the complex wavelet coding project is still in its early stages, but some promising approaches to prediction of wavelet coefficients at fine levels from their coarser-level `parents' have been devised. Finally, following on from our earlier development of error-resilient entropy coding (EREC), which allows image coders to be used with error-prone transmission channels, further enhancements to EREC have been developed. These have been applied both to conventional JPEG-like coding systems(K159) and to newer set-partitioning zero-tree systems(K160).

For image analysis and enhancement, a new transform method, known as the dual-tree complex wavelet transform (DT CWT) has been developed(K88). This has the potential for allowing wavelets to be used successfully in a significantly wider range of problems than previously. It has particularly desirable shift-invariant properties(K89), as well as good directional selectivity, and it imitates many of the filtering processes found in the human visual cortex in a computationally efficient form. The new transform is able to analyse textures in perceptually meaningful ways and it has been shown to work better than other standard methods for texture classification(K76). Other application areas that are being investigated using the DT CWT include image denoising, image segmentation, image watermarking, video restoration, and image database searching.

Three-dimensional medical and seismic datasets are also being studied, as the computational efficiency of the DT CWT is particularly useful when the size of the dataset becomes very large.

Work has continued on optical motion capture, with all existing calibration and reconstruction algorithms extended to work optimally with any number of viewing cameras. Work has also begun on tracking multiple camera data using Bayesian and conventional Kalman filtering techniques - these methods use complex models of articulated human motion. The system is being used in various biomechanical applications(K55).

Work on the use of projective geometry in computer vision using the geometric algebra framework has continued(K13,K94,K119,K143). A study of the use of geometric algebra for interpolation of rotations has given a method which is both simple and has the correct invariance properties. The technique is being applied to path planning in robotic arm manipulation and in the analysis of buckling beams. Geometric algebra has also been used to simplify and correct a complex differential topology analysis of rods(H18,K120). Members of CUED coorganised a one-day conference devoted to applications of geometric algebra in engineering held in Aston in January 1999. The conference attracted around 50 participants.

Speech, Vision and Robotics

Research in the SVR group is focussed on applied mathematical modelling and pattern recognition in the areas of speech processing, computer vision and medical imaging. The group is internationally renowned for its work on large vocabulary speech recognition, 3D ultrasound and the recovery of 3D shape from visible surfaces.

Speech Processing

Research in speech processing covers the areas of recognition, synthesis, coding, language modelling, information retrieval and dialog design. Work in the group is organised around externally funded projects and individual PhD projects.

The EPSRC sponsored Multimedia Document Retrieval project (in collaboration with the Computer Laboratory) is focussed on indexing and retrieving audio documents that are created from a stream of audio, such as recordings of news broadcasts. The MDR project is using the HTK broadcast news system to automatically create transcriptions and information retrieval techniques to index the data and return portions relevant to queries. This type of task is generally known as spoken document retrieval (SDR). Recent work has investigated improving the audio segmentation(K74); speaker clustering(K83) and tracking(K80) as well other improvements in adaptation and data normalisation. A version of the transcription system that runs in less than ten times real time and produces high quality transcriptions was created in association with Entropic Ltd(K115) and both this and the full version evaluated in the 1998 DARPA/NIST broadcast news evaluation. The full version gave an error rate not statistically different from the best reported and the fast version gave the lowest error rate by a statistically significant margin for such systems(K165,K166). A first full SDR system was evaluated in the 1998 TREC-7 spoken document retrieval conference(K81,K82) and a number of improvements to the retrieval component have been developed that integrate statistical and knowledge-based techniques(K84,K85). The fast recognition system together with the improved retrieval components were included in the 1999 TREC-8 SDR conference.

Spoken document retrieval systems have also been developed using the Abbot HMM-ANN hybrid recogniser(K1,K2,K129,K156).

The GCHQ sponsored project on the Automatic Recognition of Conversational Telephone speech has continued with good progress being made. The current HTK based system gave the lowest overall error rate on the 1998 Hub5 Conversational Telephone transcription task(K72). More recent work has focussed on improved acoustic modelling using a dynamic selection of hidden Markov models(K73). Related work has investigated vocal tract length normalisation techniques(K168).

Based on the experience gained in DARPA and NIST sponsored evaluations in this and previous years, a review of these programmes and the lessons learnt has been published(K168).

A project concluded on the use of neural networks as acoustic models in speech recognition(K38,K39,K40). Work has continued on alternative acoustic models, notably variants on autoregressive HMMs(K103) and linear dynamical systems for speech analysis(K138,K140,K141).

A project sponsored by Hewlett Packard to investigate low-cost implementation methods for use in hand-held devices terminated. The outcome of the project included several new methods for fast keyword spotting(K57,K90).

Other PhD work on acoustic modelling for speech recognition has included further investigations into discriminative training techniques(K121) and the use of models that depend on speaking rate(K155).

Work in language modelling has looked at the applicability of adaptive language models and improved methods for language modelling evaluation(K34,K35). The use of class based models based on part-of-speech labels(K115) and automatically derived units for modelling of English and Russian have also been investigated(K162).

A new very-low bit rate speech coder has been developed employing segmental models, and the interaction between speech coding and recognition has been investigated(K153,K154).

Work on the use of hidden Markov model techniques for unit selection in speech synthesis has been published(K48). This work generated considerable interest and has led to a number of Universities and companies developing similar methods.

The development of techniques for using speech recognition in computer-assisted language learning has continued. One of the key difficulties in this area is that current recognisers are not robust to the non-native accents typical of language learners. Work in the last year has studied methods for combining acoustic models of the speaker's native language with that of the target language in order to increase robustness(K162,K163).

Neural Networks

Work on on-line learning and regularisation for nonstationary signal processing continues to make significant progress(K67,K68,K69,K70). The Cambridge research group has organised highly successful workshops at major international conferences, and is editing a landmark book which will be published in early 2000.

Medical Applications of Information Engineering

Work continues on finite element modelling of the fetal skull during the birth process(K93).

Three Dimensional Ultrasound Imaging

The Stradx 3D diagnostic ultrasound system(K123) produced by the Cambridge group, is now well established as one of the best of its kind. Non-planar re-slices through irregularly sampled 3D data sets have been created at a level of detail unmatched in the literature(K58,K59,K60,K124). Our tools permit clinicians to visualise curved structures within the body as if they were flattened out on to a plane. This can clarify certain features and assist the diagnostic process.

Radial basis function algorithms have been evaluated for interpolating sparse data(K131). This work has led to a better understanding of image artefacts caused by different interpolation strategies.

Many clinical applications involve the measurement of the volume of structures within the body. To achieve this, we first have to segment the structure of interest out from the surrounding tissue(K98,K123) and then interpolate a surface through the segmented contours. Our group has published the first algorithm for interpolation through arbitrary multi-axial contours, that does not require a prior model of the surface(K145,K146,K147,K148). We have also devised novel strategies for triangulating the resulting surface(K144).

The Stradx system continues to be updated and distributed freely on the web from http://svr-www.eng.cam.ac.uk~rwp/stradx/. The software is in use in other international research laboratories.

Computer Vision and Robotics

Research in computer vision and robotics has continued to develop new theories for recovering the three-dimensional shape of visible surfaces from images taken from arbitrary viewpoints with uncalibrated cameras with important applications in 3D model acquisition, visual tracking and the guidance of robots.

Novel contributions have been made in the analysis of curved surfaces (where the dominant image feature is the silhouette or outline). It is well known that it is possible to reconstruct the shape of curved surfaces from the family of outlines obtained by looking at them from different but known viewpoints(K164). We have also been able to show that viewer motion can also be recovered from the envelope of consecutive outlines - the frontier(K11,K29,K140,K141). A particularly simple and elegant solution was found for a special type of motion in which an object is placed on a turntable, which is rotated in front of a stationary camera. A novel algorithm was introduced which exploited the symmetry in the envelope of outlines swept out by the rotating surface(K111,K112). This technique uses a single curve tracked over the image sequence and has been successfully used to recover the shape of an arbitrary object from an uncalibrated camera.

New techniques have been introduced to fit B-splines automatically to image edge data(K20) and then to group fragments of curves which are projections of bilateral symmetry in the scene(K56). A quasi-invariant parameterisation of image curves which approximates group invariant arc-length with lower spatial derivatives has been shown to be extremely powerful in detecting symmetry in natural and textured images and efficiently recognising arbitrarily complex curves under arbitrary viewpoints and occlusion(K134,K135).

Research in the visual guidance of robots has continued(K21) and resulted in the development of a real-time 3-dimensional tracking system which exploits Lie algebraic techniques and robust methods(K52,K54).

This has been incorporated within the control loop of a robot manipulator which can be taught positions in the co-ordinate frame of its workpiece by presenting it with images from target views. This research was awarded the prize for the industrial practice of vision at the British Machine Vision Conference(K53). These tracking techniques have also been applied to motion segmentation and layering of video sequences. Bayesian statistical methods are used to separate the motion seen in a video sequence into two components and to interpret each of the regions within the image as belonging to one of those motions(K145).

A new project has been set up to build photorealistic models of architectural scenes from uncalibrated photographs. Novel algorithms have been developed to calibrate the cameras from information in the images alone(K15,K27,K28,K111) and to find correspondences(K14,K47). A Windows95/98/NT/2000 application called PhotoBuilder(K30,K31) has been developed which allows a non-expert user to interactively build a 3D model from arbitrary photographs. This can be downloaded from the group's web-site (http://svr-www.eng.cam.ac.uk/research/vision/). Preliminary
results are extremely promising and a model of Jesus College has already been built.

A simple stereo system using only one camera and a prism has also been developed(K96,K97).

Collaboration with the Xerox Research Centre in Cambridge continues. The aim is to investigate various aspects of document image processing, including document scanning using over-the-desk CCD cameras.

Laboratory for Communications Engineering

Sentient Computing is a long-term research theme of the LCE. A number of projects contribute to this area ranging from novel location systems to middleware and applications. An early example of a sentient application was the Cambridge Robot Football System(K77). TRIP (Target Recognition using Image Processing) is a novel sensor technology that uses the combination of visual markets (2-D circular barcode tags) and video cameras to automatically identify tagged real world objects in their field of view. A general-purpose framework for experimenting with sentient systems has been developed(K75).

The design of thin-client systems and their use in a variety of applications continues. A general purpose CSCW (Computer Supported Collaborative Working) environment has been developed which enables users to share desktops, and also to record and replay sessions(K100,K101,K102).

Research results which point the way to design middleware for supporting multimedia applications, with particular emphasis on mobility, have been reported(K119,K137).

A new member of academic staff, Dr. I.J. Wassell, joined the LCE in May 1999 with the specific remit to act as a focal point for the research programmes in Wireless Broadband systems. His research interests are closely aligned to Wireless Broadband systems and include Combined Coding and Modulation(K99), Orthogonal Frequency Division Multiplex (OFDM) systems, fast and hardware efficient equaliser design and Dynamic Channel Allocation/Selection in wireless systems.

Work on Wireless Broadband communications is underway in the form of a number of collaborative research projects with Adaptive Broadband Ltd. (ABL). ABL is developing Wireless Broadband systems for wireless LAN and wireless local loop (WLL) applications(K42). In the WLL application, the network employs a point-to-multipoint configuration and has an over-the-air data rate of 25 Mbit/s. A result of the collaboration with ABL has been the development of fast training (low overhead), yet hardware efficient equalisers that are required to combat multipath interference whilst keeping packet turnaround time low. In addition, an extensive programme of microwave radio propagation measurements have been conducted throughout the Cambridge urban/suburban area in order to characterise path loss for WLL systems. A theoretical propagation model for this scenario have been developed and its predictions compare closely with the field measurements.

In addition, the LCE in collaboration with AT&T Laboratories, Cambridge and ABL is currently deploying a Wireless Broadband demonstration/trial network in Cambridge. This network will permit multimedia applications, networking issues air interface issues to be investigated in a real system for the first time. A preliminary demonstration of the system was given at the `Communications Explosion' event held at the University on 24 September 1999.

Information Engineering References

[Table of Contents]

Last modified: July 2000