# Lecture Notes

**learning from examples**as the problem of

**approximating a multivariate function from sparse data**- the examples. We present an overview of the theoretical part of the course and anticipate the connection between Regularization Theory and Statistical Learning Theory, the two cornerstone theories on which the course rests. We briefly describe biological implications and several different applications derived from the proposed framework, ranging from vision to computer graphics and the stock market.

**target space**,

**loss function, empirical risk, expected risk,**and

**hypothesis space**) and explain the structure of the theoretical part of the course. We look at the problem of learning from examples as the problem of

**multivariate function approximation**from sparse

**chosen**data, and then consider the case in which the data are

**drawn,**instead of chosen, according to a probability measure. This enables us to illustrate the viewpoint of

**Statistical Learning Theory**that will be used extensively during the course.

**Regularization Theory**is introduced as a natural framework for studying this classic, ill posed problem which otherwise admits an infinite number of solutions.

**Reproducing Kernel Hilbert Space**(RKHS). RKHS will be an essential tool for establishing a connection between Regularization Theory and Statistical Learning Theory. We first provide the background to the concept of Hilbert space, introduce RKHS, and then clarify the relation between smoothness and a priori knowledge on the solution in RKHS. A simpler derivation of the general form of the regularized solution is also obtained.

**Project Discussion:** This class is devoted to the discussion of the class projects.

- Classifying gender from face images
- Analyzing and synthesizing images
- Learning object detection

The first topic is about the use of a very simple HyperBF network to classify face images as male or female. After training, the centers of the network come to resemble caricatures of male and female faces.

The second topic describes how to train networks to analyze or generate new images of the same or similar object - such as a face - in terms of user defined control parameters such as viewpoint or expression. The key element is a representation of images that ensures a sufficient degree of smoothness between inputs and outputs.

The last topic is about a trainable object detection system using at its core a Support Vector Machine classifier.

**bagging**and

**boosting**and suggest some plausible justification for their success. We also describe some recent work about combining SVMs in a way similar to bagging.

**This final class is devoted to the presentation of the class projects.**

Project Presentation:

Project Presentation: