Dr. Geoffrey Hinton
Professor, Computer Science Department, University of Toronto

An alternative approach is to use layers of hidden units whose activities are a deterministic function of the the sensory inputs. The activities of the hidden units provide additive contributions to a global energy, E, and the probability of each sensory datavector is defined to be proportional to exp(-E). The problem of perceptual inference vanishes in deterministic networks, so perception is very fast once the network has been learned.

The main difficulty of this approach is that maximum likelihood learning is very inefficient. Maximum likelihood adjusts the parameters to maximize the probability of the observed data given the model, but this requires the derivatives of an intractable normalization term. I shall show how this difficulty can be overcome by using a different objective function for learning. The parameters are adjusted to minimize the extent to which the data distribution is distorted when it is moved towards the distribution that the model believes in. This new objective function makes it possible to learn large energy-based models quickly.

Speaker Bio:

Geoffrey Hinton received his PhD in Artificial Intelligence from Edinburgh in 1978. He did postdoctoral work at Sussex University and the University of California San Diego and spent five years as a faculty member in the Computer Science department at Carnegie-Mellon University. He then moved to the Department of Computer Science at the University of Toronto. He spent three years from 1998 until 2001 setting up the Gatsby Computational Neuroscience Unit in London and then returned to Toronto.

Geoffrey Hinton is a fellow of the Royal Society, a former president of the Cognitive Science Society, and the first winner of the David E. Rumelhart prize. He was one of the researchers who introduced the widely used back-propagation algorithm. His other contributions to neural network research include Boltzmann machines, distributed representations, time-delay neural nets, mixtures of experts, Helmholtz machines and products of experts. His current main interest is in unsupervised learning procedures for neural networks with rich sensory input.

<< Back