The Importance of Learning Slowly

Earlier this year I came across a review by Gary Woodwill of a book about learning, thinking and acting. It was the title that grabbed my attention "The Importance of Learning Slowly". The book is by Manfred Spitzer, The Mind within the Net: models of learning, thinking, and acting.One of the comments Woodwill makes about the book is that:

While a single event can have an impact, it usually takes many events to have a relatively permanent change in the brain (aka “learning”) and to extract general features and generate rules from experience ... and according to Spitzer "It must learn quickly for obvious reasons, but it must learn slowly in order to generalize in a way that will produce the optimal solution without oscillating around it or forgetting it because of some other stimulus.” (p. 53)

We often find our models of understanding the world in the latest technologies available to us. Piaget developed his multi-stage theories of learning from observing his own children, and then applying the dominant mechanical metaphors of his day. In the 19th century, Adolphe Quetelet, a Belgian astronomer, coined the term “the average man” based on the pendulum (Piaget’s “equilibrium”), while Herbert Spencer wrote a psychology of adaptation using the newly created thermostat as his model (Piaget’s “adaptation and assimilation” within set limits).

As Foucault taught us in Discipline and Punish, much of the psychology of the twentieth century is based on disciplinary practices derived from factories and prisons, giving us such learning technologies as classrooms, regular rows of seats, raising of hands, incremental rewards, recess, time periods, and the power of the teacher’s gaze.

In the second half of the twentieth century, two models of computing competed for dominance. One model was artificial intelligence, based on a model of inputs, storage, processing, and outputs - in other words, a factory metaphor. The other model was that of neural networks, modeled on what was then known about the functioning of brains in humans and other animals. In the 1950s, AI became the darling of computer science, leaving neural network development far behind in terms of funding and attention.

After more than 50 years of development, the hopes of those developing traditional artificial intelligence have hit a brick wall. While powerful computers using brute force in the number of computations they can carry out have beaten grand masters at chess, the same computers cannot recognize anything for which they have not been specifically programmed. In other words, they can memorize a seemingly endless set of facts, but have no flexibility to be creative with what they “know”. On the other hand, new developments in brain research has stimulated renewed efforts in using neural networks to produce flexible learning processes in computers, and to help researchers understand learning in living organisms.

Manfred Spitzer’s The Mind within the Net is one of the best non-technical narratives on how minds work using the neural network model. Some of these explanations are startling, while others reinforce positions of strong advocates of individual freedom and the power of informal learning, such as Stephen Downes, George Siemens, and Jay Cross.

Like neural networks, the brain is based on vector algebra, rather than numerical computations. Vectors have strength and direction, and many vectors, representing multiple inputs, unite to form a result. The result in the brain is strengthening or weakening of a set of neural connections, a relatively slow process. While a single event can have an impact, it usually takes many events to have a relatively permanent change in the brain (aka “learning”) and to extract general features and generate rules from experience. Spitzer says that “… there exists a tension, a problem, for every learning organism. It must learn quickly for obvious reasons, but it must learn slowly in order to generalize in a way that will produce the optimal solution without oscillating around it or forgetting it because of some other stimulus.” (p. 53) This conclusion challenges those who advocate extreme learning and other forms of speeding up the learning process. In fact, according to Spitzer, trying to learn to quickly can actually be detrimental:

“… the interactions of an organism with its environment can be generally conceived of as a sampling of data in order to predict the true (i.e., adaptively valid) values of parameters. As stated above, this task can only be accomplished if every single experience (every single input pattern) has only a very small impact on the changes in the network. If the changes are too large, estimates may oscillate around the true values rather than approximating them. Again: the system only works with learning happens slowly.” (p. 54)

Spitzer also believes that children can and need to learn more quickly than adults. Children need a rough and ready view of the world while adults want to increase their depth of understanding. Spitzer relates this to the pace of change in today’s society. “The old master violin building makes better violins than the young student of the trade. If, however, all of a sudden the customers want music synthesizers, student will adapt to change more readily.”

The importance of feedback is apparent in both brains and neural networks. Neural networks have a technique called backpropagation of errors that simulates feedback loops in the brain that slowly change the hidden layers between input and output. This means that learning is much more to do with practice and observation than being told what to do. “Children learn from examples,” says Spitzer. The brain stores its learning in “self organizing feature maps.”

Spitzer is a psychiatrist in Germany, so it is not surprising that he has a chapter entitled “The Disordered Mind” in which he discusses autism and oppression. Most of his conclusions are on the best way to raise children, making this book less applicable to the adult learning. However, there are so many insights going through it that I highly recommend it to everyone in education and training. Spitzer’s newest book is on learning and will be translated into English by the end of the year. I’m looking forward to reading it.