Do We Learn Like Machine Learning Algorithms Do?

Implicit and Explicit Learning

What is learning? The first thing that comes to mind is education – whether it’s my kids going to school and being taught how to read or add numbers, or adults at university studying the classics, or how to run a business. That’s one kind of learning – a combination of being shown something new, and gaining an understanding of how to take actions that enable the doing of that new thing.

But there is also a more implicit kind of learning that takes place all the time – especially in infants. This kind of learning takes place without an explicit human teacher and without any need for association with actions. This type of learning enables pattern recognition.

Pattern Recognition

All of our senses are constantly working to turn information from the physical world into electrical impulses in our nervous systems and brains. As adults, most of the time we are not aware of a sense of newness. However, sometimes we get surprised or encounter novelty, and this failure to recognise a pattern tends to trigger learning in our brains to help make successful pattern recognition more likely in the future.

To put it another way, learning helps us make better predictions or actions than we did in the past.

Sometimes we are aware of having previously learned a pattern – we may say “I remember when I first saw that.” For example, I might be introduced to a new person whose face I have never seen before, and the next time that I see their face I remember who that person is.  To achieve this pattern recognition, our brains have learned three things: (1) the person’s name, (2) what their face looks like, and (3) that their name and the face should be associated.

Usually pattern recognition by our brains is of the kind that enables unconscious filtering of unimportant information that bombards our senses. This type of learning lets us ignore things that are not changing or which are not of high value.

I wrote above that this kind of pattern recognition takes place without a human teacher. However, there are many factors that can help individuals learn that new variations on the same pattern belong to the same cause, and these factors can be thought of as helping teach us to learn. It might initially surprise you, but time is a great teacher! Things that happen at the same time or in a rapid sequence tend to be things that should be learned as belonging together.

Electrical Impulses

Both types of learning appear to be a very normal and routine part of life. However, the underlying biological and physical processes that achieve this are extremely complex. Indeed, biochemical changes happen that literally change our brains! This takes place solely in response to the brain receiving electrical impulses from our eyes (e.g., seeing the person’s face) and electrical impulses from our ears (e.g., hearing the person’s name).

The brain processes those impulses using its neurons in several ways. The information from the eyes and ears are assessed to see if it has already been learned. When the ‘answer’ is no, learning (i.e., changes to the brain) is triggered which changes the answer next time.

How does this learning take place in the brain?

Synapse Changes

The brain is complex at multiple scales of detail, in a similar way to how a mobile phone network has to work: (1) at the network scale, (2) at the scale of a single phone, (3) at the scale of the individual chips in the phone, and (4) at the scale of the individual transistors in each chip.

For me, the most interesting aspect of biological learning is at the scale of small brain regions comprised of networks of neurons and synapses that connect neurons. The literature suggests that our ability to learn comes from changes in synapses. Traditionally, synapses were thought to have ‘strengths’ that get larger or smaller during learning, and that these strengths encode different patterns. More recently, it has been revealed that more drastic changes typically take place, where some synapses die and others grow. The resulting changes influence which neurons communicate with other neurons.

At a high level, the ‘criteria’ that determines which synapses change is clear: somewhere there must exist a network of neurons and synapses, which before learning, did not enable ‘recognition’ of a pattern (e.g., “that’s a new face”), but after the synapses have changed, recognition was enabled (e.g., “I know that face”).

This brings us to the question in the title. I have discussed how changes in synapses enable learning, and pointed out that high-level pattern recognition criteria determine how those changes take place. Both aspects are also essential aspects of most machine learning algorithms. In this case, some mathematical parameters need to change, in accordance with some criteria, in response to exposure to data.

For example, a deep neural network is a machine learning algorithm that may contain millions of parameters. The parameter values are learned in a way to minimise mathematical functions, enabling the neural network to take in images of cats and dogs, and decide which images are cats and those which are dogs.

Improved Designs of Neural Networks based on the Brain

One area where the brain outmatches out best technology is in its power consumption – we learn using 1000s of times less energy than our computer algorithms do!

An analogy can be drawn between how synapses in the brain change, and changes to the parameters in our machine learning algorithms. By understanding how neurons and synapses cooperate to enable learning in the brain, we can design better, more powerful pattern recognition devices which can run without expensive energy-hungry computing chips.