The Structure of Creativity: Why Your Brain is Creative (and ChatGPT Isn’t)

A theory on the nature of creativity in the brain, and why LLMs are incapable of it

In my last post, I argued that the brain is a “corporation” of billions of neurons in which each neuron operates as a simple agent, communicating and working together to enhance each other’s capabilities and become collectively capable of incredible things. In essence, the mind is an intelligence composed of intelligences.

The Mind is a Corporation of Neurons

Despite the name, Artificial Neural Networks (ANNs) are not like this. Their “neurons” are not autonomous intelligent agents and lack any capacity for learning. They are nodes within a graph representing a series of mathematical operations. What “learning” they are capable of is a product of applying the backpropagation algorithm — an ingenious algorithm for updating the connections between artificial neurons during training to minimise a “loss function” i.e. how much the response diverges from the desired answer. This means that the feedback it receives during its training always comes from outside of itself, and judged by an external standard.

If we look back at biological neural networks like our brains, there is no backpropagation algorithm to update all our neurons based on an objective loss function calculated externally. Instead, our neurons must provide each other with feedback. But what standard can they judge by? I think they most likely judge based on the Free Energy Principle, aiming to minimise their surprise. The Free Energy Principle states that a self-organising system (such as a living cell) works to minimise the difference between its predictions about the world and its actual experiences (“surprisal”), both by adjusting its predictive model and by acting upon the world. (You can hear its creator Karl Friston explain it here, or watch this intro video I like.)

So, each neuron judges according to its expectations and acts to bring its expectations and reality into alignment. Importantly, these expectations are constantly evolving. As they attempt to make sense of their reality, they adapt both how they understand their world and how they respond to it. This means that the standards by which the brain judges itself are internal, evolving, and distributed across the neurons.

This work of trying to minimise surprisal means a brain will attempt to model its external world as received through its senses, but it also allows feedback loops to form, where a group of neurons evolve a set of mutual expectations and behaviours that fulfil those expectations. Behaviours that meet expectations receive positive feedback, and in turn, prove the expectations correct. In this way, the neurons engage in arbitrary patterns of activity simply because patterns are satisfying.

We can think of it as a “neural folk dance”. Each plays their part while enjoying the unity and predictability of the dancers around them. It does not matter that the dance serves no objective goal beyond itself. The dance is itself the goal.

Photo by Matheus Frade on Unsplash

What does this have to do with creative intelligence? I believe these feedback loops with their evolving expectations underlie creative intelligence. Creativity simply is these neural folk dances. They are neurons at play, spontaneously co-creating arbitrary new forms of order and harmony for order and harmony’s sake, according to a novel standard that co-evolves with the practice itself. It is not a pattern received from the outside but conceived from within the mind itself.

These arbitrary forms of order and harmony may be found to be useful. This is the case with mathematics. It is well understood by the average schoolchild that most maths will never be needed in day-to-day life. It clearly was not devised out of any sense of need, beyond the most basic arithmetic. No, we know that mathematicians pursue mathematics because they are captivated by the overwhelming sense of order, harmony, and beauty. The fact that it turns out to be useful is just the cherry on top. Maths for its own sake has always preceded actual uses being found for it.

They may also be practically useless, as in the case of art and music. They serve no practical purpose, except for the fact that society embraces them. And why does society embrace them? For the same reason as the neurons: when we participate in and co-create these arbitrary patterns of expectation and behaviour, we create new forms of order and harmony. Creativity at the cultural level follows the same basic dynamic as at the neural level!

This is why our current forms of AI, including LLMs like ChatGPT, will never possess creative intelligence. Their entire “learning” process involves absorbing the requirements of a static, pre-existing standard (albeit an extremely impressive standard). It is pure imitation, no imagination.

This is also why these AIs perform increasingly well on all the tests we prepare for them. If we can test it, it seems AI can learn it. But creative intelligence begins where tests end. Genius creates the very criteria by which it must be judged.

That is not to say that no future machines will be capable of this kind of creativity. In fact if we built machines based on the Free Energy Principle and set them to work & communicate together, I believe this kind of creativity might arise spontaneously. Although it would be incredibly expensive to attempt anything like this on the scale of the human brain in the foreseeable future.

This might inform how we redefine education in the AI age. We have spent decades educating by fixed standards, moulding children into imitators of the ideas and practices of the past. Perhaps now that we have machines far superior at the imitation game (pardon the pun) we should change tack. What if we abandoned the practice of judging students against fixed standards? What if schools became a playground of ideas and originality? What if we emphasized intrinsic motivation over rewards and punishments? I recently learned about “collaborative grading”, and I think this is a step in the right direction. If we empower students to co-create their own standards we can nurture intrinsic motivation, creativity, and even genius.

OK, I will leave this here for now. Please let me know what you think in the comments :)