One of the most popular explanations of human behavior has been that somewhere beneath our rational mind lurks an ancient “lizard brain”—a primitive evolutionary holdover responsible for instinct, emotion, fear, and aggression.

There is just one problem. Neuroscientists have long known that the lizard brain story is wrong. Humans did not evolve a sophisticated new brain by simply stacking it on top of an older reptilian one.

But a new study suggests the enduring lizard brain myth may have inadvertently pointed to something more interesting. Mammalian brains do appear to be organized around two broad computational systems—but not an ancient reptilian brain located beneath a newer, more sophisticated one. Instead, the divide appears to reflect two fundamentally different ways of organizing and processing information.

Published in Science Advances, the research combines anatomical measurements from 182 mammalian species with experiments using artificial neural networks. The researchers found a striking evolutionary trade-off between the neocortex and a collection of structures associated with smell, memory, and the limbic system.

Instead of representing “new” and “old” brains, the researchers say these regions may have evolved different wiring styles optimized for solving different kinds of computational problems.

“Rather than an ‘old’ brain and a ‘new’ brain, our results suggest two systems wired for different kinds of computation,” lead author Dr. Nabil Imam, a researcher at the Georgia Institute of Technology, told The Debrief. “The neocortex is wired to preserve spatial relationships—for example, neighboring points in the visual world are analyzed by neighboring parts of the cortex. The limbic system is more diffusely connected and supports holistic associations across different pieces of information.”

“So, the division between the neocortex and limbic system is about how the two systems are wired,” Dr. Imam added.

This distinction offers a very different picture from the famous lizard brain story.

The lizard brain concept grew from the so-called triune brain theory, which portrays the human brain as having evolved in distinct layers. A primitive “lizard brain” handling basic instincts, a mammalian limbic system producing emotion, and, finally, a newer neocortex responsible for higher thought.

The problem is that neuroscience has long since overturned that evolutionary story. Vertebrate brains did not evolve by progressively stacking newer brain regions on top of older, more primitive ones. Instead, vertebrates share essentially the same broad organizational plan, with existing regions changing in size, connectivity, and function over evolutionary time.

The new study suggests, however, that the mammalian brain may be divided very differently—not by evolutionary age, but by how different neural systems are wired to process information.

Across 182 species from 10 taxonomic groups, animals with an unusually large olfactory system tended to have a smaller-than-expected neocortex. Animals with an expanded neocortex tended to show the opposite pattern.

The nine-banded armadillo provides a particularly dramatic example. Armadillos are largely nocturnal and depend heavily on smell while foraging. As researchers note, the armadillo’s olfactory system was approximately 1.5 times larger than expected for its overall brain size. Its hippocampus—a region central to memory and spatial processing—was about twice its expected size.

Its neocortex, meanwhile, was reduced by a factor of 2.8. Primates such as squirrel monkeys sit closer to the other end of the spectrum. They rely heavily on vision and devote a large portion of their expanded neocortex to processing visual information.

At first glance, that could look like evolution trading smell for sight. But the researchers noticed something more intriguing. The hippocampus grew and shrank alongside the olfactory system, even though memory is obviously not another form of smell. Other structures traditionally grouped into the limbic system, including the amygdala and septum, also tended to change together.

That suggested the important difference might not be what these regions process, but how they are wired to process it.

Two Very Different Ways to Build a Brain

Vision offers a relatively straightforward example. Neighboring points in the visual world are generally processed through neighboring areas of the visual system. Touch operates similarly. Locations beside one another on your body retain spatial relationships as the nervous system processes them. Hearing also uses orderly mappings, including those based on sound frequency.

The result is what neuroscientists call “spatiotopic” organization. In simple terms, the brain preserves relationships between incoming information by arranging it into orderly maps.

However, smell works differently. Odors are represented through combinations of activity distributed across networks rather than being laid out in the same neat spatial fashion.

The researchers suspected that this difference might help explain the anatomical trade-off they had identified across mammals. To test the idea, they turned to artificial neural networks trained to handle vision, touch, hearing, and smell.

They then mapped the networks’ internal models onto artificial two-dimensional surfaces resembling sheets of cortex.

Results showed that networks optimized for vision, touch, and hearing produced smooth, highly ordered maps. On a scale in which 1 represented strong spatial order, the visual network scored 0.90, touch scored 0.86, and hearing scored 0.89.

Yet, the smell network scored just 0.11. Rather than producing a smooth map, information was scattered across a fractured, distributed representation. The researchers even found that imposing localized connectivity on the olfactory model harmed its ability to perform its task.

Researchers then examined memory. Using an artificial network designed to reproduce some relational computations associated with the hippocampus and entorhinal cortex, researchers again saw a fractured, distributed representation resembling the olfactory system rather than vision, hearing or touch.

For Dr. Imam, one of the most surprising results was that artificial networks arrived at an organization resembling real brains despite learning in profoundly different ways.

“Artificial networks are trained by an algorithm that propagates errors through the network, whereas brains are shaped by evolution and through localized changes in synapses,” Imam explained. “The fact that both nevertheless arrive at similar organization suggests that the structure of the information itself—whether visual, auditory, tactile, or olfactory—determines what kind of wiring works best.”

No “Lizard Brain,” But Evolution May Be Trading One Kind of Brainpower for Another

Researchers next wanted to know whether competition between these two wiring styles could reproduce the anatomical patterns seen across real mammals.

They created an evolutionary simulation containing five artificial networks. Vision, hearing, and touch occupied one computational domain based on localized, spatial organization. Smell and hippocampal memory occupied another based on distributed connectivity.

Then an evolutionary algorithm redistributed computational resources between them.

When selection favored vision, resources moved toward the spatially organized networks. When selection favored smell, the distributed networks expanded.

But something interesting occurred to the simulated hippocampal network. Even though researchers weren’t directly selecting for memory, the hippocampal network grew along with the olfactory network because both shared the same broad computational architecture. As resources shifted toward the other system, both contracted.

That closely resembled the pattern found across real mammalian brains.

“It suggests that evolution may not be selecting individual brain regions one at a time, but rather the functions produced by whole patterns of wiring,” Dr. Imam explained. “Because regions that share similar wiring tend to grow or shrink together, selection for one kind of function comes at the expense of another, given that the system must operate with limited energy and space.”

That does not mean an animal with a larger neocortex is simply more intelligent than one with a larger limbic system. Nor does the distributed system represent a primitive emotional brain fighting against a rational one.

Instead, the two architectures appear suited to different computational problems. Evolution may have repeatedly adjusted how much neural real estate mammals devote to each.

Nevertheless, researchers caution that their models are intentionally simplified. Anatomical measurements came from several datasets; some taxonomic groups are better represented than others, and artificial neural networks are not replicas of biological brains.

The evolutionary model is intended to demonstrate that a relatively simple developmental mechanism could produce the observed trade-off, not to reconstruct exactly how mammalian brains evolved.

Still, the research raises another interesting possibility extending beyond the demise of the “lizard brain.” Our brains may arrive in the world with considerably more computational structure already built in than the familiar idea of a blank slate implies.

“The brain may be much less of a blank slate than we often imagine,” Dr. Imam told The Debrief. “Evolution and development build useful assumptions into its wiring before experience-driven learning.”

It is also another example of something scientists are increasingly encountering across very different areas of biology: evolution can produce remarkably sophisticated structures and systems that human engineers are just starting to understand, much less reproduce.

In a recent study covered by The Debrief, researchers discovered that cable bacteria construct microscopic electrical pathways from nickel-rich nanoribbons arranged remarkably like braided electrical wires. The naturally grown structures may achieve extraordinary conductivity while remaining flexible, properties that researchers are now interested in replicating for bio-based electronics.

The mammalian brain represents an enormously more complicated example, but the underlying lesson may be similar. Evolution does not begin with a blank slate each generation. It can build useful solutions into biological architecture itself, leaving subsequent learning along with adaptation to operate within structures refined over immense spans of evolutionary time.

That could also help explain one of the most striking differences between biological intelligence and today’s artificial intelligence. Modern AI systems can require enormous datasets to acquire abilities that biological brains learn from comparatively limited experience.

“That built-in structure may explain why brains can learn efficiently from relatively little experience, whereas most artificial networks require enormous datasets to learn useful functions,” Imam said. “The idea has deep roots in philosophy of mind, but only recently have neuroscience and computer science begun to provide concrete demonstrations of how such built-in structure can aid function.”

The recent study, “Dual computational systems in the development and evolution of mammalian brains,” appeared in Science Advances.

Tim McMillan is a retired law enforcement executive, investigative reporter and co-founder of The Debrief. His writing typically focuses on defense, national security, the Intelligence Community and topics related to psychology. You can follow Tim on Twitter: @LtTimMcMillan.  Tim can be reached by email: tim@thedebrief.org or through encrypted email: LtTimMcMillan@protonmail.com