Reconstructing the Visual System

A couple of years ago we did a blog featuring the cognitive neuroscientist, Donald Hoffman, posing the question about the extent to which we construct our visual reality.  The discussion was based on principles Hoffman considered in his book on Visual Intelligence.

Hoffman now has a new book out, The Case Against Reality: Why Evolution Hid the Truth From Our Eyes.

IMG_6364Hoffman argues in favor of our perception of objects in the real world being a set of hacks that the brain uses to conserve resources and energy based on fitness principles.  He is fond of using icons on the computer screen as a metaphor for the way in which our visual system represents the fidelity of what is seen in the real world.  What differentiates Hoffman’s approach is that he apparently grounds his theory in mathematical principles.  Math is scarce in the book, but we get a hint of it in this YouTube video.

At the 8 minute mark, Hoffman shows a set of dots that appear stationary, followed by movement that results in a cylindrical appearance the dots in 3D space.  Here is an example of 3D coherent motion:

After demonstrating how we construct our own visual reality, and of necessity filter out much of what occurs as demonstrated by examples of change blindness, Hoffman notes that we are not infallible about our conscious experiences; they have surprises for us.  How is it that psycho-physicists can change the array of dots that literally results in a new dimension of consciousness – one that provides 3D perception?  Hoffman notes that our perception of the dots can be manipulated through the following equation:

Screen Shot 2019-08-30 at 10.22.27 AM

He cites many other examples of visual brain hacks, brought to the surface by conscious perception of various illusions.  Take Joseph’s hat, for example, showing how context can influence color perception.


The main message here is that there is normally massive compression of data in the visual system.  The human eye has 7 million cones and 120 million rods, each carrying compressed information.  The circuity of the eye squeezes this further down to 1 million signals and forwards it to the brain via the world’s best fiber optic cable, the optic nerve.  The brain must then correct errors and decode actionable messages about fitness.


In contrast, consider what happens during the bottleneck of brain injury.  The normal hacks have been compromised, the data isn’t being compressed the way it normally is, and key variables in the equations the brain normally writes to solve problems have been displaced.  This deconstruction results in a massive overload of data, as what formerly had been operating effortlessly at a subconscious level now floods consciousness.  It is fatiguing, and it is debilitating.

Neuro-visual rehabilitation can therefore be conceived as reconstruction.  The patient must learn to re-interpret visual space, to re-apply heuristics that enable shortcuts, and to re-establish these frameworks within the context of time.  The tools may be lenses and prisms, the visual illusions of stereograms, the veridicality of virtual reality, or purposeful mismatches within neuro-sensory systems, all done in the service of restoring the calculus of healthy brain function and fitness.


December, 2019 – Added bonus video:

4 thoughts on “Reconstructing the Visual System

  1. Vision falls on a continuum between expectation and exploration. Expectations are based on sense/brain, action, and environment. Change any of the three and we get illusion. In other words, illusions are the result of doing vision in new environments as we did vision in old environments. When sense fails due to injury, expectation breaks down and exploration is used to reprogram the system. While it is a good metaphor to suppose we see with our eyes or brains, it is perhaps a better metaphor to suppose we use our eyes and brains to see.

    • Yes, David – I believe we can apply math to that as well, our brains doing Bayesian Probability calculations to set expectations. And of course, the perennial metaphysical question: In the famous Macondald quote, “Eyes don’t tell brains to see; brains tell the eyes what to look for”, regardless of eyes or brains, who is it that is doing the looking?

  2. Does this not also happen in the initial development where there is no “learned” pattern and they develop a our foundation without having an “old” foundation to fall back on? From an article by Robert Steinman, Zygmunt Pizlo, Tatiana I. Forofonova and Julie Epelboim – Spatial Vision: 2002 , “Gaze is accurate in order to see clearly; not because targets can be seen clearly” – One Fixates accurately in order to see clearly not because one sees clearly. When either babies during development or redevelopment following an injury, learning to LOOK is critical. Babies have the whole world with no expectations whereas those with head injury try to rely on their “expected” way of seeing that has been disrupted.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s