February 12, 2014
How Your Eye Can Predict the Future:
Neural Computation in the Retina
Michael J. Berry II
Professor of Molecular Biology and Princeton Neuroscience Institute
Minutes of the 17th Meeting of the 72nd Year
Presiding Officer: Ruth Miller
Invocation: Lead by Joan Fleming
Preceding week minutes: Read by Michael Mathews
Guests and visitors: None
Today’s attendance: 75
New members: None
Next meeting: Wednesday, Feb. 19, 2014. Topic: Presenting “The Figaro Plays” by Stephan Wadsworth, Friends Center (weather Permitting).
After gaveling the meeting to order at 10:15 AM, President Miller requested a moment of silence in memory of long time Old Guard stalwart Jack Reilly who recently passed away.
Today’s lecture was entitled:
How Your Eye Can Predict the Future: Neural Computation in the Retina
B.F. Graham introduced today’s speaker, Prof. Michael J. Berry II. He is an associate professor of molecular biology at Princeton University and the Princeton Neuroscience Institute as well as co-director of Princeton’s Program in Neuroscience. He was awarded a BS in physics with a minor in philosophy at U. Cal Berkley in1989 followed by a PhD in physics at Harvard in 1994. This was followed by five years post doc research, also at Harvard, where he worked on the neural code used by the retina to convey visual information to the brain. He joined the Princeton molecular biology faculty in 1999. He has organized and directed many courses in biological computation including summer courses at the Marine Biological Laboratory at Woods Hole, MA.
How the human brain works has long been a fascination for Prof. Berry. The overarching activity of the human brain is much more than the ability to play chess, to remember facts and images or to preform mathematical computations.
Manmade machines, namely computers, can do these tasks even better than we. However, computers are not very good with coordinated, rapid, purposeful movements which take place in the real world. In fact, robots are really clumsy. In order to understand the most important function of our brain is helpful to look at neural activities in less complex animal models.
o start to answer the question “why have a brain?”, Prof. Berry’s first slide introduced us to the tiny sea squirt which begins life as a larva swimming freely in search of a secure structure to which to anchor. Once fixed in place, the squirt matures into adulthood, consuming its own brain in the process. Only when moving around and searching, does the sea squirt need a brain. Higher organisms need a brain for intelligent activities such as finding food, moving to a more desirable location, and avoiding predators. From simple movements, such as avoiding walking into a table, up to very complex activities, such as climbing Mt. Everest, the brain must make predictions about the future, where objects will be or what events will occur. Prediction then is a core activity of the brain, whether in the simple sea squirt or the complex human being.
Prof. Berry then changed gears to explain for us the neuroanatomy of the human brain. It is made up of 100 billion nerve cells called neurons. Although there are at least a dozen shapes and many sizes of neurons, all share three basic parts: 1) the muti-branching dendrites which receive electrical inputs. 2) the cell body which is responsible for integration and cellular metabolism. 3) the axon which carries the electrical signal output. These electrical signals are called action potentials and they travel from one neuron to the next, allowing the neurons to communicate with one another. Some dendrites have thousands of branches, each branch capable of receiving its own individual input. These details were first described by the patron saint of neuroscientists, Ramon Y Cajal (1882-1934). He was a Spanish histopathologist who used Golgi silver nitrate stain to selectively blacken only a few neurons on a microscopic slide of the human brain. Even though his elegant drawings of the neural circuitry of the brain, as he saw them under his microscope, won him the Nobel Prize in physiology in 1906, he said of the architecture of the cortex “It’s a jungle”. Instead of trying to study the entire neocortex of the brain, Prof. Berry has chosen to simplify his investigation of neural circuitry by focusing on the retina of the eye.
The retina is a five layered sheet of nerve cells lining the back of the inside of the eye ball. The cornea and the lens focus light rays onto the retina. The first layer of photoreceptor neurons convert light into tiny electrical signals called action potentials. The next layer includes horizontal neurons, bipolar neurons and amacrine neurons which process these signals. Some of these cells are inhibitory cells which spread the signal latterly to other neurons; others perform computational activities. Finally, the ganglion layer of long axon neurons transmits the processed signals to the brain. Recent study has revealed even greater retinal cell complexity. There are four types of photoreceptor cells, 12 to 14 subtypes of bipolar cells, 30 different types of ganglion cells and at least 50 types of interneurons of different shapes and conductivity. The retina acts as a little brain. It has a lot more complicated circuitry than found in the most advanced digital camera. We would expect the retina to be capable of much more sophisticated functions than photography.
The retina sends information to the brain but no information comes back. This allows for experiments which remove the retina from an animal and lay it down on a glass slide into which electrodes are imbedded. The ganglion cells put out an electrical pulse which is picked up by these electrodes. Magnifying the amplitude of these ganglion output signals allows for measuring and recording these pulses at the same moment as the retina is responding to changes in the visual input thru the lens.
Returning to the question of coordinated, purposeful movement, how is it that tennis ace Roger Federer is able to return those 120 mile per hour serves? How does the brain rapidly process visual signals recorded on the retina? A flashing light stimulus delivered to the retina in the experimental model described above allows for the measurement of the firing rates of many action potentials. The peak rate of firing of action potentials lags 60 milliseconds behind the time of the flash. When an image, such as that of a tennis ball, is flashed on the retina there is a delay of at least 60 milliseconds before that action potential is transmitted to the brain. This response delay is too long to allow even Roger Federer to get his racquet to where the ball will be in 10 milliseconds.
The retina is capable of making some of the correction for this response delay. An experiment using a stationary bar instead of a flashing light as the retinal image demonstrates a similar delay before the action potential is detected. In contrast, when that bar is moving, the retinal cells respond much earlier. They also stop firing earlier when the bar is stopped. The earlier response is again seen when the bar is moved in the opposite direction. The response to a fixed image on the retina lags behind the response to a moving neural image. This results in part from dendrites with long branches that extend to distant areas in the retina. Neurons anticipate change in position through the action of a mechanism called contrast gain control. This process turns down the amplitude of the action potential peak of those neurons whose firing has lagged behind.
The moving bar experiment was repeated using different contrasts (different colors in layman terms). When using violet, the peak firing rate was the earliest and the amplitude was the greatest. A contrasts move down the color spectrum both firing rate and firing delay decreased with red at the bottom. Anticipation depends upon image contrast. Prof. Berry’s elegant slides illustrated the details and mathematics of this contrast gain control mechanism. Limits of both time and this reporter’s expertise do not permit further elaboration here. Suffice it to say, the ability of the retina to anticipate motion arises from neurons with widely spreading dendrites functioning along with a contrast gain control mechanism. Together, these produce an image of what things will look like in the near future. This prediction is almost instantaneous, involving only a few dumb neurons. No conscious deliberation or any higher input is necessary.
The retinal image input into the cortex of the brain is not a pattern of light but a processed image which is a simple prediction of the future. The basic neural elements of the retina are also found throughout the cortex. It is probable that similar mechanisms occur in the cortex for managing response delay, modulating gain control and for future prediction. In order to achieve high order anticipation, the retinal image may be organized onto a cortical map along with many other images. As a simple example of how the brain may function, Prof. Berry asked us to consider these three images: your son, house, money. The brain maps these images on adjacent, interconnected parts of the cortex which allows for the prediction that “your son will need money to buy a house.”
The retinal image of a tennis ball, already processed for delay, may reach Roger Federer’s cortex but much more processing and integration with other images along with motor neuron output must take place before he can return that serve.
Taking another example from professional sports, Prof. Berry showed two brief video clips from a past Oakland Raiders football game when Kenney Stabler was their quarter back. The first clip showed a beautiful, long forward pass which was effortlessly hauled in by a wide receiver way down field. The second clip was of what happened when a rusher jarred Stabler’s arm as he attempted a pass. The resulting fumble turned the football into a “Holy Roller” which bounced and rolled in all directions and no one could get a hold of it. Eventually, Oakland fell on it in the end zone for a touchdown, allowing the Raiders to go on to the Super Bowl. These scenes demonstrated two properties of retinal function: 1. Smooth motion allows for prediction. 2. Abrupt speed and direction changes of motion are unpredictable.
How does the retina handle abrupt changes of motion? As demonstrated earlier, smooth motion of an object in front of the retina produces a single peak of action potential firing rates. When the direction of the motion is abruptly reversed a second firing rate spike appears. The second spike, which appears 250 milliseconds after the first peak, acts as a reset signal to indicate that prior retinal predictions are all wrong. All changes of direction of motion cause similar large peaks. A burst of firing is seen when motion is started but no spike appears when an object suddenly stops. Every neuron sends this reversal signal to the brain. After about 400 milliseconds, while neural firing is suppressed, a newly predicted retinal image is established. The retina encodes two types of data; predictable information and surprising information. Two different neural channels carry these images to the brain; one channel for predictable events and another for unpredicted images. In the cortex, each sort of information has its own behavior value.
In conclusion, it is apparent that the retina is far more than a camera. It makes simple predictions about the location of moving objects. Any violation of those predictions produces a synchronous response. Studies of the retina lead to a greater understanding of the circuit mechanisms of prediction. This will enable construction of computational models with which to gain greater insight into complex prediction.
Following his presentation, Prof. Berry answered questions from the audience. The meeting was adjourned at 11:30 AM.
Respectfully submitted,
David H. Fulmer
Invocation: Lead by Joan Fleming
Preceding week minutes: Read by Michael Mathews
Guests and visitors: None
Today’s attendance: 75
New members: None
Next meeting: Wednesday, Feb. 19, 2014. Topic: Presenting “The Figaro Plays” by Stephan Wadsworth, Friends Center (weather Permitting).
After gaveling the meeting to order at 10:15 AM, President Miller requested a moment of silence in memory of long time Old Guard stalwart Jack Reilly who recently passed away.
Today’s lecture was entitled:
How Your Eye Can Predict the Future: Neural Computation in the Retina
B.F. Graham introduced today’s speaker, Prof. Michael J. Berry II. He is an associate professor of molecular biology at Princeton University and the Princeton Neuroscience Institute as well as co-director of Princeton’s Program in Neuroscience. He was awarded a BS in physics with a minor in philosophy at U. Cal Berkley in1989 followed by a PhD in physics at Harvard in 1994. This was followed by five years post doc research, also at Harvard, where he worked on the neural code used by the retina to convey visual information to the brain. He joined the Princeton molecular biology faculty in 1999. He has organized and directed many courses in biological computation including summer courses at the Marine Biological Laboratory at Woods Hole, MA.
How the human brain works has long been a fascination for Prof. Berry. The overarching activity of the human brain is much more than the ability to play chess, to remember facts and images or to preform mathematical computations.
Manmade machines, namely computers, can do these tasks even better than we. However, computers are not very good with coordinated, rapid, purposeful movements which take place in the real world. In fact, robots are really clumsy. In order to understand the most important function of our brain is helpful to look at neural activities in less complex animal models.
o start to answer the question “why have a brain?”, Prof. Berry’s first slide introduced us to the tiny sea squirt which begins life as a larva swimming freely in search of a secure structure to which to anchor. Once fixed in place, the squirt matures into adulthood, consuming its own brain in the process. Only when moving around and searching, does the sea squirt need a brain. Higher organisms need a brain for intelligent activities such as finding food, moving to a more desirable location, and avoiding predators. From simple movements, such as avoiding walking into a table, up to very complex activities, such as climbing Mt. Everest, the brain must make predictions about the future, where objects will be or what events will occur. Prediction then is a core activity of the brain, whether in the simple sea squirt or the complex human being.
Prof. Berry then changed gears to explain for us the neuroanatomy of the human brain. It is made up of 100 billion nerve cells called neurons. Although there are at least a dozen shapes and many sizes of neurons, all share three basic parts: 1) the muti-branching dendrites which receive electrical inputs. 2) the cell body which is responsible for integration and cellular metabolism. 3) the axon which carries the electrical signal output. These electrical signals are called action potentials and they travel from one neuron to the next, allowing the neurons to communicate with one another. Some dendrites have thousands of branches, each branch capable of receiving its own individual input. These details were first described by the patron saint of neuroscientists, Ramon Y Cajal (1882-1934). He was a Spanish histopathologist who used Golgi silver nitrate stain to selectively blacken only a few neurons on a microscopic slide of the human brain. Even though his elegant drawings of the neural circuitry of the brain, as he saw them under his microscope, won him the Nobel Prize in physiology in 1906, he said of the architecture of the cortex “It’s a jungle”. Instead of trying to study the entire neocortex of the brain, Prof. Berry has chosen to simplify his investigation of neural circuitry by focusing on the retina of the eye.
The retina is a five layered sheet of nerve cells lining the back of the inside of the eye ball. The cornea and the lens focus light rays onto the retina. The first layer of photoreceptor neurons convert light into tiny electrical signals called action potentials. The next layer includes horizontal neurons, bipolar neurons and amacrine neurons which process these signals. Some of these cells are inhibitory cells which spread the signal latterly to other neurons; others perform computational activities. Finally, the ganglion layer of long axon neurons transmits the processed signals to the brain. Recent study has revealed even greater retinal cell complexity. There are four types of photoreceptor cells, 12 to 14 subtypes of bipolar cells, 30 different types of ganglion cells and at least 50 types of interneurons of different shapes and conductivity. The retina acts as a little brain. It has a lot more complicated circuitry than found in the most advanced digital camera. We would expect the retina to be capable of much more sophisticated functions than photography.
The retina sends information to the brain but no information comes back. This allows for experiments which remove the retina from an animal and lay it down on a glass slide into which electrodes are imbedded. The ganglion cells put out an electrical pulse which is picked up by these electrodes. Magnifying the amplitude of these ganglion output signals allows for measuring and recording these pulses at the same moment as the retina is responding to changes in the visual input thru the lens.
Returning to the question of coordinated, purposeful movement, how is it that tennis ace Roger Federer is able to return those 120 mile per hour serves? How does the brain rapidly process visual signals recorded on the retina? A flashing light stimulus delivered to the retina in the experimental model described above allows for the measurement of the firing rates of many action potentials. The peak rate of firing of action potentials lags 60 milliseconds behind the time of the flash. When an image, such as that of a tennis ball, is flashed on the retina there is a delay of at least 60 milliseconds before that action potential is transmitted to the brain. This response delay is too long to allow even Roger Federer to get his racquet to where the ball will be in 10 milliseconds.
The retina is capable of making some of the correction for this response delay. An experiment using a stationary bar instead of a flashing light as the retinal image demonstrates a similar delay before the action potential is detected. In contrast, when that bar is moving, the retinal cells respond much earlier. They also stop firing earlier when the bar is stopped. The earlier response is again seen when the bar is moved in the opposite direction. The response to a fixed image on the retina lags behind the response to a moving neural image. This results in part from dendrites with long branches that extend to distant areas in the retina. Neurons anticipate change in position through the action of a mechanism called contrast gain control. This process turns down the amplitude of the action potential peak of those neurons whose firing has lagged behind.
The moving bar experiment was repeated using different contrasts (different colors in layman terms). When using violet, the peak firing rate was the earliest and the amplitude was the greatest. A contrasts move down the color spectrum both firing rate and firing delay decreased with red at the bottom. Anticipation depends upon image contrast. Prof. Berry’s elegant slides illustrated the details and mathematics of this contrast gain control mechanism. Limits of both time and this reporter’s expertise do not permit further elaboration here. Suffice it to say, the ability of the retina to anticipate motion arises from neurons with widely spreading dendrites functioning along with a contrast gain control mechanism. Together, these produce an image of what things will look like in the near future. This prediction is almost instantaneous, involving only a few dumb neurons. No conscious deliberation or any higher input is necessary.
The retinal image input into the cortex of the brain is not a pattern of light but a processed image which is a simple prediction of the future. The basic neural elements of the retina are also found throughout the cortex. It is probable that similar mechanisms occur in the cortex for managing response delay, modulating gain control and for future prediction. In order to achieve high order anticipation, the retinal image may be organized onto a cortical map along with many other images. As a simple example of how the brain may function, Prof. Berry asked us to consider these three images: your son, house, money. The brain maps these images on adjacent, interconnected parts of the cortex which allows for the prediction that “your son will need money to buy a house.”
The retinal image of a tennis ball, already processed for delay, may reach Roger Federer’s cortex but much more processing and integration with other images along with motor neuron output must take place before he can return that serve.
Taking another example from professional sports, Prof. Berry showed two brief video clips from a past Oakland Raiders football game when Kenney Stabler was their quarter back. The first clip showed a beautiful, long forward pass which was effortlessly hauled in by a wide receiver way down field. The second clip was of what happened when a rusher jarred Stabler’s arm as he attempted a pass. The resulting fumble turned the football into a “Holy Roller” which bounced and rolled in all directions and no one could get a hold of it. Eventually, Oakland fell on it in the end zone for a touchdown, allowing the Raiders to go on to the Super Bowl. These scenes demonstrated two properties of retinal function: 1. Smooth motion allows for prediction. 2. Abrupt speed and direction changes of motion are unpredictable.
How does the retina handle abrupt changes of motion? As demonstrated earlier, smooth motion of an object in front of the retina produces a single peak of action potential firing rates. When the direction of the motion is abruptly reversed a second firing rate spike appears. The second spike, which appears 250 milliseconds after the first peak, acts as a reset signal to indicate that prior retinal predictions are all wrong. All changes of direction of motion cause similar large peaks. A burst of firing is seen when motion is started but no spike appears when an object suddenly stops. Every neuron sends this reversal signal to the brain. After about 400 milliseconds, while neural firing is suppressed, a newly predicted retinal image is established. The retina encodes two types of data; predictable information and surprising information. Two different neural channels carry these images to the brain; one channel for predictable events and another for unpredicted images. In the cortex, each sort of information has its own behavior value.
In conclusion, it is apparent that the retina is far more than a camera. It makes simple predictions about the location of moving objects. Any violation of those predictions produces a synchronous response. Studies of the retina lead to a greater understanding of the circuit mechanisms of prediction. This will enable construction of computational models with which to gain greater insight into complex prediction.
Following his presentation, Prof. Berry answered questions from the audience. The meeting was adjourned at 11:30 AM.
Respectfully submitted,
David H. Fulmer