THE PERCEPTUAL CORRELATES OF NEURAL REPETITION SUPPRESSION THOMAS SPRAGUE

BENE710 NEUROPLASTICITY AND PERCEPTUAL LEARNING MO AND WEDS 430PM615PM
BRANDOM AUGUST 31 1998 NONINFERENTIAL KNOWLEDGE PERCEPTUAL EXPERIENCE AND
LEEDS WORKSHOP 17 PERCEPTUAL SIMULATION AND CONCEPTUAL FIELDS IMPLICATIONS

PERCEPTUAL BELONGINGNESS DETERMINES THE DIRECTION OF LIGHTNESS INDUCTION DEPENDING
PERCEPTUAL EVIDENCE AND THE NEW DOGMATISM RAM NETA I
PERCEPTUAL LEARNING STYLE PREFERENCE QUESTIONNAIRE (COPYRIGHT 1984 BY

Visual illusions are a powerful tool for understanding the relationship between neurobiology and perception {Eagleman, 2001 #193}

The Perceptual Correlates of Neural Repetition Suppression

Thomas Sprague

Rice University


Visual illusions are a powerful tool for understanding the relationship between neurobiology and perception (Eagleman, 2001). Manipulations of objective reality that lead to an unexpected subjective perception help us place constraints on the neural architecture which encodes, processes, and integrates sensory information. Though vision feels subjectively as though it is an effortless, simple process, this could not be further from the truth. The mammalian visual system must accomplish the daunting task of turning a distorted, variable-resolution retinal image into a subjective perception consisting of objects, people, and places. Visual illusions help us determine the tricks and shortcuts the visual system takes in generating a coherent perception of the world around us. Any neural explanation for visual perception must account for these visual illusions.


The perception of time is also subject to illusion (Eagleman, 2008). Through careful manipulations of different stimulus attributes, different aspects of time perception can change. The perception of short durations is remarkably suspect to distortion. When an image is shown repeatedly, its perceived duration begins to contract with each repeated presentation. However, when an 'oddball' stimulus is flashed for the same duration, it appears to stay onscreen for a longer time than the earlier repeated stimulus. (Pariyadath & Eagleman, 2007). Interestingly, this pattern of perceived duration contraction appears to mirror the neural phenomenon of repetition suppression, in which neural activity decreases in response to a repeated stimulus (Fahy, Riches & Brown, 1993, Grill-Spector, Henson & Martin, 2006, Li, Miller & Desimone, 1993, Pariyadath & Eagleman, 2007, Rainer & Miller, 2000). It may be that the neural energy used in response to a stimulus is proportional to its perceived duration (Ivry & Schlerf, 2008, Pariyadath & Eagleman, 2007). Surprisingly, this illusion also works for an unexpected stimulus in a predictable sequence (such as 1-2-3-4-7, Fig 1A). It has been suggested that the predictability of the stimulus, rather than stimulus repetition, leads to the duration distortions observed, and that instead repetition is a special, boundary case of predictability (Pariyadath & Eagleman, 2007, Fig 1B). This also provides evidence that low-level stimulus features are not the exclusive cause of this illusion, as the number images differ considerably in low-level visual features but are instead related in a higher-order way.


Over the course of the summer, I used the Moutoussis and Zeki color-motion asynchrony illusion (Eagleman, 2001, Moutoussis & Zeki, 1997) to investigate the relationship between neural activity and perception. In this illusion, a field of squares rapidly and repeatedly alternates between two directions of motion and two colors at the same time. However, the two stimulus attributes are not perceived as changing simultaneously. The two stimulus attributes must change approximately 80 milliseconds (ms) apart (specifically, color must change 80 ms after motion changes) in order to be perceived as changing together (Fig 2). An argument has been made that this illusion results from delayed neural processing for motion relative to color. Some have taken this to mean that a percept of a given direction of motion or a given color occurs directly due to the neural activity in populations responsive to motion or to color and is distorted in time by differences in the neural latencies between these different populations. This, however, does not seem to be a likely explanation of the illusion. A color-motion asynchrony is not typically observed for a single motion/direction change (Nishida & Johnston, 2002), and temporal order judgments are accurately made with the same stimulus (Amano, Johnston & Nishida, 2007, Bedell, Chung, Ogmen & Patel, 2003, Nishida & Johnston, 2002). A better explanation may be that this illusion results from inaccurate perceptual binding of the two features (Nishida & Johnston, 2002).


Different factors have been shown to change the magnitude of the color-motion asynchrony. If a different direction change angle is used (e.g. 90 degrees instead of 180 degrees) the magnitude of the illusion sharply decreases (Amano et al., 2007, Arnold & Clifford, 2002, Bedell et al., 2003, Clifford, Spehar & Pearson, 2004, Linares & Lopez-Moliner, 2006). Additionally, motion speed has been shown to alter the illusion (Bedell et al., 2003, Clifford et al., 2004). Since both of these stimulus attributes change aspects of the neural response to the stimulus (Priebe & Lisberger, 2002), it is plausible that other changes to the neural response to the stimulus can lead to a change in the illusion.


All experimental manipulations that have been performed on this illusion use repetitive, alternating colors and directions of motion. The repetitive nature of this stimulus may cause neural repetition suppression. It is possible that this repetition suppression contributes to the color-motion asynchrony illusion. If we make the stimulus less repetitive, repetition suppression should occur to a lesser extent. Can changing the neural code in this way change the magnitude of the color-motion asynchrony found?


Testing this possibility required both behavioral and functional neuroimaging experiments. We built a stimulus in which the color changed between grey and either a repeated or random isoluminant color. First, 13 subjects were tested on a behavioral version of the task in which the stimulus was presented with a variable delay between color and motion change. A 720 ms oscillation period was used, which is comparable with other implementations of this paradigm (Moutoussis & Zeki, 1997), and 10 color-motion change delays were tested. Two conditions were tested: Repeated and Random (Fig 3A). In the Repeated condition, the stimulus predictably alternated between grey and a single randomly-chosen color for 9 cycles. In the Random condition, the stimulus changed between grey and a different randomly-chosen isoluminant color with every oscillation period (9 different colors, one for each cycle). Motion was the same for both conditions, changing between upward and downward motion with the same 720 ms period. Subjects were asked to respond with one of two keypresses to indicate which direction the colored squares were moving. Any given subject completed 8 repetitions per condition (Repeated or Random) for 8 of the 10 color-motion change delays, and 3 repetitions for the two boundary delays (0 and 360 ms) for a total of 140 trials.


To analyze the behavioral data, we first plotted average responses for each color-motion change delay for each condition. To determine the color-motion asynchrony for a subject, we fit a psychometric curve (the cumulative Gaussian) to the average responses for each condition (Fig 3B). In accordance with standard lab procedures, the fit of each curve was tested by a blind algorithm to ensure RMS error was less than .25 and the 50% point passed within the range of values tested. Subjects with one or more unacceptable fits were not used in further analysis. If no color-motion asynchrony occurred, subjects would be expected to perform at chance (e.g. 50% report color moving up, 50% report color moving down) when the color is 90 degrees out-of-phase with motion (180 ms color-motion delay, see Fig 3B). If the 50% point of the fitted curve is different from this expected value, there exists a color-motion asynchrony for a particular subject. We used a repeated-measures t-test with a threshold of p < .05 to test whether there was a difference between the color-motion asynchrony for the Repeated and Random conditions.


The Random condition showed a significantly smaller color-motion asynchrony than did the Repeated condition (t(7) = 3.73, p = .007), averaging 21.9 ms (Fig 3C). Stimulus repetition is thus an important component of the illusion, though not its sole cause. Since the motion was always predictable, it is possible that the remaining color-motion asynchrony in the Random condition (60.3 ms, t(7) = 8.1, p < .001) is in part due to the predictable change in motion direction. Since direction change angle is an important determinant of the magnitude of the color-motion asynchrony, randomly varying this parameter is difficult to accomplish experimentally. However, testing the color-motion asynchrony using stimuli with variable predictability along more than one dimension may be possible using biological or Newtonian motion.


The observed results of the behavioral portion of the experiment fit well into the hypothesis that neural repetition suppression alters fundamental perceptual qualities of a stimulus, especially within the time perception domain (Pariyadath & Eagleman, 2007). To verify that our stimulus causes neural repetition suppression, we must perform a functional neuroimaging experiment using fMRI. There are important issues to consider when designing a visual or time perception neuroimaging experiment.


The first problem we addressed was developing a method to generate isoluminant colors on a different display technology (LCD projector). Due to the proximity of the projection equipment and screen to the fMRI scanner, careful measurement using a photometer or colorimeter was impossible. These tools were used by our lab in the past to calibrate CRT monitors to display the isoluminant stimuli used in the behavioral component of this experiment. Since absolute physical isoluminance was impossible to obtain with our scanning configuration, we chose to use heterochromatic flicker fusion to establish perceptual isoluminance (Wagner & Boynton, 1972). Nine periods of the stimulus were presented, so nine isoluminant colors were found by rapidly alternating a constant grey square with each color at 30 Hz. If the luminance of the two shades (grey and the color) is substantially different, the subject will perceive that the square is gently 'pulsing'. Subjects adjusted the luminance of each color so this luminance flicker was at a minimum (Fig 4A). These nine colors were used for later stimulus presentation.


fMRI experiments must be designed in a way such that key assumptions about the hemodynamic response are met. Since fMRI measures blood flow and not neural activity directly, the response can be quite slow, typically about 2-6 seconds after stimulus onset (Logothetis, Pauls, Augath, Trinath & Oeltermann, 2001). The response also takes a fair amount of time to diminish, usually 6-10 seconds, which must be incorporated into the experimental design. Additionally, it helps to avoid a rhythmic pattern to trials due to neural 'climbing activity' in anticipation of an expected event (Nobre, Correa & Coull, 2007). We incorporated all of these considerations into our design by separating stimulus presentation and response by a random interval of 2-6 s, and waiting 6-10 s after the response is made before proceeding to the next trial (Fig 4B). Because trials are now much longer (averaging 20 s vs. 7 s for the behavioral version), fewer trials can be tested. The period was slightly adjusted to be 800 ms (instead of 720 ms) to account for the different refresh rate of the display equipment (60 Hz vs. 100 Hz for behavioral experiment). We chose to test 5 color-motion change delays (0, 100, 200, 300, and 400 ms) for each of the two conditions (Repeated and Random, as before). Subjects performed 5 judgments for each trial, bringing the total number of trials in the scanner to 50.


If repetition suppression is found in the contrast between the Repeated and Random BOLD signal in response to the stimulus presentation epoch (Fig 3 & 4), it is important to determine if repetition suppression is occurring in color-responsive regions. To accomplish this, we also performed a short scan to isolate color-responsive regions (our color localizer). We displayed checkerboard patterns for 4 s, followed by a 6-10 s random wait time (Fig 4C). The pattern was colored every other trial. 8 repetitions for each pattern type were presented, for a total of 16 trials. Subjects were instructed to passively view the patterns during image acquisition.


We are currently running subjects on this imaging experiment. A total of 20 right-handed subjects will be tested, and each subject will also perform the behavioral version of the experiment described above. Our plan for imaging data analysis is as follows. First, we will use the data from the color localizer to isolate color-responsive regions for each subject (Fig 5A). We will compare the time course BOLD activation in these voxels for each condition during the stimulus presentation epoch (Fig 5B). Data will be averaged across all color-motion change offsets for each condition. A measure will be formulated to quantify the amount of repetition suppression observed in these regions, which we can then correlate to the change in color-motion asynchrony found in a subject’s behavioral data. For subjects who did not report 100% up or 100% down for a given color-motion change offset in the imaging session, we may try to create a classifier to use imaging data to predict a subject’s response.


This illusion provides an interesting means to investigate how a particular manipulation of net neuronal function influences different aspects of a perception. Repetition suppression itself is a fascinating finding – the same physical stimulus is encoded in different ways, yet very little about its perception changes. If we can find a perceptual correlate to these changes in neural coding, we can better understand how our rich, detailed perception of the world results from the vibrations in our neurobiological machinery.


This summer has been an excellent opportunity to further my understanding of the experimental methods used to test new theoretical ideas about human perception. Our work is still in preliminary stages, but future directions will likely include detailed mathematical modeling of perceptual phenomena, basing models off known neural circuitry and physiology. As of yet, we do not have a detailed-enough understanding of the phenomenology of the illusion or the neural mechanisms responsible for it. This report discusses the first functional imaging experiment conducted using this color-motion asynchrony illusion. The results from this experiment will hopefully provide further insight into the perceptual correlates of neural repetition suppression.


Acknowledgements

I would like to thank David Eagleman, Steve Cox, Vani Pariyadath, the scanner technicians at the Human Neuroimaging Laboratory, and the National Science Foundation for making this project possible and helping me along the way. This work was partially supported by NSF REU Grant DMS-0755294.


References

Amano, K., Johnston, A., & Nishida, S. (2007). Two mechanisms underlying the effect of angle of motion direction change on colour-motion asynchrony. Vision Res, 47 (5), 687-705.

Arnold, D.H., & Clifford, C.W. (2002). Determinants of asynchronous processing in vision. Proc Biol Sci, 269 (1491), 579-583.

Bedell, H.E., Chung, S.T., Ogmen, H., & Patel, S.S. (2003). Color and motion: which is the tortoise and which is the hare? Vision Res, 43 (23), 2403-2412.

Clifford, C.W., Spehar, B., & Pearson, J. (2004). Motion transparency promotes synchronous perceptual binding. Vision Res, 44 (26), 3073-3080.

Eagleman, D.M. (2001). Visual illusions and neurobiology. Nature Reviews Neurosci., 2, 920-926.

Eagleman, D.M. (2008). Human time perception and its illusions. Current Opinion in Neurobiology, XX (X), X.

Fahy, F.L., Riches, I.P., & Brown, M.W. (1993). Neuronal activity related to visual recognition memory: long-term memory and the encoding of recency and familiarity information in the primate anterior and medial inferior temporal and rhinal cortex. Exp Brain Res, 96, 457-472.

Grill-Spector, K., Henson, R., & Martin, A. (2006). Repetition and the brain: neural models of stimulus-specific effects. Trends in Cognitive Sciences, 10 (1), 14-23.

Ivry, R.B., & Schlerf, J.E. (2008). Dedicated and intrinsic models of time perception. Trends Cogn Sci, 12 (7), 273-280.

Li, L., Miller, E.K., & Desimone, R. (1993). The representation of stimulus familiarity in anterior inferior temporal cortex. J Neurophysiol, 69 (6), 1918-1929.

Linares, D., & Lopez-Moliner, J. (2006). Perceptual asynchrony between color and motion with a single direction change. J Vis, 6 (9), 974-981.

Logothetis, N.K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature, 412 (6843), 150-157.

Moutoussis, K., & Zeki, S. (1997). A direct demonstration of perceptual asynchrony in vision. Proc R Soc Lond B Biol Sci, 264 (1380), 393-399.

Nishida, S., & Johnston, A. (2002). Marker correspondence, not processing latency, determines temporal binding of visual attributes. Curr Biol, 12 (5), 359-368.

Nobre, A., Correa, A., & Coull, J. (2007). The hazards of time. Curr Opin Neurobiol, 17 (4), 465-470.

Pariyadath, V., & Eagleman, D.M. (2007). The effect of predictability on subjective duration. PLoS ONE, 2 (11), 1264.

Priebe, N.J., & Lisberger, S.G. (2002). Constraints on the source of short-term motion adaptation in macaque area MT. II. tuning of neural circuit mechanisms. J Neurophysiol, 88 (1), 370-382.

Rainer, G., & Miller, E.K. (2000). Effects of Visual Experience on the Representation of Objects in the Prefrontal Cortex. Neuron, 27, 179-189.

Wagner, G., & Boynton, R.M. (1972). Comparison of four methods of heterochromatic photometry. J Opt Soc Am, 62 (12), 1508-1515.





THE PERCEPTUAL CORRELATES OF NEURAL REPETITION SUPPRESSION THOMAS SPRAGUE


Tags: correlates of, perceptual correlates, suppression, neural, perceptual, sprague, correlates, repetition, thomas