Visual Asynchrony & Temporally Extended Contents

2.1. Perceptual Latencies

The causal process whereby the occurrence of a stimulus in a subject’s environment leads to a conscious experience of that stimulus takes time. Psychologists divide that interval into three rough periods:²⁰

• external latency: the time it takes for a physical signal produced by the distal stimulus to reach the sensory receptors of a perceiving subject.

• neural latency: the time it takes for the physical signals produced by a subject’s sensory receptors following proximal stimulation to “ascend the neural processing hierarchy”, including different brain areas responsible for processing different features of the stimulus (Holcombe 2009; 2015).

• online latency: the time between completion of neural processing of a stimulus in central cortical areas and the occurrence of a conscious experience of that stimulus.²¹

There is evidence that different features of a stimulus can be processed at different speeds, and so incur differential neural latencies. For instance, differences in contrast intensity, or in chromatic brightness, can affect the speed at which dedicated neurons respond to such features: the more intense the stimulus, and the brighter the contrast, the faster the neuronal response. This can be observed early on in the lateral geniculate nucleus (LGN) of macaque monkeys, in their primary visual cortex, even in later stages of processing such as the superior temporal sulcus.²² There’s also evidence that colour can be processed faster than motion.²³

2.2. Differential Latencies in Vision

In an influential experiment by Moutoussis and Zeki (1997a), the display consisted of squares moving up and down at an alternation rate of 1–2 Hz. As they moved, their colour also changed from green to red at the same alternation rate: all squares simultaneously had the same colour and changed colour simultaneously, likewise with their direction of motion.²⁴

Figure 1

Illustration of two oscillations (in-phase trial) in Moutoussis and Zeki’s (1997a) experiment.

The experiment included two series of trials: one where the period of oscillation for each feature took 565 ms, the other with oscillations at 706 ms (Moutoussis & Zeki 1997a; Bedell et al. 2003; 2006). In some trials, changes in motion direction and colour were in phase (i.e., the switch in both colour and motion was simultaneous), while in other trials they were not (the chromatic change occurred either midway through the motion, or so as to allow one colour to dominate through motion in one direction), and each combination was presented four times in mixed order (see Moutoussis & Zeki 1997a: 394, figure 2), with each trial consisting of ten repetitions. Subjects had to perform a pairing (or matching) task: that is, report the colour of the squares for a specified direction of motion, and vice versa.²⁵

Moutoussis and Zeki found that, for the squares to appear with a uniform colour when moving in one direction, the change in the direction of motion must in fact occur about 80 to 140 ms before the change in colour.²⁶ In other words, subjects systematically paired the direction of motion with the wrong colour:

Color-motion pairs perceived to occur simultaneously are not simultaneously present in the real world. By changing both the color and the direction of motion of objects rapidly and continuously and asking subjects to report which color-motion pairs were perceived as coexisting, it was found that motion was paired to the color present on the computer screen roughly 100 ms later (Moutoussis & Zeki, 1997a, 1997b). If there were no perception-time difference between color and motion, then the perceptual experience of the subjects should follow the reality occurring on the screen. These results thus suggest that the color of an object is perceived 100 ms before its direction of motion, a phenomenon we have called perceptual asynchrony and that we have attributed to the different processing-times necessary for the two functionally specialized systems to “finish” their corresponding jobs. These systems are thus not only processing but also perceptual systems, creating specific visual percepts in their own time and independently from one another. (Moutoussis 2014: 202–3)

The results vary somewhat as a function of the following factors:

1. angle of direction: if the direction of motion changes by less than 180 degrees (say the object’s trajectory shifts by 45 degrees only), motion needs to precede colour only by 80 ms.²⁷

2. relative luminance: decreasing chromatic luminance reduced the effect by 25 ms; 40 ms when the luminance of the motion was decreased.²⁸

3. task: the effect is observed when subjects are asked to report which colour they see when the object is moving, say, upwards (pairing/matching task). There are suggestions (Nishida & Johnston 2002; 2010; Bedell et al. 2003; Clifford et al. 2003) that the effect seems to disappear almost entirely (reduced to about 6 ms) when subjects are tasked instead with reporting which change (colour or direction of motion) occurs first (temporal order judgment).²⁹

These results demonstrate that, owing to the differential neural latencies with which different features of a stimulus are processed, what are in fact non-simultaneous features (motion preceding colour) of non-simultaneous presentations of a stimulus (or of successive stimuli) appear to occur simultaneously in experience. This, we’ll now see, has various implications for online latencies—the interval between the completion of neural processing for a representation of a given feature and when that feature becomes conscious.

2.3. Two Models

According to the online model,³⁰ a stimulus feature becomes conscious immediately after its processing by a dedicated neural mechanism is completed.³¹ Hence, differences in neural latencies between colour and motion result in the colour and motion of one stimulus (at a time) becoming conscious at different times, which serves to explain visual asynchrony. Because the direction of motion of an upward green stimulus takes longer to process (about 100 ms more, say), such a feature becomes conscious at the same time as the colour (red) of the subsequent downward stimulus, which is processed at greater speed. As a result, subjects report seeing a red square moving upward, followed by a green square moving downward.³²

In contrast, Eagleman’s “delayed perception” model has it that the last stages of processing leading to the generation of a conscious experience involve postdictive mechanisms, to the effect that the visual system “wait[s] for the slowest information to arrive” before “going live”—Eagleman suggests a delay of about 80 ms (2010: 223).³³ On this model, online latencies might outstrip neural latencies in the sense that the visual system generates a conscious representation only after all the synchronous features of a stimulus have been processed. But if the brain waits for the slower feature to be processed before generating a conscious experience of both, why expect any visual asynchrony at the conscious level, on this model? The timeframe is key: an 80 ms delay allows the most recent information about the direction of motion to be still missing when the processing of such information will be completed, say, 100 ms after chromatic information is processed. Hence the asynchrony: when neural processing of the colour (green) of the upward moving square is completed, the direction of its motion is still being processed, even after 80 ms, given a 100 ms neural latency. This is why, the suggestion must go, its colour (green) ends up being paired, inaccurately, with the direction of motion of the previous downward-moving red stimulus.³⁴

An additional possible difference between the two models concerns feature-binding in conscious experience. For the online model, different types of features (e.g., colour and motion) are processed separately and independently—as Moutoussis insists in the last sentence quoted earlier. And since, on this model, such features will become conscious as soon as their neural processing is completed, this means that the colour of a stimulus will “make it into” consciousness slightly before its direction of motion does, if the latter is processed more slowly. In other words, the online model suggests, cases of visual asynchrony provide some evidence that there is no feature-binding of the different features of a stimulus in conscious experience (following Zeki 1993). That is, there is strictly no single experience representing, say, a red square moving upward, on this version of the online model. Instead, an experience of redness and a distinct experience of upward motion occur more or less at the same time, which explains why subjects report them as simultaneous when engaged in a pairing task.

It might seem natural, on the other hand, to view the delay model as operating under the assumption that the different features of a stimulus are bound together in conscious experience: indeed, the point of the 80 ms online delay could be precisely to allow for feature-binding, whereby colour and motion, which are processed separately and at different speeds, are then bound together in a conscious representation of both features during the online delay. Subjects thus enjoy a visual illusion of a red square moving upward, which explains the pairing judgments subjects are tasked to report.³⁵

The little evidence there is to adjudicate between these models doesn’t seem to support the delayed perception model, however.³⁶ Consider the illustration on the left of figure 2:³⁷

Figure 2

two versions of the Hess illusion.

The rectangles are moving from left to right and back in synchrony—that is, in the same direction, at the same speed, occupying the same relative location at the same time. Whereas the top rectangle is brighter on its left side, the bottom rectangle has a brighter right side. If brighter stimuli are processed faster, one should expect the brighter side of each stimulus to be processed more rapidly than their darker side. So, were the online model to be correct, the reasoning goes, one should expect that, when both rectangles are moving to the right, the apparent length of the top rectangle should seem to contract somewhat while that of the bottom rectangle should seem to expand (Eagleman 2010: 221–23; Holcombe 2015: 825). The thought appears to be that, since the brighter side of each rectangle is processed faster than the other, it will become conscious earlier, if the online model is true. Thus, as it moves towards the right, the left-hand side of the top rectangle will become conscious before the right-hand side, and will retain this temporal advantage throughout its rightward motion. When the right-hand side becomes conscious a little later, it will appear at a location which, owing to its online delay, is closer to the location occupied by the faster processed left-hand side: closer, that is, than it in fact is, hence the apparent contraction. Similarly with the bottom rectangle: its brighter right-hand side will become conscious first, while the left-hand side is delayed and appearing at a location further apart from that of the faster processed right-hand side, thereby producing an apparent dilation of the stimulus when moving rightward.

Were such an illusion to be observed, advocates of both models seem to agree, it would rule in favour of the online model (Eagleman 2010: 221–26; Holcombe 2015: 825). Though the brighter side may be neurally processed faster than the other, the reasoning seems to go, the different neural latencies at play may still fall within an 80 ms online delay, for instance. Hence, according to the delayed perception model, both sides of the square could be fully processed before becoming jointly conscious following the 80 ms delay. That’s why there’s no reason to think, on the delay model, that one side of the square becomes conscious earlier than the other, and so no reason to expect any apparent contraction or expansion of the stimulus as a result.³⁸

Yet, it appears the sort of illusory experience the online model predicts does in fact occur. And while Eagleman (2010: 226) has conceded this much, he has tried to explain the illusion away as a function of the use of a neutral density filter to reduce contrast over parts of the screen. White et al. (2008) report having produced the illusion without a density filter, however—in this case, a rotating Hess effect, similar to the illustration on the right of figure 2, where the rectangles (or lines, in this case) look to curve—thereby suggesting that different parts of the figure with different degrees of illumination reach consciousness at different times, as the online model predicts.³⁹

2.4. Significance for Retentionalism

How does the evidence from visual asynchrony bear on the retentionalist view? At first sight, it might seem as though the online model is inimical to retentionalism: if stimuli become conscious very soon after being neurally processed, one might naturally expect that two successive events (e.g., two notes in a ringtone, or two successive presentations of the same red object) would become conscious successively, as soon as each is neurally processed. Conversely, the delayed perception model might look like a natural fit for retentionalism: an 80 ms delay could help explain how two successive events can nevertheless become simultaneously conscious in a temporally extended content, as retentionalism requires. If so, any evidence supporting the online model against the delay model (as in §2.3) ought to sound like bad news for retentionalism.

The situation is a little more complicated, though—and, in fact, worse for retentionalism. For even if the evidence had favoured the delay model rather than the online model, the former isn’t really such a natural fit for retentionalism after all. In fact, evidence from visual asynchrony appears to undermine retentionalism in at least two respects, even when the “delayed perception” model is taken for granted.

To see this, it’s important to keep two commitments of retentionalism clearly in sight. First, the “no-dependence-on-memory” desideratum for temporally extended perceptual contents:

non-mnemonic temporal content: if a conscious experience e occurring at some time t represents an event y succeeding some event x, (i) e depends on sensory stimulation by x (at t-n) and y (at t-n*), neurally processed so that (ii) both x at t-n and y at t-n* can figure in the content of a conscious perceptual experience for the first time at t, and (iii) neither the representation of x nor that of y in e “retain the content” of an earlier conscious experience of x (at t-n) or y (at t-n*).

Second, clauses (i) and (ii) in this desideratum have the following implication for the neural processing underpinning temporally extended perceptual contents:

partially delayed processing (pdp): if some perceived event x occurring at t-n and some later event y occurring at t-n* can both figure in the content of a conscious perceptual experience e occurring at t, then the neural processing NP_x prompted by x at t-n and which results in experience e at t begins earlier—and lasts longer—than the underlying neural processing NP_y of y from t-n* to t.

To repeat, retentionalism requires that non-simultaneous events represented at the same time in the same experience must have different neural or online latencies.

Yet, evidence from visual asynchrony suggests there are cases where this isn’t so. For the evidence surveyed in §2.2 indicates, not just that (a) there doesn’t seem to be any neural or online delays of the sort retentionalism needs, but that (b) for those neural latencies which do impact conscious experience, the resulting experiences lack a temporally extended content.

A brief summary of the evidence for (a) includes:

• (i)    no delay: even with different synchronous features of a stimulus, when one feature (motion and its direction) has a longer processing latency than the other (colour), the brain doesn’t appear to wait until the more slowly processed stimulus feature (motion) has been fully processed. Rather, each feature becomes conscious in its own time, with colour reaching consciousness before the direction of motion. This holds for the online model, as well as with the 80 ms delay posited by the delayed perception model—the differential online latency between colour and motion is simply much shorter (about 20 ms rather than 100 ms) on the latter.

• (ii)    constant processing: if a colour of a given intensity is processed at a certain speed, it is processed roughly at that same speed through successive presentations of the stimulus (the same goes for the slower processing of direction of motion). This explains how the effect of Zeki and Moutoussis’s experiment occurs through successive presentations of the display in a given trial (consisting of 10 repetitions): the upward direction of motion of an earlier presentation is wrongly paired with the red colour of a later presentation, followed by the downward motion of yet another earlier presentation being paired (wrongly again) with the green colour of a later presentation, and so on. There is no indication, that is, that processing speeds or online delays somehow become shorter over successive presentations, as one would expect if partially delayed processing (pdp) were true.

As for (b):

• (iii)    no extended content: in a pairing task, the direction of motion of an earlier stimulus (x at t-n) is reported as synchronous with the colour of a faster-processed later stimulus (y at t t-n*): though the motion of x at t-n and the colour of y at t-n* seem to become conscious more or less at the same time (t), both features are judged (wrongly) to occur simultaneously.

• (iv)    delay not enough for extended content: if the brain did wait an extra 80 ms before binding in consciousness the more slowly processed direction of motion of x at t-n with the more quickly processed colour of y at t-n* (as the delay model predicts), such delay wouldn’t suffice to guarantee that the resulting experience is representing two successive events or their features as such: the point of the delay model is to (1) explain how colour and motion are bound into a conscious representation of a simultaneous event (e.g., an upward-moving red square), where (2) the posited delay (80 ms) is short enough to explain how information about the direction of motion (e.g., the red square is in fact moving downward) arrives too late to be factored in such a conscious representation. Even on the delay model, that is, processing isn’t so delayed as to allow that different synchronous features of the same stimulus (rather than successive ones) are represented in the same conscious experience—let alone that successive stimuli could be processed to become conscious simultaneously.

It isn’t just that the evidence from visual asynchrony fails to line up with the main claims and commitments of retentionalism. The assumption of partially delayed processing (pdp), which the retentionalist postulation of temporally extended perceptual contents demands, seems inconsistent with what happens in cases of visual asynchrony.

Consider figure 3, to begin with, which depicts the sort of experiences subjects appear to have in cases of visual asynchrony—with the distal stimuli at the bottom, the contents of the relevant experiences at the top (highlighted in grey), and the different processing latencies marked by the orientation and length of the different arrows in-between:⁴⁰

Figure 3

Standard model of experiential contents in visual asynchrony.

Note first that no model or explanation of visual asynchrony posits or needs to posit an experience at t (or thereafter) which represents both events x (stimulus presentation at t-n_ii) and y (stimulus presentation at t-n_i), let alone represents them as succeeding one another. The main difference between the online and delay models is how long after a stimulus is neurally processed do conscious experiences e₁, e₂, e₃, occur: 80 ms or much less.

But why exactly couldn’t the experiences depicted in figure 3 have temporally extended contents instead? Can’t a retentionalist simply re-describe the situation as in figure 4: the subject’s experience right after t (e₃) has a temporally extended content representing the stimulus’ changes from t-n_ii to t—that is, the downward motion of a green square, followed by the upward motion of a red square? Figure 4, that is, illustrates the kind of partially delayed processing (pdp) retentionalism would need to assume in this case:

Figure 4

Experiential contents in visual asynchrony according to retentionalism.

There are three problems with this picture. First, for experience e₃ to represent both x (the stimulus at t-n_ii) and y (the stimulus at t-n_i) in a temporally extended content, online latencies would have to be significantly longer than what the evidence suggests. The online latency for the stimulus’ downward motion at t-n_ii would have to be significantly longer than the online latency for the stimulus’ upward motion at t-n_i in order for both to become conscious simultaneously after t (similarly with online latencies for the stimulus’ green colour at t-n_i and its red colour at t). The problem isn’t just that there’s no evidence that this is how online latencies unfold. The problem is that the evidence from visual asynchrony shows that such partially delayed processing (pdp) for the same type of features is in fact rather unlikely: processing of the direction of motion remains stable in such conditions, with a constant latency across successive presentations or repetitions. Otherwise, the asynchrony effect might not hold across the ten repetitions which constitute a given trial in Zeki and Moutoussis’s experiment.

What’s more, the online delays implied by such a suggestion should strike one as implausibly long. Suppose the delay in question encompasses processing of just two successive presentations of the stimulus (e.g., at t-n_ii and t-n_i), even at the rapid alternation rates used in cases of visual asynchrony: between 250 ms between changes (in the version of Nishida & Johnston 2002) and up to 565 ms or even 700 ms (for Zeki & Moutoussis’s 1997a version). Assume it takes only about 80 ms at the earliest for the second presentation of the stimulus at t-n_i to become conscious (that is, assume a rather short neural latency with no added online latency, if only for the sake of argument): this version of retentionalism would still require processing delays of the first presentation of the stimulus at t-n_ii to range between 330 ms and up to 780 ms, in order for an experience like e₃ in figure 4 to have an extended content representing just two successive presentations of a stimulus at rapid alternations.⁴¹

Second, what about experience e₂ (right after t-n_i) which precedes experience e₃ (see figure 3)? There’s no reason why retentionalists should deny that an experience occurs at that time. But if the contents of e₂ and e₃ are both temporally extended (as in figure 4) by virtue of partially delayed processing (pdp), the content of e₃ representing the succession [green downward moving square followed by red upward moving square] from t-n_ii to t is likely to partially overlap that of e₂, since both have [green downward moving square] from the stimulus between t-n_ii and t-n_i as part of their content. The retentionalist view would then need to explain the status of that part of e₃’s content it shares with e₂’s. For if such content rests on neural processing already deployed for the same representation of the stimulus in an earlier experience (e₂), doesn’t it violate the “no-dependence-on-memory” desideratum on temporally extended contents? That is, isn’t the shared content [green downward moving square] from t-n_ii to t-n_i in fact mnemonically “retained” from e₂ to e₃—and if not, why not?

Third, and more importantly, it becomes unclear why there should be any visual asynchrony in the first place. For if partially delayed processing (pdp) is true, it remains mysterious why, in experience e₃ (figure 4), the upward motion of y at t-n_i should end up being bound with the red colour of a later presentation of the stimulus at t, rather than being (accurately) bound with its own colour (green) at t-n_i—ditto for the stimulus’ features at t-n_ii. On both the online and delay models, what accounts for the asynchrony is the fact that there isn’t enough time to complete neural processing of both synchronous features (i.e., colour and direction of motion at t-n_i) before one (colour) becomes conscious, owing to the longer neural latency of the other (motion direction). But given the longer neural or online latencies retentionalists need to posit in pdp, if the brain has all this additional time to produce a conscious experience of successive red and green moving squares, doesn’t it thereby have ample time to fully process and accurately bind the synchronous colour and direction of motion of a given synchronous presentation? There’s no reason to expect that cases of visual asynchrony arise, if partially delayed processing (pdp) is assumed, and no mechanism to explain why they nevertheless do.

Here, retentionalism faces the same difficulty the delayed perception model ran into: the latter, we saw, implies that illusions like the Hess illusion ought not to occur. The retentionalist has to posit even longer online delays, since such delays must encompass the completion of neural processing for several successive presentations or repetitions of the changing stimulus—whereas the 80 ms delay of the delayed perception model was meant to explain just one synchronous presentation of the stimulus at a time, not several. And if there’s reason to rule out online delays of 80 ms, why think that longer online delays are somehow likelier?

The difficulties surveyed thus far arise in a specific set of conditions, where the alternations of the changing stimuli from green to red and moving up and down proceed at the rate of 1–2 Hz. It’s in those conditions that evidence of visual asynchrony has surfaced. In the next section, I attempt to generalize the problem.

3.1. Constant Processing

Is the temporal gap from proximal sensory stimulation to conscious experience really constant? After all, evidence from visual asynchrony reveals that the processing of different features of a stimulus (e.g., motion, orientation) is delayed by about 100 ms in comparison to others (e.g., brightness, colour).

Properly contextualized, the assumption that the temporal gap is constant remains consistent with such evidence. First, relative to those features (bright contrasts, more intense colours) that are neurally processed faster, there’s no reason to think that the interval between proximal sensory stimulation and a resulting conscious experience doesn’t stay constant through successive presentations. Ditto with those features processed somewhat less rapidly (motion and its direction, orientation, etc.). Second, though there is evidence that neural and online latencies vary between different sensory modalities,⁴³ the constancy hypothesis can be restricted to a specific modality. More importantly, the constancy hypothesis isn’t at all a claim to the effect that different features, in different sensory modalities, can be neurally processed more or less at the same time. It’s a claim, rather, about the relative speed and duration of neural processing for successive presentations of the same stimulus’ features through the same sensory channels—it’s in this sense that processing latencies are supposed to remain relatively constant over time.

Accordingly, different perceptible features within a sensory modality can be grouped in terms of the interval it takes for them to be neurally processed and reach consciousness. For each such group, one can obtain a different specific version of the following principle, the causal constancy assumption:

Reference to “similar experimental settings w₁, w₂, w₃” merely serves to emphasise it is not enough to characterize such causal/temporal assumptions regarding neural processing solely in terms of the features being processed: the conditions in which they are perceived (e.g., whether adaptation can take place, what angular information about motion is presented, at what rate the features alternate, what stimuli they are preceded by, etc.) can make a difference too.⁴⁴

Note also what talk of “constancy” amounts to in this context: it’s not just that the temporal gap λ after which a conscious experience of a given feature F occurs takes (i) no less than λ across successive proximal stimulations caused by F, but also that it takes (ii) not much more than λ in normal circumstances. Indeed, evidence from visual asynchrony doesn’t just support a conclusion about how long a conscious experience is delayed (or not) after proximal stimulation, owing to the pace of neural processing. It provides information about the maximal relative length of such a delay. For instance, in Moutoussis and Zeki’s (1997a) experiment, the evidence suggests it always takes longer for the visual system to process information about the direction of motion than about colour. But this delay (between 80 and 140 ms) isn’t just a lower bound: neural processing of the direction of motion is always behind the neural processing for colour by more or less the same amount. Significantly, this holds for successive presentations of the same stimulus throughout a given trial (in this case, ten repetitions, in the same conditions).

This means that λ needn’t be a strict or precise value, but a limited range: there is room for some variation between 80 and 140 ms, both across different trials carried out in different conditions, and through successive repetitions within a given trial. The latter, for instance, would allow that λ compresses somewhat over several successive presentations of the same stimulus with the same stimulus features presented in the same conditions. For example, processing of a green surface could well accelerate over the course of successive repetitions—thus, λ could be a little longer for neural processing at the onset of a stimulus compared to its offset. Such compression, however, still requires some minimum delay (λ’s lower bound) in the processing of each successive presentation of the relevant stimulus feature.⁴⁵ As we’ll see, this is all the argument below really needs—provided, that is, that processing of a later presentation of the same stimulus feature isn’t so compressed as to become conscious simultaneously with an earlier one (in which case, pdp holds, and processing wouldn’t be constant at all, even in this moderate sense).

Finally, note that cca doesn’t presuppose the online or delay model. In particular, cca can still be true if the delayed perception model is preferred: for even if the production of a conscious experience is delayed by about 80 ms to allow for the slower processing of other simultaneously presented features, 80 ms serves as a maximum too.

3.2. The Argument

Imagine seeing a red fire-truck cross your visual field at great speed—in a few seconds, it is gone. Bright red and seen in broad daylight, the fire-truck’s colour is likely to be neurally processed relatively quickly. Its motion (not just the fact it is moving, but its trajectory through the scene) is the sort of temporally extended event one can perceive, according to retentionalism, via a temporally extended content. The example can be schematically described as follows:

• (i)    at t₁, the red fire-truck is located at l₁.

• (ii)    at t₂, the red fire-truck is located at l₂.

• (iii)    at t₃, the red fire-truck is located at l₃.

• (iv)    at t₄, the red fire-truck is located at l₄.

• (v)    at t₅, the red fire-truck is located at l₅.

Suppose that each time—t₁, t₂, t₃, t₄, and t₅—is separated from the next by about 200 ms, so that the interval from t₁ to t₅ lasts about 800 ms. Suppose also that the perceptual conditions from t₁ to t₅ (lighting, your visual attention focused on the truck, etc.) remain perfectly stable.

Next, apply cca:

It’s quite possible that different instances of cca, with different values for λ, need to apply: one for the colour of the fire-truck, another for its motion and its direction, or for its relative location, if these features are neurally processed at different speeds. But since the colour of the fire-truck can be processed fairly rapidly, and is seen in the same perceptual conditions throughout, application of cca to the situation ((i)–(v)) delivers the following:⁴⁶

• (1)    at t₁+λ, S has a conscious visual experience of the fire-truck’s redness at l₁.

• (2)    at t₂+λ, S has a conscious visual experience of the fire-truck’s redness at l₂.

• (3)    at t₃+λ, S has a conscious visual experience of the fire-truck’s redness at l₃.

• (4)    at t₄+λ, S has a conscious visual experience of the fire-truck’s redness at l₄.

• (5)    at t₅+λ, S has a conscious visual experience of the fire-truck’s redness at l₅.

This description of S’s conscious experience through the relevant interval—(1) to (5)—can be summarized as follows:

• (6)    from t₁+λ to t₅+λ, S has a succession of different conscious visual experiences, each with a different representational content: each represents the fire-truck’s redness at a different location.

Obviously, this carries serious implications for the different interpretations of thesis R3, the retentionalist claim about the temporal ontology of experience.

First, (6) clashes with the temporally restricted version of claim R3—that perceptual experiences with temporally extended contents are instantaneous. Since neural processing takes time, and since the time it takes for a given feature F (colour) to be neurally processed in the same conditions is relatively constant, it takes no less and no more than λ for successive presentations of such a feature through t₁ to t₅ to be processed. In which case, successive presentations lead, via λ in cca, to a succession of different experiences. Even if neural processing of feature F did shorten progressively over the interval from t₁ to t₅ (becoming faster at t₅ + λ than it was at t₁ + λ), the conclusion would still hold. For even a relatively inconstant and narrowing temporal gap λ* would suffice to guarantee that an experience of the fire-truck’s colour from t₁ to t₅ occupies an interval from t₁+λ* to t₅+λ* and varies in content across that interval.

The modally restricted version of claim R3—that the temporally extended content of a perceptual experience must be such that everything represented is represented simultaneously—is falsified too, for the same reason. Passage of a bright red fire-truck in front of you from t₁ to t₅ entails, via cca, that conscious experiences of the successive temporal parts of the passing truck—that is, the fire-truck at t₁, and then at t₂, and then at t₃, etc.—are consciously experienced after roughly the same interval (λ). Hence, an experience of the truck’s redness at l₁ will become conscious at t₁+λ, followed at t₂+λ by an experience of the truck’s redness at l₂, and so on.

As a result, the temporally relaxed version of retentionalism—that a perceptual experience with a temporally extended content can have a brief extension but still lack temporal structure and distinct temporal parts—is also contradicted by (6). An experience of the fire truck from t₁+λ to t₅+λ will have different successive temporal parts with distinct contents (representing the truck’s redness at different locations).

Finally, whether the modally relaxed version of R3—that an experience can represent the successive temporal parts of a temporally extended event simultaneously—is incompatible with (6), depends on how the modality expressed by the relevant “can” is interpreted. If it’s meant to express a “mere possibility”—for example, that there could have been some creatures with unextended or unstructured experiences, as in (m₂)—then there’s no inconsistency between (6) and the modally relaxed version of R3. The fact that our experiences do not actually represent the successive temporal parts of extended events simultaneously doesn’t rule out that the experiences of other possible creatures—including possible humans with different kinds of brains—could have. Of course, such a possibility is rather uninformative in this context: so construed, the modally relaxed version of R3 may be true, yet tells us hardly anything about our actual experiences. If, on the other hand, the possibility in question is meant to be restricted to creatures like us, with the kinds of brain we actually have, in the conditions in which our actual brains usually operate—as in (m₃) or (m₄)—then it looks as though such a possibility is ruled out by (6): cca is grounded in evidence about our actual neural processes, and implies that the perceived successive temporal parts of a moving fire-truck are not neurally processed so as to appear simultaneously in consciousness.⁴⁷

In short, every substantive interpretation of the third retentionalist thesis (R3) is undermined by cca, the alternative to partially delayed processing (pdp).

3.3. Response #1: Re-Description Strategies

As I said, the argument just articulated is hardly surprising. The interesting question is: is there room for retentionalists to resist it—without, that is, merely rejecting cca by digging their heels in about pdp? A retentionalist could insist that most experiences in (1) to (5) above ought to be ascribed, at the very least, a temporally extended content. And indeed, nothing in the description from (1) to (5) above precludes the retentionalist from offering a more detailed description of the relevant experiences and their respective contents. For instance,

• (1)     at t₁+λ, S has a conscious visual experience of the fire-truck’s redness at l₁.

• (2*)    at t₂+λ, S has a conscious visual experience of the fire-truck’s redness at l₁ & at l₂.

• (3*)    at t₃+λ, S has a conscious visual experience of the fire-truck’s redness at l₂ & at l₃.

• (4*)    at t₄+λ, S has a conscious visual experience of the fire-truck’s redness at l₃ & at l₄.

• (5*)    at t₅+λ, S has a conscious visual experience of the fire-truck’s redness at l₄ & at l₅.

This isn’t the only way in which to sketch a proposal of this sort. Perhaps, some such contents are more extended than others: perhaps, the experience at t₅+λ represents the whole sequence from t₁ to t₅, say.

No matter how the details are worked out, the suggestion risks running afoul of the retentionalist’s “no-dependence-on-memory” desideratum on temporally extended perceptual contents. For instance, the experience at t₃+λ represents some of the same events (i.e., the truck being red at l₂) already represented in the experience at t₂+λ. Hence, the content of the later experience (3*) appears to “retain” part of the content of the preceding one (2*) in consciousness (in the sense specified by Lee—§1).⁴⁸

Worse, what the retentionalist ends up with, in any case, is still an extended experience temporally structured by way of its distinct temporal parts—contra R3—owing to the distinct extended contents in the succession from (1) to (5*). What this shows, interestingly, is that claims about the temporally extended contents of perceptual experiences—that is, theses R1 and R2 in the retentionalist view—can be severed from thesis R3 about the temporal ontology of experience, once partially delayed processing (pdp) is replaced by something like cca. This serves to highlight the rather crucial role pdp plays, even if usually left unarticulated, in tying together the different strands of the retentionalist view.

3.4. Response #2: Integrated Processing of Temporal Features

Another facet of the causal argument worth noting: cca does not clash in any way with the more compelling considerations advanced to motivate the retentionalist view. Hence, it seems such considerations are impotent in blocking the argument in §3.2.

For example, one of the most interesting and promising attempts in this regard is Geoffrey Lee’s (2014a; 2014b) appeal to Reichardt detectors (2014a: 14): a simple model of a processing mechanism to explain motion perception, composed of one detector tracking edges at one location in the visual field and another detector tracking edges at another location. The first detector will be triggered by a moving stimulus first, followed by the second detector when the stimulus reaches the area it is sensitive to. But the information processed by the first detector must somehow be delayed until the second detector has been stimulated and its output processed, so that both pieces of information can be “integrated” in order to represent motion:

Suppose we have temporal information at the periphery that is contained in a temporally extended pattern of receptor stimulation, say on the retina. In order for the representation of this information to causally impact post-perceptual processing, each relevant temporal part of the initial extended stimulation has to leave a trace in the brain. If you consider the process leading up to, say, a verbal report of the information, you can see that each of these traces will have to be simultaneously present before the report is made: the alternative is that traces from certain parts of the stimulus no longer exist, and therefore can have no causal impact on the report. Furthermore, the traces will have to be integrated in the right way for simultaneous representation of each temporal stage of the input to able to control later processes in an appropriate way: if each relevant stage of the input leaves a trace but the traces are in completely different neural populations that aren’t functionally integrated, the information is not present in a useful form; it is not “explicit” in the relevant sense. For example, you can see in the motion detection example that if the representation of motion is going to have an appropriate later effect, the triggering of the first detector has to cause a trace that is integrated with the trace from the triggering of the second detector. Models of temporal computation implicitly assume that setting up such traces and then simultaneously integrating them is the task that the brain has to perform. (Lee 2014a: 14–15)⁴⁹

Grant Lee’s very plausible rationale about the need for informational integration, as well as the rest of his argument. What does this show à propos cca? Very little, I think.

True, if temporal features such as motion and duration are represented in conscious sensory experiences, the neural processing of such features must involve integration of locational information (e.g., that x is at l₁ at t₁, that x is F at l₁ at t₁, etc.) from successive stages of a moving or extended stimulus. And such integration will undoubtedly eventuate in some processing delay of the earlier stages. But notice: this is a point about neural latencies, that is, about the necessary processing requirements for the representation of certain types of features (indeed, it can serve to explain why motion and its direction incur longer neural latencies than, say, colour). It is not, in and of itself, a point about online latencies—about how and when our brains generate conscious experiences, which is what cca is concerned with.

Thus, integration of locational information from successive presentations of a moving stimulus for the purpose of representing its motion will entail a processing delay. Yet, during the delay in question, the very same locational information (e.g., that x is at l₁ at t₁, or that x is F at l₁ at t₁) can continue to be processed further in the neural hierarchy. This means that, in principle, such locational information could well become conscious even before its integration for motion representation has taken place. Motion representation, after all, is just one processing function: it needn’t signify the end of processing for a given stimulus feature involved in motion processing.

Processing models such as Reichardt detectors are thus perfectly at home with cca—and with the evidence from visual asynchrony. The different latencies required by each apply to the production of outputs of different types, such as (i) a neural representation of feature F (e.g., motion) at some level of processing, and (ii) a conscious experience of feature G (e.g., location or colour). To represent x’s motion, the relevant detectors must indeed integrate information from x’s location l₁ at t₁ with information from x’s location l₂ at t₂, and so wait, by neural delay λ_M, for the latter. As a result, the neural latency λ_M is likely longer than the neural latency λ_C which characterizes the processing of x’s colour at t₁. But if λ_C remains constant through processing of x’s colour at t₁, and then at t₂, and t₃, etc., so can λ_C in cca, which sets the onset of conscious colour experience from t₁ + λ_C to t₃ + λ_C, etc.—where interval λ_C in cca, recall, covers both the neural latency λ_C and the online latency (if any) for conscious experience of colour.

In fact, cca applies to λ_M just as well: if it takes λ_M to neurally process x’s motion from t₁ to t₂, it also takes λ_M to process x’s motion from t₂ to t₃, and then again from t₃ to t₄, etc. Hence, λ_M, which determines the onset of the relevant motion experience in consciousness, can very well remain constant from t₁ + λ_M to t₂ + λ_M, t₃ + λ_M, etc.—again, whether λ_M > λ_M or λ_M = λ_M. In which case, successive presentations of a moving stimulus will give way to successive conscious experiences with different contents, by cca—the kind of delayed processing at play in Reichardt detectors isn’t enough to rescue the retentionalist thesis R3.