2. Mental Time Travel
The word “remembering” is used loosely and imprecisely. There are actually multiple different types of memory; for example, episodic memory and semantic memory.
Episodic memory: The memory of actual events located in time and space, i.e “reminiscing.”
Semantic memory: The storehouse of knowledge that we possess, but which does not involve any kind of conscious recollection.
Semantic memory refers to general world knowledge that we have accumulated throughout our lives. This general knowledge (facts, ideas, meaning and concepts) is intertwined in experience and dependent on culture.
Semantic memory is distinct from episodic memory, which is our memory of experiences and specific events that occur during our lives, from which we can recreate at any given point. For instance, semantic memory might contain information about what a cat is, whereas episodic memory might contain a specific memory of petting a particular cat.
We can learn about new concepts by applying our knowledge learned from things in the past. The counterpart to declarative or explicit memory is nondeclarative memory or implicit memory.
Semantic memory (Wikipedia)
Episodic memory is essential for creating of the narrative self. Episodic memory takes various forms, for example:
Specific events: When you first set foot in the ocean.
General events: What it feels like stepping into the ocean in general. This is a memory of what a personal event is generally like. It might be based on the memories of having stepped in the ocean, many times during the years.
Flashbulb memories: Flashbulb memories are critical autobiographical memories about a major event.
Episodic Memory (Wikipedia)
For example, if you are taking a test for school, you are probably not reminiscing about the study session you had the previous evening, or where you need to be the next class period. You are probably not thinking about your childhood, or about the fabulous career prospects that are sure to result from passing this test. Those episodic memories—inserting yourself into past or future scenarios—would probably be a hindrance from the test you are presently trying to complete. Semantic memory would be what you are drawing upon to answer the questions (hopefully correctly).
It is often difficult to distinguish between one and the other. Autobiographical memories are often combinations of the two—lived experience combined with autobiographical stories and family folklore. Sometimes, we can even convince ourselves that things that didn’t happen actually did (false memories). Our autobiographical sense of self is determined by this process.
Endel Tulving has described remembering as autonoetic, or self-knowing, in that one has projected one’s self into the past to re-experience some earlier episode. Simply knowing something, like the boiling point of water, is noetic, and implies no shift of consciousness. Autoneotic awareness, then, is is recursive, in that one can insert previous personal experience into present awareness. This is analogous to the embedding of phrases within phrases, or sentences within sentences.
Deeper levels of embedding are also possible, as when I remember yesterday that I had remember yesterday that I had remembered an event that occurred at some earlier time. Chunks of episodic awareness can thus be inserted into each other in recursive fashion. Having coffee at a conference recently, I was reminded of an earlier conference where I managed to spill coffee on a distinguished philosopher. This is memory of a memory of an event. I shall suggest later that this kind of embedding may have set the state for the recursive structure of language itself (p. 85) [Coincidentally, as I was typing this paragraph, I spilled coffee on the book. Perhaps you will spill coffee on your keyboard while reading this. – CH]
Corballis mentions that case of English musician Clive Wearing, whose hippocampus was damaged leading to anteriograde and retrograde amnesia. At the other end of the spectrum is the Russian Solomon Shereshevsky.
The Abyss (Oliver Sacks, The New Yorker)
Long-term memory can further be subdivided into implicit memory and explicit (or declarative) memory.
“Implicit memories are elicited by the immediate environment, and do not involve consciousness or volition.” (p. 98) … Implicit memory…enables us to learn without any awareness that we are doing so. It is presumably more primitive in an evolutionary sense than is explicit memory, which is made up of semantic and episodic memory. Explicit memory is sometimes called declarative memory because it is the kind of memory we can talk about or declare.
Implicit memory does not depend on the hippocampus, so amnesia resulting from hippocampal damage does not entirely prevent adaptation to new environments or condition, but such adaptation does not enter consciousness. p. 88 (emphasis mine)
Explicit memories, by contrast, “provide yet more adaptive flexibility, because it does not depend on immediate evocation from the environment” p. 98 (emphasis mine)
The textbook case of implicit memory is riding a bicycle. You don’t think about, or ponder how to do it, you just do it. No amount of intellectual thought and pondering and thinking through your options will help you to swim or ride a bike or play the piano. When a line drive is hit to the shortstop, implicit memory, not explicit memory catches the ball (although the catch might provide a nice explicit memory for the shortstop later on). A daydreaming shortstop wold miss the ball completely.
Words are stored in semantic memory, and only rarely or transiently in episodic memory. I have very little memory of the occasions on which I learned the meanings of the some 50,000 words that I know–although I can remember occasionally looking up obscure words that I didn’t know, or that had escaped my semantic memory. The grammatical rules by which we string words together may be regarded as implicit rather than explicit memory, as automatic, perhaps, as riding a bicycle. Indeed, so automatic are the rules of grammar that linguists have still not been able to elaborate all of them explicitly. p. 126 (emphasis mine)
Operant conditioning (also called signal learning, solution learning, or instrumental learning) is another type of learning that does not require conscious, deliberative thought. It is a simple stimulus and response. You touch the stove, and you know the stove is hot. There was no thinking involved when Pavlov’s dogs salivated at the sound of a bell, for example. In a very unethical experiment, the behaviorist John B. Watson took a nine-month old orphan and conditioned him to be afraid of rats, rabbits, moneys, dogs and masks. He did this by making a loud, sharp noise (banging a metal bar with a hammer), which the child was afraid of, whenever the child was presented with those things. By associating the sound with the stimulus, he was able to induce a fear of those items. But there was no volition; no conscious thought was involved in this process. It works the same way on dogs, rabbits, humans or fruit flies. Behvariorism tells us next to nothing about human consciousness, or what makes us different.
These types of conditioning may be said to fall under the category of implicit memory. As we have seen, implicit memory may also include the learning of skills and even mental strategies to cope with environmental challenges. Implicit memories are elicited by the immediate environment, and do not involve consciousness or volition. Of course, one may remember the experience of learning to ride a bicycle, but that is distinct from the learning itself…These are episodic memories, independent of the process of actually learning (more or less) to ride the bike. p. 98 (emphasis mine, italics in original)
This important distinction is what is behind Jaynes’s declaration that learning and remembering do not require consciousness. Implicit memory and operant conditioning do not require the kind of deliberative self-consciousness or “analog I” that Jaynes described. Even explicit memory—the ability to recall facts and details, for example—does not, strictly speaking, require deliberative self-consciousness. Clive Wearing, referred to above, could still remember how to play the piano, despite living in an “eternal present.” Thus, it is entirely possible that things such as ruminative self-consciousness emerged quite late in human history. Jaynes himself described why consciousness (as distinct from simply being functional and awake) is not required for learning, and can even be detrimental to it.
In more everyday situations, the same simple associative learning can be shown to go on without any consciousness that it has occurred. If a distinct kind of music is played while you are eating a particularly delicious lunch, the next time you hear the music you will like its sounds slightly more and even have a little more saliva in your mouth. The music has become a signal for pleasure which mixes with your judgement. And the same is true for paintings. Subjects who have gone through this kind of test in the laboratory, when asked why they liked the music or paintings better after lunch, could not say. They were not conscious they had learned anything. But the really interesting thing here is that if you know about the phenomenon beforehand and are conscious of the contingency between food and the music or painting, the learning does not occur. Again, consciousness reduces our learning abilities of this type, let alone not being necessary for them…
The learning of complex skills is no different in this respect. Typewriting has been extensively studied, it generally being agreed in the worlds of one experimenter “that all adaptations and short cuts in methods were unconsciously made, that is, fallen into by the learners quite unintentionally.” The learners suddenly noticed that they were doing certain parts of the work in a new and better way.
Another simple experiment can demonstrate this. Ask someone to sit opposite you and to say words, as many words as he can think of, pausing two or three seconds after each of them for you to write them down. If after every plural noun (or adjective, or abstract word, whatever you choose) you say “good” or “right” as you write it down, or simply “mmm-hmm” or smile, or repeat the plural word pleasantly, the frequency of plural nouns (or whatever) will increase significantly as he goes on saying the words. The important thing here is that the subject is not aware that he is learning anything at all. He is not conscious that he is trying to find a way to make you increase your encouraging remarks, or even of his solution to that problem. Every day, in all our conversations, we are constantly training and being trained by each other in this manner, and yet we are never conscious of it. OoCitBotBM; pp. 33-35
But we not only use our memory to recall past experiences, we also think about future events as well, and this is based on the same ability to mentally time travel. It may seem paradoxical to think of memory as having anything to do with events that haven’t happened yet, but brain scans show that similar areas of the brain are activated when recalling past events and envisioning future ones—particularly the prefrontal cortex, but also parts of the medial temporal lobe. There is slightly more activity in imagining future events, probably due to the increased creativity required of this activity.
So far, there is little convincing evidence that animals other than humans are capable of mental time travel—or if they are, their mental excursions into past or future have little of the extraordinary flexibility and broad provenance that we see in our own imaginative journeys. The limited evidence from nonhuman animals typically comes from behaviors that are fundamentally instinctive, such as food caching or mating, whereas in humans mental time travel seems to cover all aspects of our complex lives. p. 112
However, see: Mental time-travel in birds (Science Daily)
We are always imagining and anticipating, from thinking about events later the same day, or perhaps years from now. Even in a conversation, we are often planning what we are about to say, rather than focusing on the conversation itself. That is, we are often completely absent in the present moment, which is something that techniques like mindfulness meditation are designed to mitigate. We can even imagine events after we are dead, and it has been argued that this knowledge lays behind many unique human behaviors such as the notion of an afterlife and the idea of religion more generally. The way psychologists study this is to use implicit memory (as described above) to remind people of their own mortality. This is done through a technique called priming:
Priming is remarkably resilient. In one study, for example, fragments of pictures were used to prime recognition of whole pictures of objects. When the same fragments were shown 17 years later to people who had taken part in the original experiment, they were able to write the name of the object associated with each fragment much more accurately than a control group who had not previously seen the fragments. p. 88
When primed with notions of death and their own mortality, it has been shown that people in general are more authoritarian, more aggressive, more hostile to out-groups and simultaneously more loyal to in-groups. Here’s psychologist Sheldon Solomon describing the effect in a TED Talk:
“Studies show that when people are reminded of their mortality, for example, by completing a death anxiety questionnaire, or being interviewed in front of a funeral parlor, or even exposed to the word ‘death’ that’s flashed on a computer screen so fast—28 milliseconds—that you don’t know if you’ve even seen anything—When people are reminded of their own death, Christians, for example, become more derogatory towards Jews, and Jews become more hostile towards Muslims. Germans sit further away from Turkish people. Americans who are reminded of death become more physically aggressive to other Americans who don’t share their political beliefs. Iranians reminded of death are more supportive of suicide bombing, and they’re more willing to consider becoming martyrs themselves. Americans reminded of their mortality become more enthusiastic about preemptive nuclear, chemical and biological attacks against countries who pose no direct threat to us. So man’s inhumanity to man—our hostility and disdain toward people who are different—results then, I would argue, at least in part from our inability to tolerate others who do not share the beliefs that we rely on to shield ourselves from mortal terror.”
Humanity at the Crossroads (YouTube)
One important aspect of episodic memory is that it locates events in time. Although we are often not clear precisely when remembered events happened, we usually have at least a rough idea, and this is sufficient to give rise to the general understanding of time itself. It appears that locating events in time and in space are related.
Episodic memory allows us to travel back in time, and consciously relive previous experiences. Thomas Suddendorf called this mental time travel, and made the important suggestion that mental time travel allows us to imagine future events as well as remember past ones. It also adds to the recursive possibilities; I might remember, for example, that yesterday I had plans to go to the beach tomorrow.The true significance of episodic memory, then is that it provides a vocabulary from which to construct future events, and so fine-tune our lives.
What has been termed episodic future thinking, or the ability to imagine future events, emerges in children at around the same time as episodic memory itself, between the ages of three and four. Patients with amnesia are as unable to answer questions about past events as they are to say what might happen in the future… p. 100
Once again, the usefulness of this will be determined by the social environment. I will argue later that this ability to mentally time travel, as with the ability to “read minds” (which we’ll talk about next) became more and more adaptive over time as societies became more complex. For example, it would play little to no role among immediate return hunter gatherers (such as the Pirahã), who live mostly in the present and do not have large surpluses. Among delayed return hunter gatherers and horticulturalists, however, it would play a far larger role.
When we get to complex foragers and beyond, however, the ability to plan for the future becomes almost like a super-power. And here, we see a connection I will make between recursion and the Feasting Theory we’ve previously discussed. Simply put, an enhanced sense of future states allows one to more effectively ensnare people in webs of debt and obligation, which can then be leveraged to gain wealth and social advantage. I will argue that this is what allowed the primordial inequalities to form in various societies which could produce surpluses of wealth. It also demonstrates the evolutionary advantages of recursive thinking.
Corballis then ties together language and mental time travel. He posits that the recursive nature of language evolved specifically to allow us to share past and future experiences. It allows us to narratize our lives, and to tell that story to others, and perhaps more importantly, to ourselves.
Language allows us to construct things that don’t exist—shared fictions. It allows us to tell fictional stories of both the past and the future.
Episodic memories, along with combinatorial rules, allow us not only to create and communicate possible episodes in the future, but also to create fictional episodes. As a species, we are unique in telling stories. Indeed the dividing line between memory and fiction is blurred; every fictional story contains elements of memory, and memories contain elements of fiction…Stories are adaptive because they allow us to go beyond personal experience to what might have been, or to what might be in the future. They provide a way of stretching and sharing experiences so that we are better adapted to possible futures. Moreover, stories tend to become institutionalized, ensuring that shared information extends through large sections of the community, creating conformity and social cohesion. p. 124
The main argument … is that grammatical language evolved to enable us to communicate about events that do not take place in the here and now. We talk about episodes in the past, imagined or planned episodes in the future, or indeed purely imaginary episodes in the form of stories. Stories may extend beyond individual episodes, and involve multiple episodes that may switch back and forth in time. The unique properties of grammar, then, may have originated in the uniqueness of human mental time travel…Thus, although language may have evolved, initially at least, for the communication of episodic information, it is itself a robust system embedded in the more secure vaults of semantic and implicit memory. It has taken over large areas of our memory systems, and indeed our brains. p. 126
The mental faculties that allow us to locate, sort and retrieve events in time, are apparently use the same ones that we use to locate things in space. Languages have verb tenses that describe when things took place (although a few languages lack this ability). The ability to range at will over past, present and future gave rise to stories, which are often the glue that holds societies together, such as origin stories or tales of distant ancestors. Is the image above truly about moving forward in space, or is it about something else? What does it mean to say things like we “move forward” after a tragedy?
Different sets of grid cells form different grids: grids with larger or smaller hexagons, grids oriented in other directions, grids offset from one another. Together, the grid cells map every spatial position in an environment, and any particular location is represented by a unique combination of grid cells’ firing patterns. The single point where various grids overlap tells the brain where the body must be…Since the grid network is based on relative relations, it could, at least in theory, represent not only a lot of information but a lot of different types of information, too. “What the grid cell captures is the dynamic instantiation of the most stable solution of physics,” said György Buzsáki, a neuroscientist at New York University’s School of Medicine: “the hexagon.” Perhaps nature arrived at just such a solution to enable the brain to represent, using grid cells, any structured relationship, from maps of word meanings to maps of future plans.
It is likely that a dog, or even a bonobo, does not tell itself an ongoing “story” of it’s life. It simply “is”. If we accept narratization as an important feature of introspective self-consciousness, then we must accept the ability to tell ourselves these internal stories is key to the creation of that concept. But when did we acquire this ability? And is it universal? Clearly, it has something to do with the acquisition of language. And if we accept a late origin of language, it certainly cannot have arisen more than 70-50,000 years before present. To conclude, here is an excerpt from a paper Corballis wrote for the Royal Society:
…the evolution of language itself is intimately connected with the evolution of mental time travel. Language is exquisitely designed to express ‘who did what to whom, what is true of what, where, when and why’…and these are precisely the qualities needed to recount episodic memories. The same applies to the expression of future events—who will do what to whom, or what will happen to what, where, when and why, and what are we going to do about it…To a large extent, then, the stuff of mental time travel is also the stuff of language.
Language allows personal episodes and plans to be shared, enhancing the ability to plan and construct viable futures. To do so, though, requires ways of representing the elements of episodes: people; objects; actions; qualities; times of occurrence; and so forth…The recounting of mental time travel places a considerable and, perhaps, uniquely human burden on communication, since there must be ways of referring to different points in time—past, present and future—and to locations other than that of the present. Different cultures have solved these problems in different ways. Many languages use tense as a way of modifying verbs to indicate the time of an episode, and to make other temporal distinctions, such as that between continuous action and completed action. Some languages, such as Chinese, have no tenses, but indicate time through other means, such as adverbs or aspect markers. The language spoken by the Pirahã, a tribe of some 200 people in Brazil, has only a very primitive way of talking about relative time, in the form of two tense-like morphemes, which seem to indicate simply whether an event is in the present or not, and Pirahã are said to live largely in the present.
Reference to space may have a basis in hippocampal function; as noted earlier, current theories suggest that the hippocampus provides the mechanism for the retrieval of memories based on spatial cues. It has also been suggested that, in humans, the hippocampus may encompass temporal coding, perhaps through analogy with space; thus, most prepositions referring to time are borrowed from those referring to space. In English, for example, words such as at, about, around, between, among, along, across, opposite, against, from, to and through are fundamentally spatial, but are also employed to refer to time, although a few, such as since or until, apply only to the time dimension. It has been suggested that the hippocampus may have undergone modification in human evolution, such that the right hippocampus is responsible for the retrieval of spatial information, and the left for temporal (episodic or autobiographical) information. It remains unclear whether the left hippocampal specialization is a consequence of left hemispheric specialization for language, or of the incorporation of time into human consciousness of past and future, but either way it reinforces the link between language and mental time travel.
The most striking parallel between language and mental time travel has to do with generativity. We generate episodes from basic vocabularies of events, just as we generate sentences to describe them. It is the properties of generativity and recursiveness that, perhaps, most clearly single out language as a uniquely human capacity. The rules governing the generation of sentences about episodes must depend partly on the way in which the episodes themselves are constructed, but added rules are required by the constraints of the communication medium itself. Speech, for example, requires that the account of an event that is structured in space–time be linearized, or reduced to a temporal sequence of events. Sign languages allow more freedom to incorporate spatial as well as temporal structure, but still require conventions. For example, in American sign language, the time at which an event occurred is indicated spatially, with the continuum of past to future running from behind the body to the front of the body.
Of course, language is not wholly dependent on mental time travel. We can talk freely about semantic knowledge without reference to events in time… However, it is mental time travel that forced communication to incorporate the time dimension, and to deal with reference to elements of the world, and combinations of those elements, that are not immediately available to the senses. It is these factors, we suggest, that were in large part responsible for the development of grammars. Given the variety of ways in which grammars are constructed, such as the different ways in which time is marked in different languages, we suspect that grammar is not so much a product of some innately determined universal grammar as it is a product of culture and human ingenuity, constrained by brain structure.
Mental time travel and the shaping of the human mind (The Royal Society)
Next time, we’ll take a look at another unique recursive ability of the human mind: the ability to infer the thoughts and emotions of other people, a.k.a. the Theory of Mind.