The Quest for Consciousness: A Neurobiological Approach

[00:00:00] HOST:
We can’t produce seats from nowhere, but, uh, I’m sure you’ll be intrigued enough not to mind the fact that you’re standing. Uh, these lectures are also sponsored by the Graduate Division, and, uh, my job is really just to, uh, uh, set the thing going. My colleague, George Sensabaugh, will introduce our distinguished speaker in a moment.

If you glance at your program, you’ll see it’s stated how this lectureship came to be established, uh, Edith Zweybruck had been a, a school teacher in San Francisco public school system. Uh, she had a sister, Agnes, who was married to a man called Constantine Foerster. And, uh, when, uh, the, her sister and sister, sister husband, uh, uh, brother-in-law died, she established this lectureship in their honor and memory, uh, requiring that it be delivered on the immortality of the soul, or, and I love the words that come after or, kindred spiritual subjects.

Now, rumor has it, there are people in this room who could confirm it, that actually one of the lectures in the last thirty years was on the

[00:01:22] TONY:
mortality of the soul.

(laughter)

Um, what our very intelligent speaker of today has noticed, and it, it’s quite interesting, I think, to notice that if you look at the back page of your program, you’ll see that in the early days there were reverends, even a bishop, who gave these lec- this lecture. Uh, in more recent years, it’s tended to move much more towards the sciences, and uh, that’s, I think, interesting in its own right. We are very pleased indeed to welcome our speaker today, uh, uh, Christof Koch.

Uh, he is another of those who will be interpreting the subject broadly, and I’m going to now ask George, George Sensabaugh, my, my colleague on the Foerster Lectures Committee, to introduce Christof Koch.

[00:02:16] GEORGE SENSABAUGH:
Thank you, Tony. Uh, it’s my pleasure to introduce today’s speaker. Uh, let me give you a little, uh, biographical information first.

Uh, uh, Professor Koch was born in Kansas City but grew up internationally. Uh, he took his first degree from, uh, the Lycée Descartes in Morocco. Uh, that obviously had a great influence on him because he’s been interested in mind-body problems ever since.

He’s probably heard this joke many times. Uh, the, uh, he pursued, uh, graduate study in Tübingen in Germany. He received his PhD in biophysics from the Max Planck Institute of Biological Cybernetics in 1982.

He then came to the United States, spending four years at MIT in the Artificial Intelligence Laboratory in the Department of Psychology. In 1986, he was recruited to Caltech, uh, he– where he joined the, uh, the then new, uh, uh, Institute for the, uh, Computation and Neural Sciences. Uh, he is also, uh, he remains associated with that, uh, center, but also is associated with the Sloan Center for Theoretical Neuroscience and the Center for Neuromorphic Systems Engineering.

So you can see that he has quite a, uh, a broad base, uh, uh, of, uh, interests. Um, since two thousand, he has held the, uh, the professorship, the Lois and Victor, uh, uh, uh, Troendle, uh, Professorship of Cognitive and Behavioral Biology at Caltech. He has been the recipient of a number of honors and awards, including the Presidential Young Investigator Award, uh, uh, in nineteen eighty-eight and the Alexander von Humboldt Research Prize in nineteen ninety-seven.

Uh, his, uh, primary, uh, research interests are the, uh, biophysics of computation and the neuronal basis of consciousness. He’s described his work in a recent book, The Quest for Consciousness: Uh, A Neurobiological Approach, and that, of course, is the study or is the title of his talk today. His, uh, fellow traveler in this quest for consciousness for many years, sixteen, uh, or so, was Francis Crick, who, uh, by the time, uh, Christof came to Caltech, had previously moved to the Salk Institute in La Jolla, so they were about two hours away from each other.

Uh, their, uh, collaboration ended only with Crick’s, uh, death in 2004. Uh, Professor, uh, Koch introduces the first chapter of his book, The Quest for Consciousness, with a comment from the philosopher Thomas Nagel. “Consciousness is what makes the mind-body problem really intractable.

Without consciousness, the mind-body problem would be much less interesting. With consciousness, it seems hopeless.” Uh, his lecture today, uh, I think will, uh, proposes a path to lead us out of the trough of hopelessness.

(applause and cheering)

Yeah.

[00:05:50] CHRISTOF KOCH:
All right. Thank you much, George. Thank you much, Tony, for that introduction. So despite being born in the Midwest, I talk like our governor does.

(laughter)

And, uh, dear students, dear colleagues. Um, yeah, so I’ll, um, I’ll just come right to the point. So I’m interested in, in, in this most, um, mysterious aspect of the mind-body problem, what philosophers call qualia.

So qualia, just the various elements that make up our conscious sensation. There’s a qualia for red, there’s a qualia for pain, there’s a qualia for being Christof, there’s a qualia for being angry, and they all share these different qualia, what they all share is the subject that stands, And, um, Francis and, um, and my claim has always been that if we want to have, um, that if we believe that science should have a complete description of everything in the universe, then that description has to include a theory of consciousness. Uh, otherwise, we’re just missing something, a ma– uh, otherwise, we’re just missing a major aspect of reality.

Yes, it’s difficult studying consciousness. It’s not like doing, uh, physics, uh, uh, because of the, the, the problem of the, um, of the first person account, but that doesn’t mean it’s impossible to solve, and we should get to it in an, in an empirical, um, um, manner. So as George mentioned this, the conceptual part of this has been carried out with Francis Crick, and I wanted to give you a short clip.

This, this clip of a video was done shortly before his, um, his death at age eighty-eight.

[00:07:09] FRANCIS CRICK:
Well, there are various things people want to know, but the general questions is what is the nature of memory? We already know that there are several types of memory. The, the, the one way of expressing it is that the memory of knowing what you had for breakfast is not the same type of memory as the memory of the work– the word breakfast means, or the third type, the memory of how you eat your breakfast.

Those are different sorts of memories. We even have a beginning of understanding some of the molecular, some of the key molecules underlying one of those sorts of memories, one of the so-called glutamate receptors and so on. That’s one sort of problem.

But perhaps the key problem is how is what is called consciousness, and in particular, to take a simple example, how it is we see things. How you’re aware of the visible world outside. Why is it you don’t just behave as an automaton with no picture of the world in your head as it were, but just simply behave?

As you can have or, or automata in, in factories nowadays, uh, making motor cars and things of that sort. They sense what the thing is in, and then they make the right operations, and so on, but there’s no evidence that they’ve got any awareness. So that, again, is one of the, one of the key problems.

And, uh, and then the more elaborate forms of awa-awareness, like self-awareness, or it’s perhaps one people would say the simpler ones, like feeling pain, for example. So those are the major, major problems. How you-

[00:08:43] CHRISTOF KOCH:
So, so essentially he gave, uh, the, the summary of our approach that we study visual consciousness because it’s um the most accessible. We can manipulate in a lab. I’ll show you a few illusions.

We can now do what magicians can do. We can routinely manipulate what you see. Uh, although you may be looking at something, you don’t see it, just like in a magic show.

And we– so we, we can manipulate your conscious perception in space and time with great precision. We cannot do that as well as for more elaborate forms of consciousness like self-consciousness or like pain. It’s much more difficult to recruit subjects to do really experiments with pain consciousness for obviously a reason.

So that’s why we, we study, I mean, p-personally, I study and most people in the field now study, study visual consciousness. So first of all, conceptual distinction. There are, there are at least two different usage of the word consciousness.

First of all, it relates to different states of conscious. So hopefully at this point, you’re still all conscious. Nobody’s asleep, at least yet.

So that’s one form of consciousness, and then, of course, you can up and down regulate that. You can be in, in sleep, in slow-wave sleep, or in, in REM sleep. You can be in coma, you can be in persistent vegetative s-syndrome, like a state like, like Terri Schiavo was.

So that those are different states of consciousness, and it’s a great clinical concern to understand the neural basis of that. Um, then the other form of consciousness is the, um, relates to the current content of consciousness. So right now, your consciousness should be filled with my voice.

Or you may, you know, you may have, uh, I don’t know, your little toe hurts, and so the content of your consciousness consists of the, the qualia that you have, the pain in your little toe. Or you may worry about, you know, tax da- tax day is approaching, so your content of consciousness is filled with the, with, with, you know, worrying about taxes. So the– all these different things come with this qualia, and so the question is very simply, what is in the brain that gives rise to these conscious sensations?

Because as, uh, Francis said, many things, in fact, most things in our brain, most things in our body go on perfectly well without consciousness. Uh, and the second part is m– is easier to investigate, um, than the first part. The f– second part, as I mentioned, we can manipulate quite well now the content of consciousness, very precise in space and time.

This one is much more difficult to manipulate. I mean, you go to bed, of course, each night and consciousness is turned off. But it’s not a very clean transition.

It takes time, and all sorts of other things happen at the same time. You can’t move, for instance.

(laughter)

Your memory goes away, and so it’s not a very clean experimental way to do this.

(cough)

So what are some of the things that we know for sure? Well, so consciousness is associated with certain types of complex adaptive networks with massive feedback. If you look at the networks that, that ha– that have consciousness associated with them, they all have they’re– very adaptive, they’re very complex, they have a high degree of, of, uh, of feedback, and they’re shaped by natural selection.

So we obviously think, like most of us, that consciousness gives the organism which is conscious some evolutionary advantage. That’s why consciousness evolved. Now, it’s not associated with lots of other complex adaptive biological networks.

Like ma-many of you know– Well, some of you may know that down here in your gut, you have hundred million neurons. It’s called the enteric nervous system, and usually you’re totally and happily, by the way, oblivious of it.

Because if you are, uh, if you know something about, usually it’s you get feelings of nausea or other, uh, you know, um, other negative, um, uh, sensations. So here you have neurons, they have synapses, and they have neurotransmitter, but by and large, they do not give rise to consciousness. So we have to ask where’s the difference between the enteric nervous system and, and, and supposedly things in my head?

There are others complicated networks like the immune system. There’s no evidence that we’re conscious of any of the actions of our immune system, yet it’s highly complex. It learns.

It’s a biological network. It was shaped by force of natural selection. So we have to ask, why isn’t the immune system, um, uh, um, conscious?

So those are some of the basic questions we don’t know. W-we believe like, uh, again, like many biologists, that we sh– many species shares aspect of consciousness with humans, um, particularly if you broaden consciousness not only to include self-consciousness, but to include pain and pleasure and seeing and smelling and happiness and anger and all these other states. Um, why do we believe that?

Well, a similar behavior, right? If you look at a dog, I love dogs. If you look at dogs, the dog’s behavior is not identical to ours, but it’s quite similar.

The structure of the nervous system of a dog is very similar. If I give you a little cubic millimeter of dog brain, of macaque brain, and a human brain, only very few people, only specialized neuroscientists can tell the difference, because at the hardware level, there’s almost no difference. Human brain is bigger.

Of course, it’s not the biggest brain. That’s, that’s whales and dolphins. Um, but,

But if you look at the hardware itself, with one of few minor exceptions like spindle cells, it doesn’t look that different. And of course, that’s a large degree of evolutionary continuity. So the assumption is that not only you guys out there are conscious, because you gotta remember, I don’t know whether you’re really conscious, right?

I infer you’re conscious because again, you’re very similar to me, and you, you look very similar to, to, to me, and your brain is very similar to me. Well, we, uh, we just extend that argument a little bit to other creatures that maybe walk on four legs or that are a little bit more hairy. The thing is, um, uh, we don’t know the minimum size of a brain necessary to be conscious.

We really have no idea. So if you look at, for example, if you look at insects, most people instinctively say, “Well, that’s a bug,” and bugs are unconscious. I can just squish bugs.”

But if you look at the complex adaptive behavior of a bee, and if you look at what a bee is capable of in terms of single trial learning, very, a very adaptive behavior when it flies through mazes to find sugar. What the, the feats of, of, of sort of, quote, “intelligence” can do is quite prodigious. So it’s really– we really don’t know whether it doesn’t feel like something to be a bee.

So I think right now it, it shows up the, the pro– the, the problem that we don’t have a Turing test. We don’t have a clear test that tells us whether a creature is conscious. We think it’s conscious if it talks.

That’s been sort of the conventional criterion. Of course, many philosophers believe or many linguists believe that’s the, that’s the definition of consciousness. But that’s, of course, putting the, the horse before the cart.

Just befo– Uh, just because we talk and we are conscious doesn’t mean that consciousness is tied up with, uh, with or, Or it depends on language. Is there a way we can, uh, we can kill that light? Um, thank you.

So this is, um, you know, people say, “Well, l-let’s define consciousness.” Of co– And, you know, you have to define your terms before you do experiments. Of course, historically, nothing could be further from the truth.

In all sciences, you make progress in the absence of precise definition. I mean, look at today, the, the issue of how to define a planet, right? It hasn’t stopped people from finding planets.

So, um, if you, if you fix it– what you should do here is just fixate that cross. Don’t move your eyes. Keep your eyes open.

Fixate the cross, and and then tell me what you see. Yeah, somebody said already. What do you see?

[00:15:25] AUDIENCE MEMBER:
The squares disappear.

[00:15:26] CHRISTOF KOCH:
Do you all see that? So it works best if you don’t move your eyes. The less you move your eyes, the better it works.

And one square will disappear, or do– or, but sometimes both squares will disappear only for brief times, like two or three seconds. Most of you should be able to see that, otherwise you should come see me afterwards.

(laughter)

So here you have, So this is an illusion, in fact, that was described a few years ago here at, at Smith-Kettlewell in San Francisco. It’s called motion-induced blindness. So it’s one of many, many illusions we have in the visual domain where we can make things disappear.

Here, it’s, it’s not very well controlled. It depends on your eye movement. If you come to the lab, I can control it much better.

So the point is, the yellow squares are present all the time, yet sometimes you see ’em and sometimes you don’t see ’em. And when you see ’em, you’re conscious of them. You can talk to your neighbors about ’em.

They remind you of yellow. They remind you of the yellow sun, of yellow Van Gogh sunflowers, whatever you– That, that’s what we mean by consciousness. And sometimes they’re there, they’re on your retina, but you don’t see ’em.

You’re not conscious of them. So th-th-this is what sort of our quest reduces to. Where’s the difference in your brain between when you’re conscious of these and when you’re conscious and when you’re not conscious of these?

And the hypothesis is, it may be wrong, but the hypothesis is that once we understand something as simple as visual consciousness, we’re probably a great step further along the road to understand any form of, of consciousness, including self-consciousness or, and, and high-level aspects of consciousness. All right. So Francis and I have, have focused very early on, on– be-because this field has a two thousand year history of failure

(laughter)

Um, we, we, we want to focus on something that’s exp– that where we can actually make progress in, in our lifetime. And so most people agree that something we can make progress on, particularly given the state of neuroscience today, is the neuronal correlates of consciousness. That’s abbreviated as, um, as, um, as NCC here.

And, and the NCC are just the minimal neuronal mechanisms that are jointly that are necessary for any one specific conscious content. So minimal because we know the entire brain is necessary. But let’s say if you– for those yellow spots I showed you, do you need your eyes?

Well, you don’t– we know you don’t need your eyes to see because you can close your eyes and you can still imagine, I mean, most of us can still imagine the yellow squares. And of course, at night, most of us have very vivid visual dreams with your eyes closed. So that tells you, uh, for cer-certain forms of seeing, your, the eyes are, uh, obviously necessary, but for other forms of, of, uh, of seeing, the eyes are not necessary.

Is the cerebellum necessary? Well, probably not. There’s no evidence that the cur– cerebellum is necessary for any forms of consciousness.

So you can ask, where the neuronal mechanisms, where are the neurons, and what type of the brain are the, um, that, that are necessary for one specific co-conscious content, pain or pleasure, or seeing green or seeing yellow squares, whatever. Is there anything common about them? Do they– Are they located in a specific part of the brain?

Are they a specific type of cell, like a pyramidal cell? Do they sit in a particular type, let’s say a layer five and project forward? Are there particular molecular signatures for, uh, for them?

You can ask all, uh, all those questions once you have a neuronal correlate. And once again, you, you wanna distinguish this from the correlates that you need to study states of consciousness. Those are reasonably well understood, right?

From the forties and fifties, we know in the brainstem there’s a whole set of system that used to be called collectively the ascending arousal system, um, that, uh, that is– that’s necessary for you to be conscious at all. But here it’s much more specific. We’re interested in the NCCs for seeing blue or of her smelling mom’s apple pie.

And for every such conscious attribute, w– no matter what it is, you know, if you think, you know, if you have angst and alienation because we live in a post-industrial society, well, that’s a conscious feeling. There has to be a neural correlate for that. There has to be an NCC for that.

Now, as I mentioned, there are many, many brains inside your head. We know this, I mean this in a philosophical context. This was first raised by, by, uh, Schopenhauer and probably Nietzsche and then, of course, uh, Freud in a literary medical context.

But we know that from the labs, through the works of many people, things like climbing, running, driving, biking, um, chewing gum, uh, talking, all those things that you can do perfectly well, um, unconsciously. So in fact, when, when, when you give a talk like this or when you just have a normal conversations and you, you try to think about having the conversation while you’re actually having it, you realize that it’s not conscious. I mean, it’s not there’s an– there isn’t a little Christof inside my head who says, “Okay, this is what I wanna say.

This is the verb, this is the noun, this is in German, now translated, and then it’s sent it off to my, to my, to my speech apparatus.” It doesn’t happen like that. I have a vague idea what I wanna say next, And then the next thing I hear these words come pouring out of my mouth.

And it’s a very sophisticated thing, or reasonably sophisticated thing. And so this all happens without consciousness. Same thing, I open my eyes, within a fraction of a second, I see you all.

And we don’t have access. I can’t sort of, I can’t regress and, and you know, I can’t go to a psychoanalyst and pay two hundred bucks an hour and lie down and try to see how do I see in stereo or how do I see in color. It just happens.

It’s magic. It’s this wonderful interface. And so, um, those things that we are very proud of, those things we can introspect, those things that, that, you know, the adage of Western philosophy, know thyself, you know, try to figure out why you really do things.

Those things are only a very small fraction of all the activities in your brain that you’re conscious of. And we think, of course, they’re the most important ones because we are so proud of consciousness, because that’s how we experience world the world. But we have to ask scientifically, where’s the difference at the neuronal level?

I’m not going to talk about it today between these systems. These are very sophisticated systems. They involve subcortical structures, but they also involve cortical structures that are involved in things like playing soccer or running down.

I mean, you know, when you run in the mountains, it’s amazing. You can run at high speed down a mountain. You have to make very quick, um, you know, decisions where you put your feet in, in a fraction of a second, yet your, your body does it for you.

So there are very complicated computations that have to go on depending on the visual input and the state of my, my body, et cetera, yet you do it all. So the– you can ask the question, what’s the character of these systems as compared to the neuronal character of systems that actually give rise to consciousness? So z–

Francis and I call these zombie systems. These are not the phi– These are not These are not the philosophical zombies.

These are zo– These are different zombie systems that you have. They’re zombies in the sense that it doesn’t feel like anything. You can be, of course, conscious of, of running down, but you, but you can show in, in the lab that that consciousness comes after the fact that, you know, actually moving, you know, you can respond so quickly that the consciousness for that motion doesn’t come till after you’ve already initiated the, um, the motion.

So it has to work with, wi-without c-consciousness. So these, these conscious systems are, um, are complemented by a general purpose system whose function is, and this is not — this is our speculation, whose function is to summarize the current content of, of my environment and make this summary available to the planning stages of my brain. Like any information processing system, I’m the– I, I, I always suffer from informational overload.

There’s vastly more information streaming in through my various sensors, the auditory, the visual sensors, than I can possibly compute in real time. So what my brain does, it makes a quick summary of the environment. So it says there are people there, and there are walls here, I’m in an audience.

And then this summary is made accessible to my planning stages. So for instance, uh, well, let’s say if there’s a fire here or if there’s an earthquake, and I quickly have to think, you know, where, where– you know, what’s the best way to egress here, right? What’s the best way to exit?

That’s when I need consciousness for, for those sorts of things that I haven’t done over and over and over again. ’cause if I train it over and over again, I train up a zombie system. I probably train up my basal ganglia, and I don’t need consciousness, um, um, for that.

So you have these two sets of systems. You have the zombie system, and then there’s, is complemented by a more general-purpose conscious system that does this, uh, planning that involves the frontal lobes. Oh yeah, I found this wonderful quote from an old book, um, well it’s actually not that old, from an old book, um, Herrigel.

So he’s a German philosopher who went to, uh, before the war, who went to Japan and learned art then, and the art of, um, um, um, uh, um, bow, with these, with these big bows. And in the last chapter, he talks about, then, in the art of fencing, and he has this wonderful quote, “The pupil must develop a new sense, or more accurately, a new alertness of all of his senses, which will, in a sense, enable him to avoid dangerous thrusts as though he could feel them coming. Once he has mastered the art of evasion, he no longer needs to watch with undivided attention the movements of his opponent, or even of several opponents at once.

Rather, he sees and feels what is going to happen. At the same moment, he has already avoided its effect without there being the hair’s breadth between perceiving and avoiding. This then is what counts, a lightning reaction which has no further need of conscious observation.

In this respect, at least, the pupil makes himself independent of all conscious purpose, and that is a great gain. ‘Cause these things, these zombie systems only you have to train them up, and in these cases, these monks, of course, they train for years and years and years and years before they can do that. All right, so what’s the experimental program I’m going to talk about?

Well, so first of all, the point is consciousness is an empirical problem. It’s primarily an empirical problem. It used to be primarily a philosophical problem, but things have changed now, and there’s some very interesting philosophical, uh, work going on, but it’s,

But by and large, most of the progress now happens on the, on the empirical side. And the, the experimental program essentially is to track down the NCC to identify which neurons in what part of the brain, uh, for what’s, uh, you know, what species is involved in, in conscious perception, and then inactivating them, activating them, moving from– You wanna move from correlation to causation, of course. This is very difficult to do in humans.

That’s why you need animal models, right? You want to move from– You wanna s-n-not to say, “Well, when I’m conscious of red, those neurons fire,” but I wanna say, “If I can get those neurons to fire by manipulating them genetically or pharmacologically or ideo– iontophoretically or electrically, then I create the consciousness of red.” Right?

You wanna move from cor-correlation to causation, which you can do in humans, but of course it’s, it’s you can do it much more controlled in, in animals. And so if you wanna do these animal studies, then that really requires a bunch of practical tests. Just like the Turing test for intelligence, you want a battery of behavioral assays that tells you this individual, whether it’s a baby or whether it’s a fetus, or whether it’s um, a, a patient who’s aphasic, or whether it’s a patient who looks like he’s in coma or people like Terri Schiavo.

We want a behavioral assay or whether it’s a monkey or fly or any other organism, we w- We want to have a series of behavioral tests to tell us whether this, this behavior, this organ– this organism right now has conscious be– uh, has conscious sensation, has conscious behaviors. And again, you, you just do it by analogy with a, with, with a normal human.

You say, “Well, in normal humans,” you can do this battery of tests, and these tests require consciousness.” And then you apply this, this sort of appropriately adapted behavioral assays to, to other things like to babies or like to mice. All right, so let me show you, uh, a few effects.

So while I… You– What you should do, you just should look at the cross here.

Maybe we can once again turn off the light. Um, Ellen, I wonder whether we can turn once again the, the– Thank you. And just while I talk, just look at the cross here.

This is the oldest technology in– the oldest technique in psychology. Um, it’s the easiest to use, it’s the oldest, one of the best studied. And just keep your eyes fixed there.

Don’t move them. And then what do you see now? We can do this again.

So just, uh, just fi– look at the cross. Don’t move your eyes, and what you should see… Well, hello.

(unintelligible)

Yeah. I mean, do you see these colors?

[00:26:41] AUDIENCE MEMBER:
Mm-hmm.

[00:26:42] CHRISTOF KOCH:
You move back and forth, and after a while, the colors really become very vivid. Again, here, you have– you don’t wanna move your eyes. Moving your eyes minimizes the aftereffect.

See here, by the way, this, this argues against the, the naive notions of naive realism. Naive realism just says, well, there’s a world outside there, and there’s a one-to-one mapping between the world outside and the consciousness in my head. Well, here the world is actually gray, but at least for a certain time, I saw it in color, although those– there’s no color in this case present in the world outside.

So you can now ask a question. You can combine. So this is an aftereffect.

It’s called the color aftereffect. You can, uh, I’m sorry, we need to have the light out once again. Remember this illusion.

Uh, so here, sometimes you, sometimes you see the yellow squares, and so sometimes you don’t. And I don’t see the laser pointer. All right.

So, um, the question is, this seems like silly and couldn’t be answered a couple of years ago, but now can scientifically. Do you need to see the yellow squares in order to get a yellow aftereffect? So once again, just indulge me and look at, just keep your eyes fixed on the cross, and then you should be able to see an afterimage here to the yellow square.

Do you see it? All right. So did you– we can ask the following question in the lab.

Let’s see, for two seconds you didn’t see the yellow square. Let’s see, or even better, for two seconds you saw the left yellow square, but you didn’t see the right yellow square, okay? Because it was perceptually suppressed.

Now, I look at the strength of the aftereffect. Is there a difference between the aftereffect between the one on the left that you saw for two seconds and the one you that you didn’t see? And the answer is very unambiguous.

In this case, it doesn’t matter whether you consciously see the aftereffect, you still get it. The aftereffect does not depend on consciousness in this case. Why is that interesting?

You can say, well, that’s a little, you know, arcane thing. Well, it tells you, it teaches you a valuable fact about consciousness. In this case, it tells you that aftereffect, color aftereffect happens in the brain in the hierarchy.

There’s a processing hierarchy that goes from the retina to the LGN to the various stages of the visual brain. And in this processing hierarchy, the step where the visual aftereffect happens, which is mainly in the retina, although not exclusively, occurs before the stage in the brain where consciousness happens. Visual consciousness for color happens at a higher stage.

So the- the- therefore, the aftereffect just depends mainly on, on how long the retina was stimulated by the yellow. But consciousness happens, let’s say, in the brain proper, not in the retina. So therefore, whether or not you see the, you see the yellow square, it doesn’t have– it doesn’t impinge at all on the strength of the aftereffect.

That’s useful because A, it shows you can make progress on these difficult questions, and B, it tells you consciousness is not this delocalized. When you hear many scientists speak about consciousness, they get– suddenly they, they, these are reduction scientists, but then they get very misty-eyed and talk and wave their hand. They talk about gestalt and holistic and emergent property, and it all gets very fuzzy.

But, but here this is just the, the, the, the moral of twentieth-century biology applied to these problems. Consciousness is all about specific. There are going to be specific neurons with specific molecular and synaptic constituency that are responsible for generating specific forms of consciousness.

And this tells you that the site where consciousness is generated for yellow squares is not the entire brain, but is somewhere specific and above the stages where, where aftereffect happens. Now, there are other aftereffects. For instance, you could look at this picture and you can be happy or you can cry.

So, who sees it? John Kerry? And who sees Geo-, uh, George W.?

And who sees John Kerry here? Anybody? And what about this?

Who sees John Kerry here? There are many more. Okay, this image is actually exactly the same as the first image.

So I showed three images. I can do this again. This is an image– I mean, you have to do this on a person-by-person count, but, you know, so I’m doing it here for everybody because everybody’s slightly different, of course.

Here the image is, and Mike, we edited this, was morphed, so half the, so roughly half the time you will see, uh, George W. and half the time you’ll see, um, uh, you see Kerry, okay? Then you looked at an image hundred percent George Bush, and then I showed you the same image again, but now because you adapted to George Bush, you are less likely to see George Bush and more likely to see John Kerry. Bless his soul.

So—

(laughter)

Right, I was supposed to talk about the soul. So, um, this is a— it’s a different aftereffect. It’s called the face-specific aftereffect.

So not only can you have adaptation for, for colors, but you can also adapt to faces. And you just showed— uh, I just showed you one consequence of that. So again, we can ask a similar question.

This is more tricky to arrange experimentally, and I won’t— I won’t bother you with the detail. Essentially, what we d– can do for, for experts, we use binocular rivalry. We show an image of one of four different faces in one eye.

We project a face into one eye, and then for four seconds, we hide it from view by showing a series of other, of, um, of flashing things and moving dots in the other eye. So perceptually, although one face is present for four seconds, you don’t see it consciously. Okay?

It’s a reli– it’s a very qu-uh, qu-uh, quite reliable technique. And then we, we, we test adaptation, and we test it by doing very, by doing careful psychometric, we’re using four different faces. You have to tell us which of four different faces is present.

And what you can see here that if you suppress, if you don’t see the, if you don’t see the face, you don’t get the shift in the psychometric curve. In other words, if you do not consciously see the face you, you, you were supposed to adapt to, you don’t get the after effect. So unlike for color, where you don’t need to see the inducer in order to get the after effect, if you don’t see the inducing face, you don’t get the, the after effect.

So this tells us that consciousness for faces has to be at the same location, um, or before the location where aftereffect happens. And so again, we know something about the site of aftereffect using fMRI in faces. So again, it tells us something where in the processing hierarchy, um, uh, uh, consciousness, in this case for faces.

Because of course, there’s no guarantee that consciousness for faces is gonna be generated by the same neurons that generate consciousness for yellow squares. Those are different things, and they may well involve different neurons. The question is, is there anything common about those neurons?

Like are they all layer five pyramidal cell that, that, you know, that project the axons down to the pulvinar or something like that? That, that’s a question people are asking. So this shows you, you can do what’s called psychophysics.

You can use psychology to infer something about anatomy called psychoanatomy. Now, you can also use fMRI. I’ll show you a simple fMRI experiment.

Probably most of you have seen this. This is change blindness. It’s discovered by or was popularized by these people at the Nissan Center.

So it’s two images. It’s the original image, and then the image was doctored, and you probably all see the change. Don’t say what it is.

Who sees the change? All right, not everybody. That’s a big change.

This should worry you if you don’t see it.

(laughter)

You see it there? Okay, this is, uh, KOST. Christmas at KOST in Southern Cal. You can see it’s German Christmas. Those are my dogs. What do you… Do you see the change? It’s a big change. It’s not the dogs.

(laughter)

Yeah. No rug.

(laughter)

Oh. Okay, I can show you a third illusion. Can we have that light off, please?

(laughter)

Uh, this is a c-difficult one, and you have to concentrate. And I would ask you please not to cheer or talk to your neighbor, just to be very quiet. It’s a difficult and demanding one.

I’ll show you, um, a video with six, uh, six people, three people dressed in black T-shirts and three people dressed in white T-shirts. And the th- the two groups of three people each, each have a basketball, and they’re, they, they’re bouncing it among each, among each other. And your mission is to count, to neglect the people in the black T-shirts and just count how often do the people in the white T-shirts pass the ball.

Okay? So please be quiet, and it, it takes like 20 seconds, so you have to concentrate. So how many people saw the gorilla?

(laughter)

It’s about two thirds of you. We can do it again here.

(laughter)

(laughter)

(laughter)

Now, you know, this makes you skeptical when you go to witnesses, right? So, I mean, it’s, it’s quite troubling because here what you have, you have an event that’s perfectly visible for five seconds. It’s a big event, right?

It’s an unusual event. I mean, you don’t have that many gorillas walking across, uh, across an office corridor yet, you know, and this gorilla takes five, six, seven seconds to calmly walk across, and many of you totally miss, miss this event. So again, it shows our limited ability to actually see, particularly to see unexpected things, right?

And I cleverly, of course, I misdirected you. If I would have told you something interesting is going to happen, tell me what it is. I mean, then probably ninety-nine percent of you would, would have seen it.

Anyhow, so this is known as change blindness. It’s a large set of different illusions, and it just tells that we may be blind to large changes in the environment. Um, it also, uh, you know, we also have this illusion, I open my eyes, I see everything.

And of course, the last three, this and the previous one shows you, you don’t see everything. We think you see, you see every– you see have a, what, what psychologists call gist. You have a very quick impression of, you know, this is an office, you know, it’s an audience.

And then you see at any given point in time, one or two things. All right, so we can now ask, what is the correlate of that in a, in a mag- in a, in a magnet? We can look at in a human, for example, the first part.

So the, the visual input goes from the eye through an intermediate relay to the back of your head. You can all feel it here. There’s a little bump at the back of your head, and a little bit above that bump is your visual cortex.

So if somebody whacks you with a baseball bat, which you shouldn’t do, uh, you can see– you can feel stars. That’s because you’re mechanically stimulating this brain. So we can ask the question now in an fMRI, in a, in a, in a magnetic scanner, for example, ugh, to what extent do the neurons in the, in the first cortical stage of vision, do they already reflect, do they already reflect, do they reflect what’s just coming in from the retina?

Do they reflect the physical input, or do they reflect already the conscious perception? Now of course, we in a, in, um, uh, we can’t use really this gorilla because you can only use it once obviously, so we do a much more difficult task, much more boring of course, we’re scientists after all. So we get you to fixate here, and then you have these, uh, rings you have to monitor, and you have to tell us when one of the orientations on the rings changes.

And focus on this just flipped. And it’s a, you know, it’s a difficult task, and even if you’re, you know, eighteen and well-rested and very alert, you will miss as– a large number of these, of these changes just because of the nature of our visual system. Of course, we designed this to d-display to be, to be, uh, to be difficult to find all these changes.

Although once I point them out to you, of course, you, you, you, you see them. So now we look inside your scalp, we look at in your very s– uh, in the back of your head, as I said, pr– um, there’s primary visual cortex, there’s secondary visual cortex, and we can essentially look at, uh, the, the… Uh, here we’re n-not looking at neurons, we’re looking at a proxy.

We’re looking at changes in blood flow. So essentially, what fMRI, it gives you is power consumption of the brain. If those neurons are a little bit more active, they consume more power, uh, they need oxygen, they need ATP, ATP needs oxygen, oxygen is transported in, in with the red blood cells.

They rush in, and that’s essentially what, what we’re tracking. We’re, we’re looking at hemo– deoxyhemoglobin. And it’s, it’s sluggish.

It’s, it’s, it’s over big areas, many sc-cubic millimeter. You gotta remember per cubic millimeter, there are one hundred thousand different cells in each cubic millimeter and a few million, um, uh, well, a few, um, uh, eight hundred million synapses per cubic millimeter and two kilometers of axon wiring. So here we’re looking at big areas and very sluggish.

This is a timescale of seconds. And what you see here, here you see the, the signal in response when, when one of the ring ch– flips its orientation and when you actually saw it, called a hit. So the person saw tells me by pushing the button for ring number three, ring number three just flipped.

And then you see a signal here. It’s, it’s small. It’s a quarter of a percent, but it’s highly reliable.

You can measure it. That’s a standard deviation. You can measure it very reliably.

It increases and it decreases with a slow timescale. All right, that’s expected. Then what happens if you miss the change?

So missed change, there was a change there, but you didn’t perceptually see it. Well, in that case, sort of the signal is green. It’s sort of more or less noise.

So physically, there’s no difference between the green and the blue curve. In both cases, there was a change outside in the image that in prin– wha-was highly, you know, w-w-was in principle clearly visible, but I didn’t see it because I didn’t attend to that loca– the subject didn’t see it. So the subject didn’t see it, it doesn’t register in, in the BOLD response.

Conversely, here the person thought there was incorrectly, he thought there was a change, but really there wasn’t a change. And then the signal you get is almost the same as the signal you get when actually there was a change and the person saw it, and here when there wasn’t a change and the person didn’t see it. So what’s– again, what’s remarkable that if you compare green and blue, in both cases there was the same physical input, but here there was consciousness associated with it, and here there was not.

Red and, uh, blue are the same conscious, uh, events. In both cases, the person saw, at least claimed he saw, he pushed the button, that there was this con– He saw this change in orientation.

In one case, there was, in one case, there wasn’t. So here you can clearly see that in the– already at the level of primary visual cortex, uh, as far as you can tell in a magnet, the signal in cortex itself followed the conscious perception rather than the physical signal. All right.

So let me tell you about the n- uh, last experiment. This is, uh, very new. Just, um, so we, we, uh…

yeah, everything here I’m talking today is, is about humans. Uh, I, I, I guess I didn’t put in any animal experiments, I just realized. So, um, most of what we… well, ninety-nine point nine percent of what we know about the brain, we know about, uh, animals, from animal brains.

Particularly, almost everything we know about nerve cells, neurons, we know from animals. And if you wanna understand the analogy I made this morning, if you wanna understand chemistry, you need to know about molecules. If you wanna understand physics, you need to know about atoms or subatomic particles.

Likewise, if you want to understand perception, consciousness, memory, action, you must know about nerve cells. Now, nerve cells are sort of the atoms of the nervous system. And there are, you know, ten– there are a hundred billion of them in the brain, and you want to query them, you want to listen to them.

In ge– In general, that’s very difficult to do in humans, obviously. But there are conditions. So th-this is, uh, sort of the people in my lab who did this.

He’s now at MIT. He’s, uh, he’s, he’s now in England. Um, and we do this in the lab of the neurosurgeon Itzhak Fried at UCLA.

There are a few conditions when you can actually record from, uh, from neurons in the human brain, and the advantage is you can directly ask them about, are they conscious? You can manipulate– you can, you know, get them to play video games or sort of things, and they can directly tell you about their state of consciousness, rather than you have to infer it indirectly. So, um, the c- the setting here is you have epileptic patients that are severe epileptic.

In other words, the, the pharmacological intervention doesn’t work anymore or, or it doesn’t work very well. So these patients may have one or two or ten seizures a day. You gotta realize, if you have one seizure here in California, the next half year you’re not allowed to drive anymore.

So seizures seriously impairs, you know, the, um, uh, the way, uh, I mean, has seriously consequence, adverse consequence on, on your quality of life. So, um, then what you do is a very successful intervention. The surgeon goes in, identifies where the focal seizure is, if there is a focal seizure, and takes out that part of the brain from which the, the seizure originates.

And by and large, it’s a very successful operation. In some subset of patients, you can’t do that. You look at from the outside, you do EEG, you do MRI, and you don’t know, the brain looks relatively normal.

You can’t identify where’s the part of the brain, where’s the seed, if you want, inside the brain where– from which the seizure originates. Then what the surgeon does is like, Fried, and this is also done here by, by, uh, Berger, for instance, at– and by other people at, at UCSF. It’s done in many places throughout the country and the world.

You insert various types of electrodes. So these are so-called depth electrodes. So here you insert them, here you, uh, typically ten or twelve of them, you drill a, a little burr hole, and you insert the depth electrode here into the brain.

This case, part of the brain called the hippocampus. And then you use these leads, these are platinum-iridium contacts. You essentially use these leads to do like EEG, except they’re inside the cranium, intracranial EEG.

And essentially, the patient is then hooked up to a, to a machine that’s monitored twenty-four seven. The patient stays in the, in, in the ward for, here you see a patient in the bed, you know, it’s, uh, since there are no pain receptors inside the brain itself, um, uh, the patient doesn’t need to be anesthetized. So the patient’s sitting in the bed and lying and reading or watching TV, talking to his mom, whatever, while there are these, um, ten or twelve microprobes inside his head that you can see the Y goes to his bed stand and, and these are being monitored.

And then if he has seizures, you can now tri– the, the, the neurologist can essentially triangulate where the seiz-seizure originates. Then they take out the electrode, then they take out the, the, the, the part of the brain that gives rise to the seizure. What Fried did and his collaborators, they added tiny wires here.

They moved through, they hollowed out this inner volume of this microelectrode, of this electrode, and they inserted these wires. So essentially, we have nine wires. And so we have ten of these micro– of these big macro wires, and then each one has eight or nine or ten micro wires.

So we have on the order of a hundred wires in, in the head of a patient. And so we can now do exactly what we can do to– in an animal. We can listen, we can do a lot of signal processing, I won’t, uh, I won’t bore you about.

And you can listen to the individual neurons firing. You can hear them chat away. You can put them on an audio monitor, and you can hear.

You may know the way neurons communicate, they have these little, it’s an asynchronous pulse code, different from a computer code. They communicate by these little pulses. They’re like, um, uh, a millimeter, um, a millisecond across and maybe a tenth of a volt in amplitude.

Particularly if you’ve got a German accent.

(laughter)

So, um, Gabriel Kreiman, who did this early work for his PhD, discovered, um, this neuron. So I’ll show you a bunch of these graphs. So here what you have, this is time.

Time runs this way from left to right. This is three seconds. We show this image, in this case, this, this, this house and with a tree for between those two vertical marks.

For one second, the patient saw this picture, and for one second, the patient saw this draw– line drawings. And here is the firing rate. This is the, uh, the, the horizontal bar, gives you the average firing rate.

So on average, this neuron fired, you know, I don’t know, uh, um, two spikes a second or something like that. And here’s, uh, are the individual tri-trials. So each of these splits is one of these action potentials, and that we just average and represent here.

So this was early, um, five years ago, and what you can see is that it didn’t respond. We showed many, many things. I’m o-only showing the best responses here.

It didn’t– Most of the neuron didn’t respond at all. This neuron didn’t respond to most images, except it responded to this image of our ex-chief in commander and he– a line drawing of him, a presidential portrait of him and him and his wife. Responded very significantly, very strongly.

You don’t have to do statistics to see this. And here there is, and, uh, now this is again from Gabriel, another set of neurons that responded to The Beatles or to the, uh, to The Simpsons. And then we, we got very interested, and we started an entire research program to look for these neurons and spent four years, this was the work of Quian Quiroga, um, to look for these neurons using a whole set of the– various experimental paradigms, and, uh, this is what we found.

So again, same thing. This is three seconds, one second before the image, then for one second, this image is on shown, and then one second after the image. And so these are– here in this case, we showed like eighty-eight different images.

Uh, we, we are re-really only limited by how much time we have with the patient. So, you know, the patient may be bored or may go to sleep or, you know, he has to go to the bathroom. So usually experiments are twenty minutes, thirty minutes, forty minutes.

And here you can see this is a, a famous actress, um, uh, Jennifer Aniston, she’s called. And, and you can see the neuron here responds quite nicely to these very different pictures of, of Aniston here, and doesn’t respond. So Aniston used to be married to some other star and doesn’t respond when she’s there with her, um, at the time, this is when they were still married, when she’s here with her husband, uh, the, the neuron doesn’t fire for whatever reason.

(laughter)

And here there are various others. I don’t… I mean, these are other actresses.

This is Pamela Anderson. Uh, these are famous, you know, like Eiffel Tower, Golden Gate Bridge, Pisa Tower, et cetera. So this is a neuron that fires very selectively, only to, uh, pictures of Jennifer Aniston.

What we found remarkable as visual scientists is, is also the degree of invariance, particularly if you’re interested in, in object recognition. In other words, this picture and this picture at the bit level, they differ totally, right? They have different backgrounds, different hairdo, yet still the neuron responds to it quite selectively.

This is an, a neu-neuron that responds to, um, um, um, the Sydney Opera House. And interestingly, the patient confused this, and it only became apparent, and we having a new paradigm to exploit this after we finished the experiment, the patients thought this was actually also the Sydney Opera House. Well, this is actually the Baha’i Temple, which is very interesting.

Um, and, uh, here it, uh, th-this neuron, and we find a subset of these neurons also respond to, uh, strings, letters called Sydney Opera. So it’s a very high-level degree of in invariance. This is to another actress called, um, um, Berry.

Halle, Ha-Halle Berry. So you can see again, they’re very different. Here she’s in a cat, she plays some cat woman or some woman who runs around dressed in a cat suit.

I don’t know why, actually.

(laughter)

Um, um, there

(clears throat)

you have a line drawing of her. You have her in a cat suit. You have the text, Halle Berry. There’s some other woman who’s also dressed in a cat suit, but this– the patient knew that this wasn’t Halle Berry.

(laughter)

And, uh…

(laughter)

Now from the same microwire, don’t ask me why. So, you know, we can pick up multiple– by listening carefully, by doing what’s called, um, uh, um, cluster cutting using, uh, wavelet-based analysis, we can distinguish different neurons from one wire. So from the same wire nearby, there was a different neuron that responded to Mother Teresa.

Um, This is a neuron response to Pamela Anderson. This neuron– Oh yeah, here’s her husband, Pitt.

That’s right, Brad Pitt. So this is a neuron in the same patient as the one, um, as the one I showed you at the beginning, and this pa– this neuron only re– But it’s on the other side of…

This is in the hippocampus. Oh, I should mention, these neurons, by and large, most of them are in hippocampus. Some of them are also in, uh, perirhinal cortex and amygdala.

We put the electrodes there because the pa– the surgeon puts the electrodes there, right? We are, of course, totally dependent on– because we’re doing– we’re piggybacking with this experiment under the clinical, um, expe– under the clinical experiment. So this was on the other side, uh, of the same patient but the other side of the brain.

There’s a neuron that responds to Aniston and Brad Pitt, but not to Aniston by herself. Anyhow, so you f– so what we have here are some very specific neurons that respond, that are, um, that are quite invariant, and they find a very explicit manner, uh, um, to and in a very sparse manner to famous individuals. And there are probably gonna be many neurons like that that respond to things that the patient is very familiar with, either famous people or familiar things like your mother, you know, your grandmother, um, you know, the computer font you use, The brain probably wires up neurons that respond specifically to that.

Now, if you care about what’s BMI are called, brain-machine interfaces, if you, if you wanna think, um, uh, people are starting to think about that. If you wanna implant a device into people to help them to see, for example, because they’re blind, or to enable them to see, or enable them to move, you want to be able to do what’s called decoding. You want to be able to s– to take a group of neurons, record from their activity, and then infer what was present in the environment.

It’s called a decoding technique. And I’ll just show you intuitively what we mean by that. So here you have different images.

Here we’re recording from ten units. It’s the same unit, unit one here and unit here responding to thirty-two different images. And so now the question is, let’s say you have an, you have, um, you know, you have a particular spiking pattern.

Can you infer or with what probability can you infer based on, you know, you know, unit number two fired strongly, unit number three fired sort of middling, and unit number one, two, and five didn’t fire at all. Can I invert the process? Can I say, well, based– Can I essentially read the patient’s mind?

Can I read the patient’s neurons in order to read the patient’s mind? Okay, now I can do that quite well with far, far better per– uh, chance performance. So here we– I, I average over all the experimental sessions.

This is chance performance always. So here, let’s say if I have thirty images present, in order for me to prece– to guess which of thirty images present, chance would be one out of thirty, right? If I just do it randomly.

On average, I can do far, far better. So in other words, I can do decoding. I can decode the visual thoughts, quote, from the patient by listening to s-small subset.

Here we talk, on average, we’re talking about twenty to thirty neurons, um, that the patient, uh– So I can infer what the patient is conscious of by recording from these neurons. Now, that’s all still observation. Ultimately, what you would like to do is in– is go to causation.

What happens, for example, this is possible in the clinical context. The neurosurgeon does it often. What happens if I stimulate those neurons?

Can I create the percept? Can I not only read out, but write in? That’s for the future.

(clears throat)

All right. Um, let’s skip this. And I can, I ca-I can also do imagery.

We’ve also done that. Uh, so you can– you know, I can ask the patient, “Well, remember I just showed you this picture of Van Gogh. Uh, can you think about that?”

So here, for example, I have, um, a neuron that responds to the image of, uh, Paul McCartney, here the green. And here are the images when I showed him the images of this house. So you can see whenever the the patient saw a photo of McCartney, it fired relatively, not always, but fired very strongly.

When I showed the patient– when we showed the patient the picture of this house, it didn’t fire, or only fired very weakly. Now here, I asked the patient, “Close your eyes, and for three seconds think about Paul McCartney, think about the house, McCartney house.” And again, you can see there’s a nice selectivity that these neurons are as selective, statistically speaking, almost as selective doing a physical input as well as doing imagined input.

All right, let me come to a conclusion, to, to the end. So I have two last slides. So in this, in this quest to study consciousness, what is the long-term strategy?

Well, first of all, most importantly, although I didn’t actually give you an e-example, w-we learn more, we– for obvious reasons, partly because we can in– well, mainly because we can intervene in the animal brain, particularly in the brain of mi-mice and monkeys. We can intervene with ever more fancy techniques, particularly with molecular techniques. Um, if we want to understand at the circuit level, if we really want to understand which neurons, you know, um, in what part of the brain using what circuit, you know.

I like to understand the series of events that goes from layer four C alpha to layer three B involving which interneurons. In– At this level of specificity, like we can do in molecular biology, I need to intervene with the system.

We need to perturb it, and that requires an animal model. So therefore, people are developing sort of, uh, uh, animal models, particularly mammalian models, to, to study these sorts of things. Then I, I need to identify the coalition of neurons that give rise to a specific neural correlate of consciousness.

And then I want to deliberately, transiently, delicately, and particularly reversibly, I wanna inter- interfere with a neural correlate of consciousness. For instance, can I take a mutant mouse? Can I use a specific cell-specific promoter to turn on and off consciousness?

In other words, can I create a little zombie mouse that can do certain things, uh, you know, like, like it can do all sorts of behaviors that animals or we can do without being conscious, but it can’t do those behaviors anymore that require consciousness. That would be an experimental tour de force. For instance, can we search for random mutations that selectively interfere with consciousness?

Uh, uh, yeah. So I mean, that’s one technique, how you find them using random mutation. Particularly, that’s how you do it in the medical pharmacological industry.

And then as, as I mentioned, in both humans and animals, we need to d– move from correlation to causation. And finally, none of this is really a theory of consciousness. I have not talked about a theory of consciousness.

Ultimately, what we need is a– what we want is a theory of consciousness. We want to be able to say which systems under which conditions are conscious.

Why is my, uh, you know, why is my enteric system not consciously? Presumably, why is my immune system not conscious? Why is my brain conscious under certain condition?

Um, what about a fly, a fetus, you know, um, a person like, like a patient like Terri Schiavo? What about the Internet? Ultimately, we would like to answer this question for synthetic devices.

Can I make a, a synthetic device that’s conscious? On the, the Internet, a robot, a computer. So two concluding remarks.

So one is people say, “Well, this is all very nice,” Professor Koch, and I believe you in the fullness of time, with adequate funding, of course, um, we’ll– we, we can– we, we will be able to understand that if I’m conscious of red, then those neurons will fire in this particular way. And if I’m conscious of pain, some other neurons will fire.

Well, so what? That doesn’t explain consciousness. Of course, people made exactly the same argument for life.

Now, this is, um, an analogy. It may not be right, but I think it’s a– It’s– it may be a very telling analogy.

So people in the twenties, thirties, and certainly before in the nineteenth century, he had great difficulties with understanding, uh, how life arose. And there’s this wonderful quote I discovered from a book. Okay, so, uh, Thomas Hunt Morgan, just before he moved from Columbia to Caltech, wrote this book where he summarized his research on fly, right?

He, he developed the fly as men– as, as genetic tool. Well, he argued essentially modern language, that genetic information is stored along one-dimensional strings. And then Bateson, who was sort of England’s preeminent, um, uh, geneticist, reviewed that, and this is what he said in his review in, in Science, I think.

“The properties of living things are in some way attached to a material basis, perhaps in some special degree to nuclear chromatin. And yet it is inconceivable,” so that’s what Bateson, uh, writes, who is, of course, a, um, uh, who was a reduction scientist. “It is inconceivable that particles of chromatin,” i.e. chromosomes, “of any other substance, however complex, can possess those powers which must be assigned to our factors of genes.”

It’s his spelling. “The supposition that particles of chromatin, indistinguishable from each other, and indeed, of course, this is a killer, almost homogeneous, right? Of course, they’re not homogeneous.

Almost homogeneous, but they didn’t know any better. Under any known test can, by their material nature, confer all the properties of life surpasses the range of even the most convinced materialism. So people said, “I don’t understand.”

I understand chemistry, and I don’t understand how you can get out of chemistry, a single egg, how you can get all the specificity it makes up and– that makes up, um, an individual in the next generation. I know chemistry. It can’t happen.

I need new laws. I need elan vital. I need all sorts of new…

I need a new physics.” Even Schrödinger argued that. Well, in fact, of course, what people didn’t realize, they didn’t understand the prodigious power of macromolecules.

You have to understand, when this was written, people didn’t even realize that hemoglobin, historically, that hemoglobin was a single molecule, that in, in an individual like myself, all the hemoglobin is always exactly the same molecule. They had a sci– Well, at the time, there was a science called colloidal science, which studied the, the sort of what happened if you have large collection of indist– of sort of, um, um, of, of different molecular weight of the same, uh, of the same molecules.

They didn’t realize that you can store, you know, the great specificity of macromolecules. They didn’t realize you can store prodigious amount of information in one-dimensional nu-nucleotide strings. So I don’t think we should make the same mistake, uh, twice that we kn– because you very often hear this argument say, “Well, we understand biology, and no matter what, biology or physics…” I mean, there’s some people even who say, “I know physics.

I know biology. There’s nothing in principle that can explain consciousness.” And I think that’s a defeatist argument, and we’ve heard it before when it was wrong.

That’s not to say it may be, it may, may be right in the future. And lastly, there are also ethical implications of this sort of, uh, of this sort of research. Because once we have really a much better, I mean, I think you can say this already today, but certainly in the fullness of time, once we understand which creatures can suffer and which creatures cannot suffer.

So for instance, mammals, if you look at the behavioral evidence, there’s no question that all mammals are capable of suffering. They can– they are, they are of course capable of, of great joy if you have dogs or cats or other animals, but they can also suffer, and particularly the ways we treat animals, particularly in this country. And, uh, so I think this, this, um, this, uh, research does have ethical implications.

(laughter)

Once we have a theory of consciousness, once we can tell for with great confidence which animals are conscious and which animals are not conscious, we have to enlarge the circle of life. We have to enlar-enlarge the magic circle, those individuals, those organisms to which we we say, “Well, they are special.” Right now, the only things that are really special are us humans, people that look like us, that have two legs, that can talk.

But of c– uh, um, but then, of course, there are people who can’t talk. There are people like Terri Schiavo or other p– a patient that twenty to fifty thousand patients like Terri Schiavo today in in in the United States of America alone. People who, as far as we can tell, uh, who are certainly alive, but there’s no evidence for consciousness, yet we accord, of course, a much more legal protection to individuals like that than to any of our animals, any of our, our, um, mammalian friends.

So I think in the fullness of time when, when, when this research has run its course, um, it will have all sorts of implications, including, of course, for our self-view. Thank you very much.

(applause and cheering)

[01:01:13] TONY:
Well, wow, that was, that was a great lecture. Um, it was a bit scary I found, because I don’t know about you, I could see the gorillas, but– uh, the gorilla, but I could also always see the two yellow squares. So I guess, um–

[01:01:26] AUDIENCE MEMBER:
He’s hyper-attentive.

[01:01:27] CHRISTOF KOCH:
Yeah. Hyper-vigilant.

[01:01:28] TONY:
I wish we could have more of you, but we still have a little more of you, and so some questions can be asked.

[01:01:38] AUDIENCE MEMBER:
Yeah, over there. You have to use the microphone. Please make your questions very much brief and to the point.

We’ll try to get to as many questions as possible. Thank you. Um, professor, on the question of the Jennifer Aniston neuron, how do we know we’re not just seeing something that the patient enjoys, here regardless of the actual image?

[01:02:05] CHRISTOF KOCH:
It’s a very good question. Yeah, so we have to control for that. So in this case, you can see that they have, uh, some s– I mean, so the patient likes many of these actresses when you ask them, but in this– but one particular neuron will only fire to one part–

Uh, in this case, will only fire to one specific individual that we’ve tested here. So, so you have to rule out that. It’s a very good question.

You have to rule that out, but you can rule it out by s- by spec– by finding, uh, you know, two or three actresses that the pa- that the patient likes, but one neuron fi– you know, one neuron will only respond to different images of one but not to the other actress. That’s one way you can rule it out. And I’m, I’m very careful about not–

I’m trying not to call it Jennifer Aniston neuron because it’s well possible that the same neuron will also respond to some-something else. I just don’t, you know, because I only have half an hour to test this patient, I can’t show all possible images and all possible things. What is it– It is clear that these neurons are amazingly specific.

I don’t think they are so specific that they will only fire to Jennifer Aniston, but, but, that you know, that’s a quantitative question. That’s not easy to answer. Yeah, good.

[01:03:07] AUDIENCE MEMBER:
Um, you said in directions for future work that you hope to be able to eventually find some way of turning off, on and off neural c- Uh, the neural correlates of consciousness, and then perhaps create zombie mice and see what– come up with a task that conscious mice can do but unconscious mice can’t. Could you give an example of such a task where you’d be able to measure it?

[01:03:29] CHRISTOF KOCH:
Yeah. So for instance, one thing we’re trying is that, um, do you know Pavlovian conditioning, right? You get the bell and then, you know, that’s, uh, you can also do aversive conditioning.

So what we do in, in our human volunteers, we give them sounds, and we shock them, and we can also do that to mice. And it turns out that different forms of… Well, there are many different forms of con– um, of this aversive conditioning.

In one case, called delay conditioning, the, the, the sound or the image comes at the same time as the shock. And in the other one, first you have the, let’s say, the sound, and then two seconds later or fifteen seconds later, you have the shock. The second one, research shows, requires you to be conscious of what’s called the CSUS contingency awareness.

In the second case, when you first have the, the tone and fifteen seconds later, the shock, you have to be aware of that. You have to realize, oh, there were tones, maybe even different tones, high tones and low tones, but the high tones was always followed by the shock, and the low tone wasn’t. Unless you’re aware of that, you will not be conditioned.

In the previous cases, psychologist Larry Squire, folks from UCSD, has shown this. In the, in the other case, when they overlap, you don’t need to be conscious of that. In fact, you can be totally distracted.

You can watch a movie, you can still be conditioned with delayed condition. We essentially showed the same in mice. We showed that you can distract mice by changing the house lights up and down, and then if you, uh, if you do trace and delay conditioning or context-dependent conditioning, um, if you, if you manipulate that– if you distract them, it interferes with trace conditioning, but not with delay conditioning.

And so again, by analogy, you have to move by analogy. You say, well, in humans, if you interfere with it, they’re not conscious of it, and you can’t get the conditioning. In animals, all I can tell you in mice, I interfered with their, with their behavior, and just like in humans, they couldn’t do the one form, but not the other form.

So that’s what that, for example, that’s one example. So any behavior that we know in humans requires consciousness. Usually, you require consciousness when you need to do– store some information for some time, like ten seconds, twenty seconds online.

Those are usually behaviors that require consciousness.

[01:05:27] AUDIENCE MEMBER:
Uh, could you comment about, uh, split brain studies? And in particular, I’m thinking of cases where, uh, the right side of the brain can manipulate an object meaningfully, but the left side doesn’t know what it is and can’t name it. Is consciousness in both places or one or the other?

[01:05:40] CHRISTOF KOCH:
Yeah. So, so the, in fact, these were experiments done in, at, at Caltech, uh, by Roger Sperry, of course, thirty years ago. So it turns out when you split– when you cut the corpus callosum, the two hundred million fibers that connect the left and the right brain, by and large, in most people, only the left hemisphere can talk, but by and large, both hemispheres are conscious.

Both hemispheres are capable of complex adaptive behaviors. Both hemispheres can learn. Both hemispheres can store information online.

One can talk very well, can– one has a normal fluency, one has a normal preoperative intelligence. The other one can grunt, the other one can sing, but the other one, uh, can do all sorts of complicated things that doesn’t involve language. Though the evidence seems to suggest, and Roger Sperry, who did this, certainly also believed this, that both hemispheres are conscious.

So what you have, you have two conscious minds in one skull. So it’s a rather interesting, And I, I smell a great existentialist novel here that’s yet to be written. Yeah, because you have…

I mean, it’s an interesting question, where do your memories go? Because if you spla-if you split, probably different sort of visual memories, they may go more with the right side, and other sort of more abstract memories may go more with the left side. And early on, when you, when you look at these patients, early on, they have these conflicts.

So for example, where the left hand will, they open the shirt, and the right hand will close it. Because, and I’ve seen vi-videos, for example, of one patient, when– This was shortly after the operation, when the, the doctor asked her how many seizures she had, and she said, “Three.”

And then with her left hand that’s controlled by her right brain, she made this, “Two.” And then the doctor pointed this out to her, and then she did this, and then she said, “Oh, yeah, my, my hand always make– do, does these things.” And then she started fighting.

And, you know, then I had the same audio reaction. You know, I and the other member of the audience started laughing, but then you realize this is really sad. And then she broke down in tears ’cause she, she doesn’t know what’s going on, right?

Right, it’s her– Well, if you talk to she, if you say she, it’s really her left hemisphere, and of course, there’s this confusion because suddenly your body, you know, this left hand doesn’t belong to my, to, to my left brain anymore. Fortunately, this quickly subsides.

Within a few days, dominance is established. Usually, it’s the left hem-hemisphere that establishes dominance, but as far as we can tell, there are probably two conscious entities in that one skull. And the other question you can ask in normal, these are, of course, all pathologies.

You can ask, and people have asked this as way back as the middle of, uh, the nineteenth centuries, “In normal life, can you see echoes? Can you hear echoes of two conscious minds?” Are there cases when, for example, when you have two…

When you, for example, ha– and there are, uh, nice litera– nice examples in the mountain literature, in the mountaineering literature, on an extreme case of anoxia, like if you ever read “Touching the Void” by Joe Simpson. When, for example, he describes as he crawls over the glacier, that there’s this clear, compelling voice that spoke to him. And then on the other hand, there was this very seductive images that kept on being generated.

Seductive images of him being home, him being warm, and this definitely sounds like a left brain, right brain. But, you know, still it’s, it’s difficult to test. Anyhow, it’s an interesting question to what extent you hear echoes of those two minds that normally are unified in a normal brain, but that in a split brain are, are rent asunder.

[01:08:39] AUDIENCE MEMBER:
Yeah, it seems like one of your… Okay, like with, with the example with the, uh, yellow squares. So we all have a lot of brain activity going on, and then when this yellow square, whether it’s visible or not, we might be able to say there’s a small part of the brain here that changes its state.

But there’s a lot of other brain activity going on that doesn’t change with that. So I guess my question is, if you find those specific neuronal correlates of consciousness like that, how can you be sure that that small change isn’t just a sort of switch, but that the experience of consciousness actually depends not just on that little part of the brain, but in a much more, say, gestalt way, or at least spread out sort of way on the other brain activity that isn’t changing between the two?

[01:09:26] CHRISTOF KOCH:
Uh, I mean, there’s no answer to that except ex– more experiments. No, I mean, so it’s, it’s a very good question. So you focus on what happens if I can put an electrode in, in that location, can I actually make it switch?

Then what happens if I take those neurons and inactivate them, uh, but the neurons themselves are fine, but I activate– I poison their postsynaptic so they can’t release synaptic vesicles anymore. W-what– I mean, what about the conscious perception in the monkey?

What happens if I, if I leave all that intact, but I, I, I just, I inactivate the, those synaptic tar– the, the postsynaptic targets? So you have to do it. So it’s not gonna be easy.

The brain’s a very interconnected network, and, uh, it’s gonna be difficult to untangle cause and effect, but we did it for, for heredity. It worked quite well. We know it’s, it’s a very complicated chain, but ultimately they are over the long term, they’re stored in one particular molecule, in one particular location in the cell nucleus.

So it may be different for consciousness. It may be that consciousness involves most of the brain in an inc-incredible complicated way, which, which make it, would make it much more difficult to understand scientifically. So it’s an empirical question.

(clears throat)

[01:10:23] TONY:
Uh, I, I think we have one more time for one more question.

(cough)

Gentleman there in the middle with the-

[01:10:33] AUDIENCE MEMBER:
Thank you. Could you comment on the, uh, use of bispectral analysis of consciousness in anesthesiology and, uh, the, um, findings by Davidson of the high gamma synchrony in Tibetan meditators?

[01:10:49] CHRISTOF KOCH:
I don’t know what to make of that. I mean, it’s, it’s a very interesting finding that in, in, um, in monks, you have… I mean, all it tells you that if you, you know, meditate for twenty years, it leaves traces in the brain, and their brain is slightly different than your brain or my brain.

Uh, it doesn’t really, I mean, ex– it doesn’t tell me anything specific for my, and for my research. But anything you do, anything that affects your mind will affect your brain, right? And so if you meditate for a long time every day for four hours, it’s going to affect your brain.

In this case, it seems to enhance certain frequency bands. Um, it, it doesn’t tell us anything– Well, well, what’s the–

Some people claim it tells us that, that oscillations are specific, that these gamma, um, oscillations in the forty hertz range are specifically involved in consciousness. Um, it’s a very– it, it may be true, but, um, it’s, it’s unspecific. I mean, that’s the trouble with all this human stuff.

Ultimately, you gotta work with animals because you can’t really… You know, usually you have this interesting phenomenology, but you want to move beyond that, and that’s just very difficult to do in, uh, I mean, for obvious reasons, in, in humans.

[01:11:52] TONY:
Thank you very much. I think here we better end the formal discussion, but thank you for attending and thank, uh, our speaker again.

(applause)