Can Images Unlock the Mystery of a Healing Brain?

Maybe. Maybe not. But they can offer pathways for investigation. This article comes from MedPage Today.

Can Images Unlock the Mystery of a Healing Brain?

By Nancy Walsh, Staff Writer, MedPage Today
Published: February 01, 2013

Click the bottom right corner of the video player for full screen

Reports that sophisticated imaging had revealed residual brain activity in former Israeli prime minister Ariel Sharon, who's been comatose for more than 6 years, focused on the spectacular technology involved, but overlooked a more important clinical story -- that some patients with severe brain injury actually do regain some degree of cognitive function.

The concept of the minimally conscious state as a separate entity from the vegetative state has been a crucial diagnostic refinement in recent years, according to Nicholas D. Schiff, MD, who directs the Laboratory of Cognitive Neuromodulation at Weill Cornell Medical College in New York City.

"In the past, these patients were all lumped together, until we started asking the question of why it should matter if some patients appear to have some awareness and ability to track although they can't speak, respond, or organize movements," he said.

"Actually, it turns out that it matters a lot," Schiff told MedPage Today.

Consciousness and Recovery

What the brain imaging studies and neural research have demonstrated is that, in the weeks and months following brain injury, there is a vast difference in evolution of their condition between patients with minimal consciousness and those who are truly vegetative.

"Recent studies have demonstrated that it is important to disentangle both clinical entities as functional neuroimaging studies have shown differences in residual cerebral processing and hence, conscious perception, as well as important differences in outcome," Schiff wrote online in the journal NeuroImage.

What the research has begun to show is that recovery of important neural networks may be an occult process in some patients, with no visible changes in their ability to speak or move, he explained.

In one program that has prospectively followed 400 patients with traumatic brain injury over the course of 5 years, a significant proportion of patients advanced at least to a minimally conscious state despite having initially been at the lowest level of the Glasgow coma scale.

Approximately one-third of the patients regained some degree of independent function, and some even acquired vocational re-entry level of functioning, according to Schiff.

One possible factor in the gradual return of consciousness after injury, sometimes over a long period of time, "is that the normal recovery process ... includes a component of structural remodeling that could plausibly relate to reestablishment of goal-directed behaviors and driving of learning and memory mechanisms," Schiff wrote in the December 2009 issue of Trends in Neurosciences.

Imaging the Broken Brain

Following reports of spontaneous cognitive recovery after brain injury, a group of researchers from Cambridge, England, and Liege, Belgium, conducted a series of experiments in which they attempted to identify the level of awareness and the ability to communicate in 54 patients with severe brain injury.

Functional MRI tests in healthy controls have shown that the same areas of the brain activated by motor and spatial tasks can also have increased blood flow when the person imagines doing an activity such as playing tennis or walking through a house.

First, the researchers requested that the patients, while in the MRI scanner, imagine performing those activities.

For patients whose functional brain scans revealed increased intensity in the motor cortex or parahippocampal gyrus, the researchers then increased the complexity of the task, asking a series of autobiographical questions that would have yes or no answers, such as if the patient had a brother named Tom.

To answer yes, patients were instructed to imagine playing tennis, and to answer no, to imagine walking through the house.

The hypothesis was that the motor area of the brain would light up with an affirmative response, while the parahippocampal area that reflects spatial function would show increased signal intensity with a negative reply.

Among the 54 patients, five were able to "willfully modulate their brain activity," in response to the task instructions -- in effect, correctly answering the questions, wrote Martin M. Monti, PhD, now of the University of California Los Angeles.

This proved that "in a minority of cases, patients who meet the behavioral criteria for a vegetative state have residual cognitive function and even conscious awareness," Monti and colleagues reported in the Feb. 18, 2010 issue of New England Journal of Medicine.

But functional MRI is expensive and not always accessible, so another group tested 16 apparently vegetative patients using easily available electroencephalography (EEG) techniques and found that three were able to provide appropriate responses to commands that they imagine moving their hands and feet.

The EEG results were equivalent to what is seen for healthy controls, Adrian M. Owen, PhD, of the University of Western Ontario, and colleagues reported in Nov. 10, 2011 issue of The Lancet.

In effect, all these researchers were having simple conversations with their patients, Michael S. Beauchamp, PhD, of the department of neurobiology and anatomy at the University of Texas in Houston, told MedPage Today.

The Need for Assessment

Although the spectacular achievements being seen with neuroimaging have been garnering headlines, Schiff argued that another important need is far from being met.

"What's missing is an articulated framework for tracking patients and translating what we're learning from the research side into a system of care where they are reassessed over time," he toldMedPage Today.

Right now a person who is severely brain injured typically leaves the hospital with a diagnosis of vegetative state. There is no ICD code for minimally conscious state, no registry to follow these patients, and there is no requirement that the person undergo follow-up examinations.

The heightened understanding that the brain can undergo recovery has not been incorporated into the system, according to Schiff. Follow-up assessments are not reimbursed by insurance companies, and there aren't even many centers that provide these services.

The real hindrance has been this structural flaw in the system, he stated.

Thus far the only progress has been made by the defense department, which has proposed -- although not yet implemented -- the establishment of polytrauma centers, where brain injured veterans will be provided with reassessment of levels of consciousness to allow for adjustments in treatment, Schiff explained.

"As of now, it's very much the case that people just keep the diagnosis they were given when they leave the hospital," he said.

That may not be of tremendous concern for a patient who has been comatose for 5 or 10 years. "But certainly a person who is minimally conscious after a couple of months can have a very wide range of outcomes, including some that might not be acceptable to many patients and others that would be acceptable to all," he said.

Beauchamp concurred with the importance of differentiating patients who have no residual brain activity from those who may improve over time, particularly with regard to their treatment.

"If someone is truly brain dead, and there's no brain activity at all, it's probably not worth investing tremendous time and effort in attempting to rehabilitate them. But if there is some brain function and they just have not been able to let you know, you would definitely want to try to help them," he said.

And while it may be only up to 10% of comatose patients who have this minimal consciousness, it would be a tremendous service to provide more aggressive treatment to the one patient in 10 who does have potential for recovery, Beauchamp told MedPage Today.

Limitations and Next Steps

As to the implications of the brain scan results on prime minister Sharon, both Schiff and Beauchamp agreed that, while the specifics of the tests have not been publicly revealed, it seems unlikely that the treatment team found the types of responses seen in the published imaging studies, where patients were able to follow commands and perform other complex tasks.

Imaging studies have also demonstrated that patients who are vegetative can have a considerable amount of ambiguous brain activity, such as apparently responding to one's name or to a familiar face.

"The researchers who tested Sharon said they saw a response to his son's voice, but that's a lot simpler than being able to answer a yes or no question," Beauchamp said.

"They were very careful not to speculate about any possibility of recovery," he added.

"Animal studies also show that there is quite a lot of highly differentiated activity in the brain occurring without any conscious processing," Schiff noted.

Accordingly, imaging tests such as those given Sharon are really not ready for prime time in the clinic, according to Schiff.

In fact, they will probably not ever be useful as screening tests, because they aren't sufficiently sensitive. The false-negative rate is much too high, he said.

The most important role for brain imaging in these patients is for confirming the presence of consciousness. "When the technology works it's an unambiguous rule-in test," Schiff said.

That imaging tests remain imperfect tools for assessing consciousness was echoed by Stuart Hameroff, MD, director of the Center for Consciousness Studies at the University of Arizona in Tucson.

"Imaging studies don't tell us about awareness, they tell us about neuronal activity, behavior, and perception, which may be conscious or nonconscious," Hameroff told MedPage Today.

"Most people would say that consciousness is an emergent effect happening at a higher level of complexity, but my personal belief is that consciousness happens at a deeper level, inside the neuron, and is perhaps even a quantum effect," he said in an interview.

What's most needed now is the adoption of a variety of tools not only including ever-advancing imaging technology but also skilled clinical examinations being done on a regular basis, to identify misdiagnoses and signs of recovery and adjust treatment as needed, according to Schiff.

"We are trying to fundamentally understand what's happening when recovery occurs in a severely structurally damaged brain and to develop models of the underlying physiology, and then to develop rational therapies for those patients we can help," he concluded.

NPR - Peering Into The Brain, But At What?

This is an interesting segment from NPR's Talk of the Nation Science Friday on the limitations of brain scan imaging - we get a lot of pretty pictures, but there is no real indication that we are any closer to understanding the complexity of brain function.

Peering Into The Brain, But At What?

Listen to the Story

Talk of the Nation

[20 min 21 sec]

• Add to Playlist
• Download

November 4, 2011

Modern brain-imaging techniques have given researchers an unprecedented level of detail about the structure of the brain, but are they any closer to puzzling out how the brain really works? Harvard neuroscientist Jeff Lichtman talks about the limitations of brain imaging, and the challenges of trying to use imaging techniques to decode the brain's behavior.

* * * * * * *

IRA FLATOW, host: This is SCIENCE FRIDAY. I'm Ira Flatow. Your thoughts, your memories, as you know, all come from your brain cells, billions of them packed together in your head. My next guest would like to make a map of how all those cells connect to one another, talk to each other, learn new things, make new memories.

But it's going to take a lot to untangle those neurons. After all, your brain cells are just nanometers thick in some places. They flicker with electrical activity that's just a few milliseconds long, and there's evidence that their connections may vary dramatically from one person to another.

And our brains are changing, adapting, responding to our environments all the time. So how do you capture that? Well, my next guest thinks that we can capture it, and as an expert in imaging, Dr. Jeff Lichtman is a professor of molecular and cellular biology and a member of the Center for Brain Science at Harvard University. He joins us from Harvard. Welcome back to SCIENCE FRIDAY.

JEFF LICHTMAN: Hi, Ira, how are you?

FLATOW: You're - fine, thank you very much. You're an expert in imaging techniques, correct?

LICHTMAN: Yeah, we look at the brain in my laboratory. That's what we do, almost exclusively.

FLATOW: And what would be the ideal imaging technique? What are you looking for to be able to examine all of these connections?

LICHTMAN: The perfect technique, actually, is a combination of two, at the moment, extreme opposites. One is a technique that gives you enough resolution to see the finest connections between nerve cells, which requires resolution at the level of nanometers, as you've already mentioned. And the other is the scale to trace out wires that can extend for centimeters, or if you're a giraffe, from your spinal cord to your toe, even meters.

And those two kinds of technologies are often very different. And to fuse them into one technique requires going a little bit beyond the comfort zone of modern technology.

FLATOW: So we're not quite there yet?

LICHTMAN: Well, we're working on it, but it is awesome, truly disturbing how much data one has to obtain if you wish to map the entire brain at the level of every synapse.

FLATOW: How much data are we talking about here?

LICHTMAN: Well, let's take a cubic millimeter of brain, which is about the size of the smallest point you would see in an image taken with this technique called functional magnetic resonance imaging. So those images show you where blood flow in the brain goes up when you think, and they're very highly resolved. A cubic millimeter is the voxal size, the three-dimensional pixel size.

And one voxal of an FMRI image, if we image that with an electron microscope to see all the synapses with sufficient resolution, that would be about a 1,000 terabytes of data or one petabyte. A terabyte is 1,000 gigabytes. So we're talking about a million gigabytes of data per cubic millimeter, and that's just one cubic millimeter of the brain. And if you wanted a whole brain, you'd need thousands of petabytes, essentially more data than is the digital content of the world, if you will.

So it's more than fits on my laptop, to be sure.

(SOUNDBITE OF LAUGHTER)

FLATOW: Could you learn anything, though, at a single-cell level that, let's say, that Eric Kandel didn't learn with the single cells in sea slugs if you could go down that - drill down that deep?

LICHTMAN: I mean, there's extraordinary advances that have been made, certainly, in understanding the way synapses talk to each other and how they change with experience. But one of the mysteries of the brain is that the network that connects cells is a lot like an Internet network, and that is it's one to many and many to one.

There are interconnections between nerve cells and thousands of target cells, and thousands of different target cells impinge and talk to each nerve cell. And if one ever wants to understand how a network like that works, you actually have to look at the network, and that requires seeing more than a single cell.

FLATOW: And can we learn anything from, let's say, a network of networks like the Internet, that may apply to how the brain works?

LICHTMAN: Yeah, I mean, one interesting thing to keep in mind is that the network of the Internet is connecting the brains of individuals together. So it is a form of communication between the neurons in one brain with the neurons in other brains. That's really all it is: It's just wires that extent our connectivity.

So perhaps the same strategies that are wiring us up are used again when brains talk to other brains. Almost - we don't even realize it, but that may be what's going on.

FLATOW: Is it possible in your imaging world, and in the world you would like to create, if you have the right tools, to actually watch a thought originating or something being remembered?

LICHTMAN: Absolutely, that's the long-term goal of work like this, which is to see first how information about the world gets implanted in the brain. And once it's there, what form does it take that allows it to persist over decades, or if you're very lucky, even over a century?

There must be some structural substrate, a trace, if you will, of that memory, but we have very little idea now because these tools are just now being developed to actually map out what that would look like.

FLATOW: And so do you think these tools will be available in our lifetime, so to speak?

LICHTMAN: Yeah, I guess it depends how old you are.

(SOUNDBITE OF LAUGHTER)

LICHTMAN: I think these - my laboratory and a number of other labs are working very hard right now to generate tools that have the speed to generate these images quickly enough. I'll just give you an example that when we started about five years ago, we were obtaining information at about 1 million pixels of brain image per second, which sounds like a lot, but that's actually quite slow. To do a cubic millimeter of imaging at that rate takes about 140 years, and to do let's say a rodent brain would take about 7,000 years at that rate.

And over the past five years, we've sped up about 100-fold, and we think in the next two years we'll be at about a billion pixels per second. And then doing a mouse brain within a year might even be contemplatable. To do a human brain, however, is still an extraordinary challenge because humans have much bigger brains than mice. But the techniques would be the same.

FLATOW: And if you - let's stay then at the mouse brain level. Would one mouse brain look the same as another mouse brain?

LICHTMAN: Almost certainly not. The little part of the mouse nervous system that we looked at to completely wire - get a wire diagram of, from one animal to the other, even from the left side to the right side of the same animal, where the function should be quite similar - we found every single instantiation of this wiring diagram was unique.

And I think a lot of people take that to mean what's the point of doing this at all, with all this variation. It's worth saying that if you watch two football games, or you watch two chess games, you'll find that every game of a particular sort is different from every other one, but after watching one game of chess, for example, you could infer the rules, so no other game would really be surprising to you.

And I guess that's the same thinking we have here, that there will be certain motifs, certain strategies of connectivity, if you will, of the way nerve cells are connected that from learning from one brain, would allow us to extrapolate in other brains.

FLATOW: Let's get some phone calls in, 1-800-989-8255. Jim(ph) in Muskegon, Michigan, hi Jim.

JIM: Hello, thanks for taking the call. I was curious whether or not you can map any changes in the brain as a result of PTSD, or does your work lead to any treatment possibilities? I particularly had some clients who relived some events based on triggers, otherwise benign things.

LICHTMAN: Yes, I think this is an extremely important point about our primitive knowledge of the brain. Compared to other organ systems, where most abnormalities have a physical, histological trace that you can see in a microscope, for most brain disorders, we don't have a physical trace. And I think this is largely a sign of how low-level our imaging is, relative to the questions.

At the moment, we're far away, to be perfectly honest, from getting a physical manifestation of something like post-TSD, but I think at some point, one would hope that mental illness, learning disorders and other kinds of behavior problems will be amenable to these kinds of studies.

FLATOW: And then I would imagine you need to be able to see a large part of the brain to see how that might originate.

LICHTMAN: Yeah, of course, one doesn't even know where the problem is. And this is this problem of size, the big and the small. You have to be able to accommodate a big area but at very high resolution, and that gives rise to datasets that are just at the moment so large that no one would know exactly how to work with them.

FLATOW: Do you need a supercomputer, you know, like the old Cray or any, put a bunch of them together to get a giant computer to do this? What kind of computer power do we need? Give us an idea.

LICHTMAN: I think what people - there's one thing is to generate the data, and then you need to store it in a large place. So you need storage capacity that exceeds what most people are used to, you know, many petabytes of storage, tens or hundreds of petabytes. And that is already one far end of computation.

But that's not sufficient. You then have to analyze this data to turn these pictures into an actual map, and that requires a kind of computational image analysis that is being developed right now but is very computer intensive. And the way this is done is typically with clusters of computers that parse this large problem into many small, little pieces, and so thousands of CPUs or even GPUs working simultaneously are necessary to do this.

So it is a large amount of computational space, but it's not the classic Cray single supercomputer but many small computers, each working on a teeny-weeny part of a very big problem.

FLATOW: Well, good. Could our home computers become part of a network like that, work together?

LICHTMAN: Well, this is one of the - yes indeed. I think one of the ideas, just as the Galaxy Zoo has been very potent as a way of analyzing images of deep space, my laboratory and a colleague of mine at MIT, Sebastian Seung, and another colleague of mine, Hanspeter Pfister in the engineering department here and several other groups, as well, are thinking about ways of recruiting interested parties to help us do this tracing and mapping out.

So not only your computer but your visual system we would take advantage of, as well.

FLATOW: Yeah because we know that people are much better than computers at visually taking things apart and putting them back together.

LICHTMAN: Yeah, I mean, one of the great ironies of this work is that we are trying to get computers to do something that humans do quite trivially. Any five-year-old can trace these wires. Computers have a hard time doing this. And what we're trying to trace is the wiring diagram that explains basically how humans do this.

It's a very circular and philosophically interesting problem.

(SOUNDBITE OF LAUGHTER)

FLATOW: Jeff, can I ask you to stay with us?

LICHTMAN: Sure.

FLATOW: We're going to go to a break. We're talking with Jeff Lichtman, professor of molecular and cellular biology and member of the Center for Brain Science at Harvard. We're going to pick his brain a little longer, and stay with us. We'll come back. Our number, 1-800-989-8255. You can tweet us @scifri, and we'll continue right after the break. Stay with us.

(SOUNDBITE OF MUSIC)

FLATOW: I'm Ira Flatow. This is SCIENCE FRIDAY, from NPR.

(SOUNDBITE OF MUSIC)

FLATOW: You're listening to SCIENCE FRIDAY. I'm Ira Flatow. We're talking about how your brain is wired and attempts to take a look and make - snap pictures of it, with Dr. Jeff Lichtman, professor of molecular and cellular biology, member of the Center for Brain Science at Harvard University. And if his last name is familiar, that's because he is the father of Flora Lichtman, our multimedia editor. And we thank you for that project too.

(SOUNDBITE OF LAUGHTER)

LICHTMAN: That's the best thing I ever did, or one of the two. I also have another daughter. They're both, the pair, the best things.

FLATOW: Well, we're very happy for you doing that. 1-800-989-8255. Let's go to the phone. Let's go to Steve in Chico, California. Hi, Steve.

STEVE: Good morning, gentlemen.

FLATOW: Hi there.

STEVE: My question is this: If I think a thought in an image, like if I think of an old dog I had when I was a kid, and that image is in my mind, and we know that it comes from brain cell activity, right, but if a surgeon cut into my brain, he would not find a little picture of my dog. He would simply see the neural activity, right?

LICHTMAN: The surgeon wouldn't - yeah, go ahead.

STEVE: Right, so do you have any sense where the actual image is, the picture of that dog that's in my mind? Where might that be in the universe?

LICHTMAN: Well, it's in your mind, that's for sure, and because it's a visual picture, it almost certainly is at least one rendering of it, and there are probably many different parts of your brain involved, but it'll certainly be in the parts of the brain that are responsible for image processing, that take visual information and process it progressively farther along.

FLATOW: It has been clear that it's been very hard to find a local stroke, for example, that damages a small part of the brain where a person ends up with a perfectly normal brain, except the image of their dog is missing. And that implies, of course, that your dog is distributed over a rather large area, or there are multiple copies of your dog.

LICHTMAN: There are places in your brain, and maybe this will come up later in the hour or in the next hour, of - where recognition of faces occur, and if that part of the brain is stroked out, for example, a person can't recognize anyone's face. It's very hard to get very specific memories lost, suggesting this distribution, that it's not localized specially the way you would if you were an engineer, you might put your little dog in one place and a spoon right next to it and your cat on the other side. It's not so clear how it's organized.

FLATOW: And one of the things you write about and we've talked about is how plastic your brain is, right? It can be refolded, reshaped.

LICHTMAN: Yeah, so I think the - you know, the emphasis, especially as we get older, is on how plastic our brains are. But of course there's the other side of the coin, and I think this is often left unsaid, but I'd like to emphasize this, that the purpose of memory is to give you the opportunity based on often one trial learning, at some point in development, a lasting, indelible impression about the way the world is.

And that is a bit at odds with a constantly changing brain. And I think if my own daughter's comments to me are any reflection, as I've gotten older I get the impression my children think that my brain has hardened, calcified. I'm a little less open to new ideas than I was when I was younger.

And I see this as wisdom, not really a bad thing, you know, that I'm left with a brain that's consistent with the world, but of course the world is changing very rapidly now. So this is a somewhat painful thing for people my age, as new tools get invented.

But I think memory's main purpose is not to constantly change but to allow a person to hold on to, for example, how to ride a bicycle. If you learn as a child how to ride a bicycle, you can stop riding a bicycle for 20, 30 years. You get on a bicycle on an adult, and after a moment or two of unsteadiness, you're riding pretty well.

But look at an adult who's never ridden a bicycle as a child, and it's clear there's something about their brain, there's some indelible trace about bicycle riding that's missing. And they have a hard time learning.

FLATOW: But we also have the case where it's been shown that recall is very unreliable, that - isn't it true that we see that you can make up something in your mind, and your mind, at least the scans will show that it's as if you had actually seen it?

LICHTMAN: Yes, I mean one of the amazing things about memory is every time you recall something, it's up for grabs again. And often that is because when you recall it, you want to dress it up with more recent data. So if the recent data tends to overturn something that you remembered earlier, gradually that memory can morph into something that's quite opposite of what the original memory is.

And thinking about something long enough, you can begin to believe things are true that aren't. My brother and I growing up, I kept telling him over and over again a particular thing, that he was adopted, in fact, and I think as he - there was a point in his life when he was uncertain whether this was a fact or not, even though - just because he began running it through his mind, even though I was just teasing him.

FLATOW: Let's see if we can get one more call in here before we have to go. Clay(ph) in Oklahoma City. Hi, Clay.

CLAY: Hey, thanks for taking my call.

FLATOW: Hi, go ahead.

CLAY: Yeah, I would like to hear your - I forgot the scientist's name, I'm sorry. So our brains accumulate information from the time we're born until the time we die. And I want to know what you think about where is that information being stored. Is it being stored molecularly? Much of the research around brains is focused on the neural network and the depolarization, the signals sent to each nerve.

But I believe the information must be stored actually inside of the neurons, giving it identity, reason to respond a certain way. So could you please speak about that? Where might the - where might the information we gain as we grow old be stored physically, molecularly?

FLATOW: All right, good question.

LICHTMAN: I think this is a very good question and one that's somewhat contentious. I think we now understand that synaptic connections between nerve cells molecularly can change in ways that persist for long periods of time. And that has sometimes been mistaken, I think, as thinking that the memory per se is built into those molecules.

Ultimately the brain is just a behavior machine. Input comes in, it churns around inside, and then there comes an output. And that is through the connections between nerve cells. So for example, if I tip your - tap your patellar tendon, and your knee jerks, that's because of a reflex of nerves that activate cells in your spinal cord that then send information back out to the muscle and cause that kick.

You could think of all learning, all information, being the same way. If I say what is two plus two, that goes in your ears, it rattles around by activating nerve cells, and out comes first in your head the idea of four, and if you're a child, then a signal would be sent down to your deltoideus muscle in your shoulder, pulling your arm up. So you wave it back and forth so the teacher can see you, so you can then say it's four.

That is not coded molecularly. It's coded in a wiring diagram that connects the idea that comes in, the idea that's in there, to an output.

FLATOW: Thank you very much, Dr. Lichtman, for taking time to be with us today, very fascinating.

LICHTMAN: My pleasure.

FLATOW: Jeff Lichtman is professor of molecular and cellular biology and a member of the Center for Brain Science at Harvard University.

The Connectome Viewer Toolkit: An open source framework to manage, analyze, and visualize connectomes

Excellent - as research into connectomes increases, it will be cool to have an open source platform for sharing findings and images. Anyone can download the viewer software and look at the images - more importantly, to me, it creates an open access and open source platform for the sharing of brain science information. Other fields of science might learn something from this project.

Before the sharing the article, a brief definition of a connectome might be useful for those who do not follow brain science in all its geeky depth.

A connectome is a synapse-resolution mapping of connections between all neurons in a model organism's brain. In other words, a synapse-resolution circuit diagram of the brain. Current approaches to mapping the connectomes of model organisms employ serial block face scanning electron microscopy (SBF-SEM) and transmission electron microscopy (TEM). The only connectome that has been mapped out to date has been from the flatworm, C. elegans, which has only around 300 neurons. Candidate future connectomes include the fly, with around 10,000 neurons, and the mouse brain, with 100 million neurons.

A projectome is more modest than a connectome, and is defined as a mapping of all inter-areal connections, at the single axon level, within an individual brain.

Source: Kasthuri N, Lichtman JW. The Rise of the Projectome. Nat Methods. 2007 Apr;4(4):307-8.

Here is the introduction to the article, from the open access Frontiers in Neuroinformatics, which explains some of the technology and features of the tool kit that has been developed.

The Connectome Viewer Toolkit: An open source framework to manage, analyze, and visualize connectomes

Stephan Gerhard¹*, Alessandro Daducci¹, Alia Lemkaddem¹, Reto Meuli², Jean-Philippe Thiran¹ and Patric Hagmann²

• ¹ Signal Processing Laboratory 5, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
• ² Department of Radiology, University Hospital Center and University of Lausanne, Lausanne, Switzerland

Advanced neuroinformatics tools are required for methods of connectome mapping, analysis, and visualization. The inherent multi-modality of connectome datasets poses new challenges for data organization, integration, and sharing. We have designed and implemented the Connectome Viewer Toolkit – a set of free and extensible open source neuroimaging tools written in Python. The key components of the toolkit are as follows: (1) The Connectome File Format is an XML-based container format to standardize multi-modal data integration and structured metadata annotation. (2) The Connectome File Format Library enables management and sharing of connectome files. (3) The Connectome Viewer is an integrated research and development environment for visualization and analysis of multi-modal connectome data. The Connectome Viewer’s plugin architecture supports extensions with network analysis packages and an interactive scripting shell, to enable easy development and community contributions. Integration with tools from the scientific Python community allows the leveraging of numerous existing libraries for powerful connectome data mining, exploration, and comparison. We demonstrate the applicability of the Connectome Viewer Toolkit using Diffusion MRI datasets processed by the Connectome Mapper. The Connectome Viewer Toolkit is available from http://www.cmtk.org/

1 Introduction

What nervous systems do – essentially – is to connect. Investigations into the connectivity properties of nervous systems have a long history (Douglas and Martin, 2007; Fishman, 2007). Despite many efforts, contemporary knowledge about the specificity of structural and functional connectivity is still poor. The new field of connectomics is emerging to tackle the challenge of mapping complete neural circuitry, or connectomes.

Connectomes represent the fundamental pathways on which complex spatiotemporal activity patterns evolve. In turn, these activity patterns modify underlying structural pathways. For an understanding of how activity patterns arise (physiology) and what they are able to produce and mean (behavior), it is indispensable to have connectome data (neuroanatomy) on all spatial descriptive levels.

On the cellular level of description, light, and electron microscopy are the main imaging tools for mapping neuronal circuitry. Partial and complete connectomes have been mapped in a variety of organisms and structures such as the nematode Caenorhabditis elegans (Ward et al., 1975; White et al., 1976), the crustacean Daphnia’s optic lobe (Macagno et al., 1979), cat visual cortex (Binzegger et al., 2004), macaque (Felleman and Van Essen, 1991; Markov et al., 2010), the rabbit retina (Anderson et al., 2011), the mouse interscutularis muscle (Lu et al., 2009), hippocampus (Ascoli, 2010), or Drosophila melanogaster (Cardona et al., 2010; Chklovskii et al., 2010; Hampel et al., 2011).

On the macroscale level of description, diffusion-weighted magnetic resonance imaging (MRI) is the main imaging technology employed for mapping the structural connectivity of the human connectome (Hagmann, 2005; Sporns et al., 2005; Sporns, 2011). Magnetic resonance connectomics (Hagmann et al., 2010) is increasingly recognized as a tool for basic and clinical neuroscience. Several methodological advances in image acquisition, reconstruction, and tractography (Wedeen et al., 2008; Johansen-Berg and Behrens, 2009) suggest that automated processing pipelines will make it possible to generate comprehensive in vivo whole brain statistical connectomes.

Despite the big differences in spatial scale and data size, both levels of connectome mapping consist of similar stages. Connectome mapping workflows involve image acquisition, registration and segmentation, data organization and sharing, high-throughput pipelining, analysis, and visualization. Advanced neuroinformatics tools will be required to meet challenges that each stage presents. In this article, we will focus on the development of neuroinformatics tools in the emerging field of macroscale connectomics.

The efficiency of sharing data and source code would benefit if a transdisciplinary lingua franca for programming was available. Especially in the neurosciences, where researchers with varying degrees of scientific knowledge and programming skills meet, a common programming language helps to bridge gaps between theoretical and experimental worlds of investigation. Moreover, the programming language must be high-level, cross-platform, easy-to-learn, and have a large number of scientific libraries available. In recent years, Python¹ has become a viable alternative to Matlab, Java, or C++. More and more, Python is becoming the language of choice in scientific computing communities (Oliphant, 2007; Langtangen, 2009). Python is a free, open source, cross-platform programming language with a rapidly growing number of high-quality scientific libraries and interfaces to legacy code. A special-topic issue on “Python in neuroscience” in Frontiers in Neuroinformatics, and a number of publications (Kinser, 2008; Spacek et al., 2008; Davison et al., 2009) give an indication of the significance of the Python programming language.

Due to improved data acquisition methods, it is now possible to acquire large multi-modal datasets in projects involving thousands of subjects. For instance, the Human Connectome Project² is currently underway and collects Diffusion MRI, fMRI, EEG, MEG, behavioral, and genetic data in a cohort of 1200 healthy subjects. In such large-scale projects, the neuroinformatics challenges of data handling, sharing, and analysis become unnecessarily difficult without common infrastructure and data format standards. Due to their longer tradition, in the fields of volume-based and surface-based analysis in neuroimaging, standardized data formats have already been established such as NIFTI³ for volume-based data and GIFTI⁴ for surface-based data. Importantly for MR connectomics, no common format for network-based data yet exists. To approach the task of specifying such a format for connectivity-related neuroimaging data, the Connectivity InFormatics Technology Initiative (CIFTI) was launched. Furthermore, a dedicated program of the International Neuroinformatics Coordinating Facility (INCF⁵) on standards for data and metadata sharing was established.

For data management and sharing of large and multi-faceted datasets, a flexible data format is necessary. The key requirements of such a flexible data format under the macroscale connectomics perspective are severalfold:

⊳ A standardized container format for raw and processed multi-modal datasets that is based on common neuroimaging data formats, extended by a standard format for network-based datasets.

⊳ A minimal set of required metadata that can be extended flexibly by user-defined metadata, and that allow easy sharing of data and metadata across collaborating groups.

⊳ The possibility of relating different data modalities to each other.

⊳ An interface to database infrastructures.

⊳ A mapping to an object model in common programming languages.

⊳ To enable the storage of behavioral data.

⊳ To enable the storage of provenance information such as processing scripts and runtime environment

⊳ The ability to link data and concepts to semantic frameworks.

⊳ To enable easier data visualization (Benger, 2009) and analysis.

To establish a novel data format, it must come with appropriate libraries for reading and writing. Only when it is possible to easily read, modify, and save data in the new format, does it benefit the researcher who wants to focus on analysis and visualization. There exist many standard formats for the different data modalities, but no one has yet tried to combine these multi-modal datatypes into a single format.

Complementary to common data formats, investigators in the field of macroscale connectomics will require interactive research and development environments for data analysis and visualization. An optimal solution would be an integrated neuroinformatics environment based on Python. It would need to provide a graphical user interface (GUI) with extensive libraries, an interactive scripting shell and built-in script editors with code-highlighting and debugging functionality. Graph analysis libraries are required to unravel the complex brain network organization of structural and functional systems (Bullmore and Sporns, 2009). Furthermore, such a macroscale connectomics research environment needs to support an interactive mode of analysis and visualization of multi-modal datasets. Scripting interfaces provide the required flexibility to allow the implementation of a variety of multi-modal data exploration and data mining strategies in an interactive way (Akil et al., 2011). The environment needs not only to provide a methodology to automatically perform elementary functionality, but also guidance for the performance of more complex analysis and visualization tasks. Moreover, a modular software architecture fosters contributions by the open source research community. Open interfaces facilitate the reuse of a diversity of tools and external libraries that the connectome researcher can draw from.

We used the Python programming language to develop the free and open source Connectome Viewer Toolkit. We specify the Connectome File Format (CFF) as a container data format for multi-modal neuroimaging datasets, specifying a connectome file. We present the Python-based Connectome File Format Library (cfflib) for data manipulation and data sharing of connectome files. The Connectome Viewer provides a framework for interactive visualization and analysis of connectomes and multi-modal datasets. We will illustrate the application of the Connectome Viewer Toolkit on Diffusion MRI datasets processed by the Connectome Mapper. The Connectome Mapper is a Python-based tool that currently implements structural connectome mapping. Figure 1 summarizes the general connectome processing workflow for structural and functional data. It highlights the use of the Connectome Viewer tools presented in this article.

FIGURE 1

Figure 1. General processing stages of a connectome workflow. The Connectome Viewer Toolkit currently supports the workflow highlighted in yellow. Mapping streams for structural data, such as the Connectome Mapper, or functional data may converge to a connectome file and can be further managed, analyzed, and visualized with the Connectome Viewer. Connectome files may be reused in other frameworks for analysis and visualization tasks.

Citation: Gerhard S, Daducci A, Lemkaddem A, Meuli R, Thiran J-P and Hagmann P. (2011, June 6). The Connectome Viewer Toolkit: An open source framework to manage, analyze, and visualize connectomes. Frontiers in Neuroinformatics, 5:3. doi: 10.3389/fninf.2011.00003

Tags: The Connectome Viewer Toolkit, open source, framework, manage, analyze, visualize, connectomes, connectomics, connectome, neuroimaging, python, multi-modal data, data management, network analysis, visualization, brain, mapping, imaging, neuroscience, neuroinformatics