Behind The Cover: The Story Of The Original fMRI Image

Behind The Cover: The Story Of The Original fMRI Image

By Gary Boas

The cover image accompanying the 1991 Science paper by Jack Belliveau and colleagues reporting the first demonstration of functional MRI is, quite simply, iconic. In that single, evocative picture, we can somehow see the endless possibilities of the emergent imaging technique.

To take a peek behind the cover—to learn a bit more about how the image was conceptualized and ultimately rendered—we checked in with the two authors of the Science paper who were primarily responsible for producing it: Mark Cohen and David Kennedy, both of whom worked with Belliveau in the MGH-NMR Center (now the MGH Martinos Center for Biomedical Imaging).

Here’s what we learned.

Belliveau and his team started thinking about ways to represent the study visually well before they knew Science was going to devote the cover to it. They knew that the report and results were going to be important, said Cohen, who today is a professor in the UCLA Semel Institute for Neuroscience and Behavior, so they wanted to go the extra mile in highlighting the work.

They began to play around with the images from the paper, and in particular with the image from an oblique cut of the head showing the brain activation in response to the visual stimulus. When they learned from the editors at Science that the paper was being considered for the cover, they came up with the idea of presenting the findings in the context of the subject’s head, by producing a computer-generated 3D model of the head.

This in itself was a fairly audacious idea. At the time, cover images for Science were rarely overtly depictive. Amy Henry, the art director for Science, generally preferred more textural images: a photo of a field full of rocks, for example, or a high-resolution microscopy image that played on light and dark. She originally would have preferred to do something similar with this issue, Cohen said, but ultimately agreed that the cutaway of the head would be a compelling way to represent the groundbreaking study.

Of course, actually producing the image was another matter. “Back in those days, surface rendering in 3D was not common,” said Kennedy, who is now the director of the Division of Neuroinformatics, Department of Psychiatry, in the University of Massachusetts Medical Center. But the researchers had access to an advanced Sun workstation with a high-performance TAAC-1 graphics and image accelerator. The accelerator came with a number of demo videos showing surface and volume rendering, so they knew what they wanted to do was possible. And in fact they were able to replace the data in one of the demos with their own MRI scans. (Watch a similar demo here.)

They knew both the angle and the depth at which the oblique scan they were using had been acquired, and were able to cut into the 3D model at the same angle and depth, so the MRI scan in fact appeared in the right part of the head. This much was relatively straightforward. Things got slightly tricky when they had to account for the viewing angle of the head itself. “We wanted enough of the face to be recognizable [as a face], but we also wanted the exposed part of the head in view,” Kennedy said. “We probably spent days arguing over the exact angle.”

Once the head was turned, the MR scans no longer matched the rendering of the head. The researchers could no longer just overlay them because the aspect ratio was now off. So they manually adjusted the skew and the magnification of the image, “probably in Photoshop or some such thing,” to get the registration just right, Kennedy said.

From there, the image was handed back to Cohen, who did a number of adjustments and enhancements—the color toning of the skin, for example—also in Photoshop. But just as they were finalizing it, another consideration arose. It started to look as if the paper were going to appear in the Nov. 1 issue of the journal, so the researchers came up with the idea of making a Halloween cover. “We could go this whole sort of macabre route with it,” Cohen said. The initial thought was to place the image against a black background with the 3D rendering of the head and all of the lettering—the “Science” in the header, etc.—in shades of orange. The cover would even show the skin surface where the oblique cut was made in blood red.

In the end, the decision was made not to go with a macabre cover. Because, well, because it was macabre—an aesthetic not usually associated with Science. Still, Henry, the art director, agreed to keep the orange lettering. It was, after all, Halloween.

Working with the art director for Science, the investigators came up with several possible designs for the cover. These included the Halloween variant shown on the left.

History has shown, of course, that the cover was no less influential for not including the rim of blood on the head. But as iconic as the image now is, the researchers confess that they might do a thing or two differently if they were given the chance.

Not least: the color scale. There was no accepted color scale for functional MRI at the time, simply because, prior to then, there was no functional MRI. So Cohen came up with one. In representing the changes in the brain in the fMRI image, he chose a scale of red to yellow to indicate blood volume increases and a scale of dark blue to cyan to indicate blood volume decreases—based in part on the conventions used in positron emission tomography to show absolute signal. This became the standard in functional MRI studies pretty much instantly, Cohen said. He soon realized, though, that it may not have been the best choice.

Studies have shown that we are physiologically conditioned to respond to colors in particular ways: red is hot and dangerous and active, blue is calm and relaxing. So the color scale used in the image appears to privilege the positive signal changes (represented by red to cyan), as if there were more information contained in the increases in blood volume than in the decreases. Cohen admits that he sometimes worries over the potentially negative consequences of this. “Had we chosen a less active color scale it would have had significant impact on how we treat these images,” he said.

Of course, none of this changes the historical significance of the image—it has been reproduced widely in the scientific and popular press, and even in academic works on art theory and criticism—or the inestimable effect it has had on the MRI and neuroscience communities. The image showed, vividly and for the very first time, that we could in fact localize and visualize activations in the brain. Twenty-five years later, we’re still just beginning to see the entirety of what this can tell us.

The final and alternative cover designs can be found on the fMRI25 Instagram page.