The symbolic language of mathematics can be beautiful—but even more striking are some of the patterns and forms that arise from the visual representation of math.

The power of mathematical images attracted Nina Samuel, an art historian, to investigate how visual information can inform scientific discoveries. "Images are not only a by-product but at the core of science," says Samuel, a visiting assistant professor at the Bard Graduate Center in New York City. She gained access to hundreds of papers and images in the office of mathematician Benoît Mandelbrot soon after his death in 2010. "I found images I didn’t expect to find," she adds.

A selection of those images, courtesy of Aliette Mandelbrot, the late mathematician's wife, make up an exhibit, The Islands of Benoît Mandelbrot: Fractals, Chaos and the Materiality of Thinking, now at the center's Focus Gallery in Manhattan. The display opened early this fall and will run until January 27, 2013.

The exhibit features a wealth of previously unpublished work: printouts of experiments on polynomial equations, found images such as magazine ads and photographs that reminded Mandelbrot of fractals as well as drawings from other mathematicians. Samuel curated the show based in part on her PhD thesis in art history at Humboldt University of Berlin.

Mandelbrot, a Polish-born French and American mathematician, invented the word "fractal" in 1975 to describe complex shapes that remain so at different levels of scale and show self-similar patterns. Fractal geometry is often found in the jagged edges of nature—ferns, clouds and mountains, for example—but the artificial fractal images that most resemble these forms are produced by trial and error. Some computer-generated fractal mountains do not resemble any rocky spires of earthly geology—they are crazed pinnacles of implausibility. As Samuel explains, “You can make the connection to nature if you want to, but you really need to want to.”

Images from the field of fractal geometry entered popular culture in the mid-1980s via wildly colored, psychedelic images of a fractal that Mandelbrot investigated and now bears his name, the Mandelbrot set.

In his memoir, published after his death, Mandelbrot wrote, "I don't feel I ‘invented’ the Mandelbrot set: like all of mathematics, it has always been there, but a peculiar life orbit made me the right person at the right place at the right time to be the first to inspect this object, to begin to ask many questions about it, and to conjecture many answers."

To demonstrate the process of discovery, Samuel intends viewers to first see the back wall of the small exhibit room, which is covered with "The White Series." The series is Mandelbrot and his team's effort to visualize complex polynomial sets. While working on these problems, the team was able to see solutions by running the equations through a computer. A small selection of those printouts is arrayed on the back wall of the exhibit. The exhibit brochure explains, “Seeing patterns in these pictures helped Mandelbrot decide how to adjust the input of data. Forming an important part of Mandelbrot’s thought process, these changing forms provoked new questions and led to new discoveries.” The spread hints at this, and with a little imagination the viewer can see the process—some sheets show cloudy, bubbling forms boiling up from the white page. Others show hints of shapes that foreshadow the iconic Mandelbrot set. Still others are barely freckled with dots.

Visitors can look through a peephole in the wall to see a hidden copy of a 1985 Scientific American issue, which features a colorful picture of the Mandelbrot set on the cover. Samuel explains that she wanted to show the scientific process, which is often outside of public view, and hide the more familiar, multihued image.

Printouts similar to the white series led to the discovery of speckles surrounding the larger set. At first, Mandelbrot thought that the specks might be dirt from the printer. Zooming in to the image with the computer allowed him to clear “ambiguity between mathematics and dirt,” Samuel says.

Mandelbrot sought to understand if those speckles, which he dubbed "islands," were connected to the larger whole. The first time the set was published, in the Annals of the New York Academy of Sciences, the publication’s photo editor removed the dots—likely mistaking them for dust or a printer artifact. Samuels recalls that even years later Mandelbrot would get upset about this error. When he sent the text to colleagues he drew dots in by hand. Later, mathematicians Adrian Douady and John H. Hubbard would mathematically prove that the term "islands" was a misnomer and the speckles were indeed part of the whole.

The exhibit also reveals the work of Mandelbrot's contemporaries. Visitors can turn away from the white series and examine a scrolling printout of two lines crossing each other. Neat handwriting on the top of the printout proclaims it the original. It is the handwriting of Edward N. Lorenz, a meteorologist, mathematician and pioneer of chaos theory. The printout is a visualization of chaos—both lines are from equations that are initially very similar to each other, but over time they become drastically different. Chaos theory explains that it is nearly impossible to make long-term predictions about certain systems—the weather, for example—because they are highly sensitive to small initial perturbations.

The entire exhibit explores the idea that whereas the computer enabled a revolution in mathematics and visual thinking, the interaction of hand, pencil and paper remains vital to understanding and discovery. One section displays sketches and letters from several researchers that show their visual thought processes as they shape and advance theory.

The drawing corner includes four pages covered in blue-ink sketches by Otto Rössler, a German biochemist who works on chaos theory. In these drawings Rössler starts with a dynamic system known as the Lorenz attractor. Then, with pen and paper, he discovers a new type of chaotic attractor. The process of drawing is the process of discovery, Samuel says. Before he put pen to paper, Rössler did not know the equation for the new attractor, but by the fourth page it is clear in his mind. The moment he discovers the final form of his attractor is marked with a green arrow.

The exhibit packs a great amount of information and history into a small space. For this it is notable. But the show also offers a rare opportunity to peek into the workings of renowned math minds.