ADVERTISEMENT
latest stories:

No Easy Pieces: Sloan Telescope Builders Battled Moths, Balky Software and Broken Mirrors

A new book about the Sloan Digital Sky Survey documents the myriad unexpected challenges in undertaking a large and ambitious astronomy project
SDSS book by Ann Finkbeiner



COURTESY OF SIMON & SCHUSTER

Editor's note: The following is an excerpt from A Grand and Bold Thing: An Extraordinary New Map of the Universe Ushering In a New Era of Discovery by Ann Finkbeiner (on sale August 17 from Free Press). The book chronicles the development of the Sloan Digital Sky Survey, an influential astronomical survey that has charted the position of hundreds of thousands of galaxies and turned up large numbers of distant quasars. The excerpt below details some of the problems encountered by Princeton University astronomer Jim Gunn, the project's leader, and "French" Leger, an engineer at the University of Washington in Seattle (UW), in preparing the telescope at Apache Point Observatory in New Mexico for the survey.

One item on the observers' checklist was to inspect for moths. Apache Point called them miller moths—later research revealed them to be the adult form of the black cutworm—and every spring in New Mexico, they migrate to high elevations. During a bad year, Apache Point buildings could have hundreds of moths a day. Moths like dark, tight spaces, including the insides of electronic devices and the drives of telescopes. When the telescope moves, the moths are crushed on the drive surfaces, and the telescope slips or loses track of where it is. So the telescope drives had to be cleaned during times when observers had better things to do. "The moths cause us all kinds of hell," said French.

A college student interning with the UW engineers was commissioned to investigate moth-ejection measures. The moths were most active during hailstorms and seemed agitated by jingling keys. So the UW student, who had a scientific bent, smacked the walls and floor, then stomped and clapped, but the moths showed no sign of being annoyed until the racket was a few inches away. Then he aimed a speaker at a group of moths and broadcast sounds in a wide range of frequencies. At 0.4 kilohertz, the moths startled but settled back down; at 0.6 kilohertz, one moth moved briefly; at all other frequencies, the moths remained unconcerned. Then he tried shining beams of both incandescent and fluorescent light by turns on the group, and they all moved to the edge of the light. He turned the light off and back on again and forgot it until the next morning, when he found that the moths hadn't moved at all. Next, following advice from Apache Point veterans that blowing air seems to profoundly disturb moths, he proposed what he called a conceptual deterrent system that involved an air gun, small rubber tubing, a valve, and an electric timer. He built the system and blew jets of air at the moths; they tended to hunker down and wait until the jet was over. He was impressed that they could handle upward of 80 pounds per square inch, though they became restless with as little as 20. He installed this air-jet blowing system on the telescope and while it was working, no moths were run over—implying, he thought, some degree of success.

Further research found that, as French said, "interrupted air blasts they do not like." Eventually he built a moth ejector system that blew air intermittently through pipes with holes along their sides: "A 2 hertz puffer system is what we found to be most effective." He mounted the system on the most sensitive parts of the telescope, and though the moth problem didn't completely resolve, it was much improved.


The telescope's real problem was the one that had made first light so nearly frustrating: it couldn't point. The observers would type in the coordinates on the sky and hit Enter, and the telescope was supposed to go to those coordinates immediately. Night log May 29, 1998: "Another clear night. We aimed to fill in the stripe of the data we took last night. In order to do this, we had to obtain the current position of the telescope. We got on the sky at 9:10 p.m.; we had a position at 11:15 p.m."

Or they'd try to find a kind of star with standard and known brightness, called an FK5 star. Night log May 16, 1999: "We were unable to acquire our field. We were then unable to acquire an FK5 star. We were then unable to acquire Arcturus." The pointing was 3 arc minutes off, a tenth of a moon off, and the telescope was aiming at something that was an arc second across. The observers that night were Chicago and Princeton postdocs, and the Chicagoan, Scott Burles, wrote the night log: "At 1 a.m., we found Arcturus and returned to our FK5 star. We chased the fast and wily FK5 star for several hours." "Fast and wily" is a joke; no FK5 star is anything but steady and predictable. "Can't wait to get up in six hours and do this all over again," he wrote.

Part of the pointing problems were, strangely, because the telescope thought it was in the wrong place. The telescope has to know where it is on the earth: at the North Pole, Polaris is straight overhead, but at Apache Point, Polaris is 57.2 degrees off north, and the telescope didn't seem to know that. The problem was that the telescope had been installed with the slightest tilt, about one millimeter off perpendicular. That one millimeter on earth, translated to the sky, meant that when the telescope thought it was looking straight up at Polaris, it was actually looking 1 arc minute off to the side. One minute of arc translated to the 8,000-mile-diameter earth, Jim figured, meant the telescope thought it was living about a mile away from Apache Point. Jim thought the sensible solution was to just lie to the telescope, to tell it that it was a mile west, out in the rift valley. The UW engineers thought this was a jaw-droppingly brilliant solution.


Pointing was one of only two things the telescope had to do; the other thing was tracking, and it couldn't track either. The advantage of drift scanning is that the telescope sits still and the earth turns it across the sky with a steadiness and smoothness no telescope controls could hope for. The disadvantage is that drift scanning works effortlessly only when the telescope aims at the celestial equator, the great circle drawn on the sky directly over the earth's equator, where the stars move in long arcs that look like straight lines and slip evenly across the CCDs. But aim the telescope up off the celestial equator, and the stars move in increasingly tight arcs until, near Polaris, they are moving in small circles. So to get the star moving evenly across the CCD, the telescope needs to track, to move automatically to counter the earth's motion.

A year after first light, in May 1999, the observers were still reporting they couldn't get the telescope to track. It would refuse to move at all, it would move at inconstant speeds, it would oscillate violently, it would start to move and then slip its leash and run away to the zenith. One night that May, Dan Long—one of the few observers who was not a PhD astronomer, just experienced and natively smart—was out on the cliff with the telescope and later wrote up a report: "I heard the sound the alt lvdt makes when the windscreen changes position with respect to the altitude. I looked up with a flashlight to see oscillation in altitude. By the time I got to a stop button, the oscillations were so violent that the turnbuckle rods in the windscreen were slapping the sides of the windscreen making a loud clang. At least that is what I hope the noise was and not the telescope hitting the windscreen."

As a result of the telescope's bad behavior, most of the data during that first year came from drift scans at the celestial equator —nothing was wrong with that except the data were all coming from a strip near the horizon, and the Sloan was supposed to be surveying the whole sky. Many of these problems were in the software, much of which had been handed off from the UW engineers to Fermilab, whose software writers, said Jim Gunn, "had never seen a telescope." So Princeton took over the software: Robert Lupton, whose PhD thesis had been the code that analyzed the data from PFUEI and Four-Shooter, went to Apache Point for three months and rewrote the code until it worked.


So far, the problems had been more or less as expected. But on October 19, 1999, an Apache Point engineer named Jon Brinkmann was showing another engineer, John Briggs, where on the telescope the spectrographs were mounted. Briggs looked up through the open mounting hole and said, "Jon, is that a crack?" Brinkmann and Briggs moved their heads back and forth, to see if it was just a trick of lighting; it wasn't. Brinkmann called French. "French," he said, "do you know there's a crack in the secondary?"

"Is that really a crack," French said, "or is there something on the mirror?" So they crawled up inside the Tinkertoys to the mirror, and sure enough, it was a crack. French walked over to the operations building to get Bruce Gillespie and brought him back. "See anything weird about that mirror?" Briggs said to Gillespie. "Look at the center."

"Shit," Gillespie said. "Oh shit."

This was scary: mirrors are under high internal stress, and cracks propagate. All work stopped, nobody touched anything. French and Gillespie called Jim Gunn and then Fermilab's Bill Boroski, the project manager who had just replaced Jim Crocker. That was early afternoon, and within a few hours everybody was discussing options on phonecons. French thought it might be a project killer and worried about people's jobs: fixing the mirror could cost half a million dollars and take a year or two, and French had just moved with his wife from Seattle to Apache Point, and newly hired observers had been buying houses nearby.

Jim and Boroski both flew out the next morning; glass experts from Arizona's Mirror Lab came to inspect. The mirror had cracked in a meandering circle around its center, the crack stopping an inch or so before closing on itself. "Oh, man," French thought, "it could shatter the whole mirror, it could fall into a million pieces. You just don't know what's going to happen, mirrors are very unpredictable." Jim, French, and another new UW engineer, Larry Carey, covered the cracked center with foam and tape, then gingerly took the mirror off the telescope and moved it into the support building. The Mirror Lab experts drilled tiny holes at the ends of the crack, a temporary fix that stopped the crack from continuing.

Jim flew to the Mirror Lab and everyone discussed what to do. They examined the mirroring surface and found it had not distorted. The glass was under too much stress to just squirt cement into the crack. But because the primary mirror has a hole in its center through which light passes on its way back from the secondary, the center of the secondary doesn't receive much light. So French drove the mirror to the Mirror Lab, the Mirror Lab cut out the crack, leaving a hole in the center, and capped the hole. French's worries had been pessimistic: the fix cost only around $50,000, and the mirror came back to Apache Point in three months, the following January, the beginning of the new millennium.

Meanwhile, Jim rechecked the design he'd done on the computer for the mirror's controls and found he'd made some typos that caused one little rod to be a little too long, so every time the mirror moved, it hit the rod and eventually broke. "There was no problem with the secondary until I broke it," Jim said. "Most decidedly it was my mistake." Gillespie thought it was Jim's dark moment: "Nobody yelled at anybody about it, though," he said.

Michael Strauss thought the project had just about used up most of its nine lives. The Sloanies all knew they were running on borrowed money, and they thought, as Jill Knapp said, that their sponsors' reaction would be that the broken mirror was "the last damn straw—these people can't do anything right and now they've gone and broken their mirror. Forget it." That the sponsors worked out a way to stay in the game, Jill thought, was partly because of Strauss's and Fan's quasars. "This beautiful science—it was like lifting up the corner of the tent and getting a look at what was inside, and the world could see it. And that got us through, I think." A month later, on January 26, 2000, the fixed secondary was reinstalled on the telescope, and it worked just fine. The observers went back to battling moths and white sands, rebooting software, and looking for sucker holes in the cloud cover.

Excerpted from A Grand and Bold Thing: An Extraordinary New Map of the Universe Ushering In a New Era of Discovery by Ann Finkbeiner. Copyright © 2010 by Ann Finkbeiner. Excerpted with permission by Free Press, a Division of Simon & Schuster, Inc.

Share this Article:

Comments

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Scientific American Holiday Sale

Give a Gift &
Get a Gift - Free!

Give a 1 year subscription as low as $14.99

Subscribe Now! >

X

Email this Article

X