"Oh, sh*t": NASA engineer almost destroyed a $500 million rover weeks before its launch
A mistake made by former NASA engineer Chris Lewicki nearly turned the $500 million Spirit rover test into a real disaster. Only two weeks before the scheduled launch date, the expensive spacecraft could have turned into a pile of scrap metal.
Lewicki told the story of the incident in his blog, describing the work on the Spirit rover at NASA's Jet Propulsion Laboratory (JPL) in Pasadena (USA). The event that could have scuppered the United States' plans to explore the red planet took place in February 2003.
On that day, the spacecraft had to be thoroughly tested for the last time on Earth. Since the rovers were among the most complex spacecraft ever built, everything had to be in perfect working order. Especially considering that NASA's total investment in development was almost a billion dollars.
"No pressure," joked Lewicki, who was 29 years old at the time.
On the day of the final tests, he worked his unofficial second shift, putting in 12 hours. His task was to check the integrity of the motors of the rock abrasive tool (RAT) attached to the Spirit's robotic arm.
To check the motor, there were two options: either to disassemble it after each test, which, as the engineer explained, was impractical, or to identify any problems by looking at the electrical characteristics.
To do this, the researchers disconnected the motor and connected it to an external power source. If the engineers saw an exponential decrease in electric current during the connection to the network, it was a good sign. If there were voltage surges, it was a bad sign.
Lewicki performed this test many times with a junction box.
"My various roles on the project gave me experience deciphering the maze of diagrams showing the 10,000 pin connections that made everything on the spacecraft work, and my responsibility for writing the instructions for wiring and controlling all the motors on the rovers made me an obvious choice for this test campaign," he explained.
When the tests began, a quick pulse to the reference motor confirmed that all was well. Spirit's RAT motor was connected and Lewicki was ready to go.
In order to see the slightest flaws in the engine, it was necessary to give it "as much power as it wants". Therefore, it was very important not to mix up any of the connected cables, as "a wrong connection can lead to catastrophic damage."
The pulse was sent to the engine and the team immediately received the result, but it was not what they expected. As the engineer explained, it looked neither like a working nor a broken engine.
"My mind rushed to find an explanation and in a moment found the most likely one. My eyes followed the wires from our junction box on the test cart to the spacecraft, and the cause of the unfamiliar signal stabbed me in the heart like a dagger," Lewicki said.
It turned out that the powerful charge of energy did not go to the RAT engine, but, due to a faulty switch, was sent "directly to the spacecraft."
"Oh, sh*t," the engineer described his emotions.
At the same time, the team saw that all telemetry from the rover had stopped. Levitsky admitted that he thought he had killed Spirit just two weeks before the launch, and there were very few options to fix the seemingly faulty rover.
But he still had hopes that the surge of energy had gone into something that could handle it and that maybe there was just a large-scale failure related to the incident. To test this theory, the team powered up Spirit, hoping it would reboot. But nothing happened.
Dejected by the mistake and the realization that this could be his last day at NASA, Lewicki.went home and told his wife the story.
"I was convinced that in the morning I would lose my job and my name would be written in a special chapter of infamy in the history of space exploration," he admitted.
The next day, he and his team returned to JPL, trying to analyze the events of the previous evening and understand exactly what had happened. For a while, everything seemed hopeless, until the engineer remembered that before the fatal test, he had used a multimeter, an electronic device that is used to measure various electrical parameters. It was connected to the entire system, but just before the test, Lewicki disconnected the device.
"The monitoring multimeter that I disconnected was actually completing the circuit that powered the telemetry of the spacecraft's ground tests. I accidentally turned off the connection at the same moment I disconnected the (multimeter) wires," he admitted.
When the entire circuitry was restored again, the researchers returned power to Spirit and "it worked."
"There was a collective sigh of relief when the telemetry came back to life - Spirit wasn't dead after all," Lewicki said.
History knows that this incident did not affect the Martian mission, which was a success.
According to The Register, Lewicki later made a brilliant career, including becoming the mission's flight director.
Earlier, OBOZ.UA reported that an AI robot created a catalyst for oxygen production on Mars. It coped with the tests that would have taken people 2000 years.