Challenger and the evil 'O-ring'

[Bear669]

........... The Challenger shuttle blew up because statisticians misread the temperature data, thinking the distribution was linear instead of curvilinear. So, they ordered the launch in colder weather than was prudent, and due to the failure of a rubber O-ring (cost: US$0.50) the whole shuttle was destroyed............

Thought this topic was worth its own thread.

I do not doubt TGN's summary of the proximate cause of the disaster. Does anyone recall the fundamental cause? Hint- Why the heck was there an:neutral: 'O Ring' anyway?

 

[papasmerf]

I remember the shuttle all too well.

One of the most disturbing things about it was they found a working cassette tape recorder in the recovered cabin. The recorder was being used by Christa McAuliffe to document the mission. I have always wondered how that could have remained intact when we are supposed to belive the crew was killed instantly in the explosion.

 

[Iambad]

[video]https://nycaviation.com/2010/01/previously-unseen-amateur-video-of-space-shuttle-challenger-disaster/[/video]

 

[Tgirl Nikki]

Hmmm, ok... 4times, are you still here? Elassowipo? WilsonJo? Or any other statisticians who would care to give me a hand? :arf:

Until one of those guys gets here, I'll try my best to explain how a faulty interpretation of the data led them to order a launch, even though they shouldn't have.

First off, here's a summary of the decision to launch in cold weather, including a description of the joint erosion experienced by O-rings:

https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_launch_decision

There's no doubt that the failure of the O-ring was the physical cause of the disaster; the real question is why the launch was ordered in a lower temperature than was prudent, given that the NASA people knew the O-rings were sensitive to drops in temperature.

And here's a graph showing the distribution (temperature on the X axis, erosion on the Y) from the prior launches they had data for:

The limited data was viewed as having a negative linear distribution (erosion goes down as temperature goes up, and vice-versa, along a fairly predictable line; that's the straight line you see in the middle of the graph, between the two semi-curved confidence interval lines). However, that model was incorrect; the proper model would have been curvilinear (starting out like the straight line, but curving further up as it goes further to the left, until it's almost vertical around the 40-degree mark) meaning that, as temperature drops, erosion increases significantly more for each degree colder that it gets.

Linear: Y = (X - 2)
Curvilinear: Y = 2(X - 2)

(I really suck at algebra, hopefully one of you math whizzes can fix this for me if I'm getting it wrong... it's meant to be illustrative rather than accurate).

In plain English, a linear model suggests that a drop in temperature has a cumulative effect on erosion; for each 1-degree drop in temperature, the erosion is increased by a factor of 2 units. A curvilinear model has a multiplicative effect on erosion; for each 1-degree drop in temperature, the erosion is multiplied by a factor of 2 units (again, not real numbers, meant to be illustrative rather than accurate).

Now, if the spread of the data for temperature is only 2 degrees, then nobody notices a difference;

2 + 2 = 4
2 x 2 = 4

It's only when you get farther away from the mean temperature that the differences start to show up. Here's what happens if the spread is increased to 4 degrees:

Linear model: (2 + 2 + 2 + 2 = 8)
Temperature drops by 4 degrees; erosion increases by 8 units
Curvilinear model: (2 x 2 x 2 x 2 = 16)
Temperature drops by 4 degrees; erosion increases by 16 units

The temperature on the day of the launch was around 31 degrees (F). The people at NASA only had data from 23 launches, none of which had occurred at such a low temperature, so they had to extrapolate the limited data into uncharted territory. Because they thought the linear model was the best fit, they assumed the O-rings would hold (around 80 on the Y axis, which, as the graph suggests, would have been within the 95% confidence interval). However, because the distribution was actually curvilinear, the appropriate data point was actually around 135 on the Y axis, which is outside this interval, suggesting catastrophic failure of the O-ring was likely to occur. Sadly, they didn't consider this possibility, and the shuttle exploded during launch.

Here's the more complicated explanation:

The Presidential Commission on the Space Shuttle Challenger Accident (1986) established that the loss of the space shuttle Challenger on 28 January 1986 occurred because "a combustion gas leak through the right Solid Rocket Motor aft field joint initiated at or shortly after ignition eventually weakened and/or penetrated the External Tank initiating vehicle structural breakup". Engineers at Morton Thiokol, the makers of the solid rocket motors, and experts at the National Aeronautics and Space Administration (NASA) debated the possibility of field-joint failure due to primary and secondary O-ring failure the night before the ill-fated launch. The point of contention was the effect of the temperature at the time of launch on the O-ring performance. The engineers were in disagreement about the implication of data from 23 previous shuttle launches on the thermal distress to the field-joint primary O-rings. Despite some objections, at the conclusion of their discussion the engineers at Morton Thiokol transmitted a facsimile to NASA stating that "temperature data [are] not conclusive on predicting primary O-ring blowby". On the morning of 28 January 1986 the estimated temperature of the primary O-rings on the Challenger solid rocket motors was 31 ◦F (−0.6 ◦C). This was 22◦F (12.2 ◦C) lower than the minimum temperature recorded in all previous shuttle launches [◦C = (◦F − 32)5/9]. The Presidential Commission on the Space Shuttle Challenger Accident (1986) found that "a careful analysis of the flight history would have revealed the correlation of O-ring damage in low temperature".

The Challenger disaster has thus become a paradigm for improving risk analysis of the space shuttle (Paté-Cornell and Dillon, 2001). In particular, Dalal et al. (1989) present a procedure for assessing the probability of catastrophic failure at launch due to a failure of 6 at least one of the six field joints. A key input is the probability of primary O-ring damage, conditional on the temperature at the time of launch. They calculate this probability at 31◦F using logistic regression and, admitting uncertainty about the logistic regression parameter estimates, they also construct a 90% bootstrap confidence interval around the calculated probability. But as Lavine (1991) points out, this approach presumes that the logistic regression is the correct model whereas, in fact, it is possible to fit the Challenger O-ring data equally well using other model forms from the class of generalized linear models. He shows that these other models provide vastly different probabilities of O-ring damage at 31 ◦F. Dalal and Hoadley (1991) defend the generalized linear models by stating that a detailed analysis should involve experts who would assign a probability to each of the possible models and then combine the response probabilities.

Lavine (1991) correctly identifies the problem of determining the probability of O-ring damage at 31 ◦F as an extrapolation problem. Assuming a monotone relation between probability and temperature, he computes nonparametric bounds on the probability of O-ring damage at 31 ◦F; these bounds give the interval [1/3,1]. Yet, his suggestion of using the lower bound of 1/3 as the probability of O-ring damage at 31 ◦F in a risk assessment falls short of providing a theoretically justified forecast. First, the proposed approach does not extract all information from data and ignores expert judgment. Second, the proposed lower bound on the probability underestimates the risk of the catastrophic failure. When it is recognized that in order to solve this forecasting-extrapolation problem, the empirical evidence from previous shuttle flights must be combined with subjective input from experts, Bayesian approach is not only structurally optimal, but also practically advantageous. The purpose of this paper is to present a Bayesian forecasting model for the probability of primary O-ring damage, conditional on the temperature at the time of launch, and to compare that model with the generalized linear models and nonparametric approach advocated in earlier risk analyses of the Challenger O-ring data.

Hence, the Challenger disaster is now used as an example in all university-level statistics textbooks. A terrible shame that seven people had to die as a result of faulty data analysis, and today, this debacle serves as a potent warning against assuming too much from a small sample of data.

 

[Bubba]

You got me there Nikki. You lost me after Linear: Y = (X - 2) :666:

 

[Cycleguy007]

You got me there Nikki. You lost me after Linear: Y = (X - 2) :666:

My mind wandered after "curvi"...

 

[Bear669]

The BASIC reason is simpler-

The BASIC reason is simpler-

Again, no issue with TGN's excellent research.

But it still raises the question- why put together a massive rocket with an 'O Ring'?

I read this once in a main stream publication at the time of the disaster; never heard it discussed since.

The rockets could and SHOULD have been manufactured in one piece. But then they are too massive to be shipped by rail to Cape Canaveral. Huge barges are the way to go. UNLESS there is no water access to the manufacturing plant.

It was the pork barrel turn of some senator in some state with the manufacturing facility, BUT no water access. Hence rocket built in pieces, shipped by rail, assembled with 'O ring'.

Bullshit and sleazy politics were the killers. :grrrrrr:

 

[anger]

Bullshit and sleazy politics were the killers. :grrrrrr:

No surprise there Bear.

 

[tboy]

A couple of things: I highly doubt anything to do with NASA costs 50 cents. Especially a critical part on the solid rocket boosters....

O rings were needed there, and millions of other places as a seal to prevent the leakage which occured. Just like foam insulating blocks are required to protect the belly during re-entry.

There's probably a billion other parts that could be made another way if technology existed (in this case the ability to move a massive structure over great distances). Take the new 777 for example (or was it the Airbus?): parts are made all over the globe and then shipped to europe for final assembly. In one case there is only 1 day a week when the tide is low enough to move a part by barge under a certain bridge...and the clearance is inches, NOT feet.

Not sure where I read it but one NASA report stated that the astronauts were alive until they hit the water.....this is plausible since the cockpit was designed to withstand a complete explosion of the main shuttle, tank and boosters. If there was an escape hatch they probably all would have survived. The thing on the report said though: the g forces put upon the astronauts would have prevented them from a) getting to an escape hatch b) probably would have rendered them unconscious and c) caused skeletal damage so they wouldn't have been able to climb out.

Anyhow, if you knew how many people have been killed for advances in space technology you'd wonder why anyone would ever go up. But that's the name of progress and trial and error. I mean if you are worried about the cost of an O ring vs an astronaut's life, just look at how many thousands of soldiers died during WWII because of poor design? Sherman tanks were called Zippos for a reason: they always lit. (The ammunition was stored at the front of the tank with little armament protecting it). And who the fuck's bright idea was it to try and get a tank to FLOAT by using canvas??????

Anyhow, further to that concept: companies are killing thousands in the name of profit so 7 deaths is no big deal (to NASA).

 

[Tgirl Nikki]

Again, no issue with Nikki's excellent research.

But it still raises the question- why put together a massive rocket with an 'O Ring'

Thank you for the compliment, but this is well beyond my area of expertise - I'm pretty good at doing statistical analysis, but when it comes to designing spaceships, I'm not exactly an expert. I'm pretty sure that O-rings are used in joints because they act as an airtight seal, one that compensates for expansion and contraction of metals under extreme temperatures... and while I'm sure the NASA people considered a one-piece rocket, I suspect they had good reasons for ruling it out. I won't presume to know more about shuttle design than NASA's "Rocket Scientists."

Here's what wiki says about O-rings:

https://en.wikipedia.org/wiki/O-ring

O-rings are one of the simplest, yet most engineered, precise, and useful seal designs ever developed. They are one of the most common and important elements of machine design.

I imagine that engineers prefer to use existing technologies with a proven track record of success, rather than designing fancy custom technologies with unknown limitations... especially when there's no guarantee that the custom technology would be any better than the pre-existing technology. Everyone knew that O-rings were sensitive to a drop in temperature... but it was good ol' human error that caused them to order the launch, because they underestimated the O-rings' actual sensitivity to cold.

I'll say this again - there was nothing wrong with using O-rings in the rockets, and they had never failed on any of the previous launches (nor have they since). The problem came from human beings asking the O-ring to do something it wasn't designed to do - withstand a significant drop in temperature without catastrophic erosion.

If you want more in-depth answers to your questions, you'll have to ask the people at NASA... but don't be surprised if they don't get back to you right away. :tongue: The main point is that even the most advanced technology still requires humans to design, create, operate, analyze, and troubleshoot it... and we'll never be able to eliminate the variable of human error from the equation.