CAPAtrak Blog

Comments on Quality

Posts Tagged ‘Wil Swan’

Lessons Learned While Reaching For the Stars

Posted by Diane Kulisek on September 18, 2009

apollo_11_launchLast night I had the wonderful opportunity to attend a special session of the American Institute of Aeronautics & Astronautics (AIAA) Space Transportation Technical Committee at the Hilton in Pasadena, CA (home to NASA’s Jet Propulsion Laboratories and CalTech).  I wish to thank Bob Tarn, a colleague of mine from The Engineers Council and Pratt & Whitney – Rocketdyne, for sharing an invitation to the event with me.

For those of you who may not know this, despite my obviously enduring passion for quality, my first and greatest love has always been for manned space flight.  I was enthralled.  You couldn’t have kept me away from this event with a flame thrower.

A few words of praise for the Hilton Hotel in Pasadena

Even though it may seem off topic, I like to give praise where praise is due, so bear with me about this.  When I arrived at the Hilton, I was really pleased with how easy the things I needed were to find.  Google maps helped get me from Simi Valley to the right address in Pasadena, of course, but the short term parking for AIAA attendees was very clearly identified and my specific event was on the list shown me by the parking attendant, who cheerfully told me which level the event was being held on, the easiest way to get there from the parking structure and which level would probably be best to park on.  Wow.  Kudos to Hilton for that!

Traffic had been kind of heavy and I was running just a few minutes behind my schedule, so I walked as quickly as I could into the hotel, right onto the floor where the event was located…. but I knew that finding the meeting room might be a little bit of a challenge.  Not so.  An employee wearing a Hilton badge was right inside the entrance and kindly directed me to the proper room.  Wow, again.  Hilton has it down, eh?!

Meet the Moderator and Panel

When I got into the room where the presentation was to be held, I noticed three gentlemen sitting, panel style, at a table on a riser, with another gentleman standing at a podium to my right.  I believe that the gentleman at the podium was the Event Moderator, probably Peter Montgomery, Deputy Branch Manager, Space and Missiles Testing for the Aerospace Testing Alliance/Arnold Engineering Development Center at Arnold AFB in Tennessee, who had sent out an update, adding one of the panelists, earlier in the day.

Those three fellows on the panel were pretty remarkable people and deserve an introduction here.   

First, there was John Casani, former Program Manager for several major spaceflight projects, including Voyager, Galileo, and Cassini, currently Special Assistant to the Director at JPL.

Next, there was Wil Swan, former Program Manager for the Apollo landing impact systems.  Will also worked on a variety of satellite programs during his time in the industry.

Last, but certainly not least (none of them were ‘least’), there was Chuck Lowry, former Program Manager for the parachutes on the Apollo program and consulting today on parachutes for Orion and Blue Origin.

Each of these esteemed speakers were asked to share a couple of memorable moments from their work on space transportation programs.  Although the word ‘quality’ was not mentioned much, the stories often had to do with something (or someone) not doing what was expected or not working the way it was supposed to work.  Their descriptions of the problems flowed smoothly through to root cause analyses and explanations of the solutions.

Following are those examples.  Please forgive me for not being able to write fast enough to capture the details any better:

From Wil Swan

  • Wil Swan discussed how, in order to determine what pressure to apply to prevent landing wheels from locking, a fifth wheel was added to the typical 4-wheel landing configuration.  The purpose of the fifth wheel was to collect pressure data so that the amount of braking pressure to be applied and/or withstood prevented the main wheels from locking.  This was the first application of an anti-lock braking system.  One can only imagine how many landing wheel lockings it may have taken to figure out this might be necessary.
  • Wil, at the jovial urging of his colleagues,  explained his infamy for having been involved with the spectacular sinking of a capsule due to an unknown water landing impact pressure (after it was thought that couldn’t happen).  This was solved by ‘gluing’ additional/thicker heat shielding to the capsule.  I guess, in both of Wil’s cases, more really was… more.

From Chuck Lowry

  • Chuck Lowry noted that, when things did go wrong, there was usually a lot of embarrassment.  He cited an example involving what was eventually found to be the failure of an MRI Redstone mission abort indicator.  When the vehicle was supposed to have launched, it just hovered a little bit and then sat right back down on the pad.  Unfortunately, without the mission abort indicator functioning, the stage separation pyrotechnics and parachute deployment pyrotechnics activated, perfectly, with the vehicle still sitting on the pad.  Once events had been set in motion, all anybody there could do was watch.  It sounded like it was quite a show.  
  • Chuck continued on to describe how a NASA project for parachute testing  had  required some poor guy to ride in the back of a plane and manually arm the pyrotechnical device for the parachute just prior to dropping the test vehicle out of the plane.  It sounds as though it was a hair-raisingly risky job and, wouldn’t you know it, the first test failed miserably.  When they pried the failed test vehicle from its position, deep in the dessert ground, it was found that the pyro device had not been armed.  Naturally, the company moved quickly toward firing the guy who was supposed to have flipped that switch.  NASA intervened to save the guy’s job, saying something like: “We’ve just invested a hundred million dollars in training this guy.  He’s the LEAST likely person to EVER repeat this particular mistake.”

 From John Casani

  • John Casani’s first contribution to the conversation had to do with a Mariner 69 launch effort.  Apparently, a pre-valve had been left open and ignitable gas was spewing out of the tank.  Some guy from General Dynamics actually ran under the vehicle to shut off the valve and save the spacecraft. This led to John’s observation that people matter most in the success of a program.
  • John’s second example to illustrate this concept had to do with a technician working on the Magellan launch of the Titan.  There was a Lockheed Martin Safe/Arm pyro device for internal and external wrenches.  The technician had gone over the system with the QA representative and everything checked out, but something was nagging at him.  After he had left the base, he asked the driver to turn around and take him back.  Sure enough, during a drafting translation from the supplier drawing to the customer drawing, the internal and external wrenching plugs had been reversed.  His intuition, experience and willingness to act upon them had saved the mission.  

 What Mattered Most?  People and Communication.

A general discussion about what mattered most in their experience ensued and, to a person, they all agreed that people were the most important element for success.  People needed to be alert, listen carefully to one another (with this being cited as the most important skill), be mindful of the work they were doing and look to the frontline for the most crucial information.  Being willing to make tough calls was critical, such as was the case for the situation whereby John had to call a mission abort when he had evidence that the internal and external power check (pyrotechnic) switch for one vehicle were not working correctly.  Chuck said that, without a doubt, technicians actually work out solutions to more real problems than engineers.  Based upon my personal experience, I sure agree with that.  My advice is to never, ever, underestimate the knowledge, ability or courage of those working on the front line.  

 The Importance of Knowing How to Listen

The moderator for the session, Peter Montgomery, mentioned something he had once been told by Col. Vic Whitehead, USAF (Ret.), former System Program Director for Expendable Launch Vehicles and former Vice President of Space launch Systems for Lockheed Martin Astronautics.  Vic had told him that, when you are monitoring the launch command channels, which there can now be about 24 of working simultaneously, you cannot listen to all of the words on all of the channels.  So, Vic said, you listen to the tones of voice.  If you hear changes in the tones of voice from what they normally sound like, you tune into the words.  An interesting way to address information overload, eh?  Listen for the critical changes in the behavior of others.  I imagine this would apply to watching for critical changes in the behavior of others, as well.

The Economic Case for Open Communication and Empowerment

John Casani talked about how the cost of on-board payload experiments always being overrun and the challenge this posed for him as a space flight program manager.  He was pleasantly surprised that the solution for that problem came from a CalTech Economics professor.  This professor said, more or less, just open the communications between the various parties with the experiments, disclose ALL of the resources available to them, then let them know: “That’s it.  That’s all there is” and tell them they are free to work it out among themselves through barter, but, when all is said and done, they’ve got to come back to you with their solution.  This took the program manager out of the frustrating, time-consuming, middleman role, empowered the stakeholders, and solved the problem.  From that point forward, the on-board experiments were consistently addressed within budget and on schedule.  Open communication and empowerment were the keys.

The Importance of Expecting the Unexpected or “We’re All In This TOGETHER”

Even when somebody is heroic (or foolish) enough to volunteer for  what seems to be the most risky job for a program, there is no assurance that the actual final risk will behave as it was predicted.  Chuck Lowry was asked to describe his experience with a Mach II (2000Q) ejection seat project.  A physiologist working with the program had agreed to ride in the ‘hot seat’ (pun intended), which had been loaded with pyrotechnics to assess the result of using human restraints upon deployment of the ejection seat.  In order to avoid decapitation during ejection, a face shield being pulled down from over the pilot’s head would activate the ejection seat pyrotechnics.  The physiological challenge was to reduce the pilot’s risk of loosing other body parts, due to flailing amidst shrapnel, by restraining the limbs tightly, as close to the rest of the body as possible, with automated clamping devices for wrists and ankles.  When the test had been set up, and the human guinea pig was mounted on his chariot of fire, those there to witness took their positions behind some thin metal shielding.  Somehow, they had failed to consider the possibility that, when the pyrotechnics went off, the shrapnel would fly OUTWARD.  The test subject, inside the circle of explosions, ended up being in the safest place in the room.

Questions and Answers

At this point, the event was opened up for questions to the panel.

Question 1:  The average age of Apollo flight directors was 28 and the ages ranged from 25 to 30 years old.  Is the reason things seem to take so long today because of the age of the workers?  Would younger people be faster?

Answers 1:  Not necessarily.  Age probably doesn’t have as much to do with it as risk management does.  Flying in spacecraft is dangerous but current space program managers do not want to accept that risk, so the safety precautions slow everything down to a crawl.  If Wilbur and Orville Wright had refused to take some reasonable risks, they would never have flown.  If you don’t want to fly, you should stay home.  Unfortunately, as one ‘common sense’ astronaut told one of the panelists, NASA personnel seem to have a mindset that they must compete with each other to see who can be most conservative with regard to flight safety.  One attendee concurred that safety and liability concerns are hobbling the space program and that there is no tolerance for any reasonable level of risk.  She noted her annoyance with employees of one company wearing t-shirts with the slogan: “Failure is NOT an option.”

Question 2:  How did the failure to handle standard unit to metric unit conversions effectively lead to failure of a multi-million dollar space program?

Answers 2:  The hardware for that program was designed entirely using standard units but, when it came to the navigation system, because the supporting technology had been developed in Europe, the units for it were metric.  This was not taken into account by the young engineer (fresh out of college) who had been asked to design the hardware for the system in standard units.  The unfortunate thing is that there were at least five subsequent points in the program whereby the problem could have been, and should have been, detected and corrected.  In one case, the problem was actually communicated and documented, but the person to whom it had been assigned left his position before he could bring it to closure, and it fell between the cracks.  Failure by many, who could have, to document and follow through on observed problems was identified as the most significant, although secondary, cause of the program’s failure.

Question 3:  How did the failure that led to the deaths of astronauts upon launch in the earlier space programs happen?

Answers 3:  A pressurized pure oxygen atmosphere in the command module was used to keep the weight of the launch load down (to reduce costs), instead of the less volatile oxygen/nitrogen mix used now.  Although the root cause of the spark that caused the highly volatile pressurized oxygen to explode could not be found after the vehicle had incinerated, it was believed to be a an electrical arc caused by wire damage which could have occurred while working in the cramped space of the crew compartment.  

Question 4:  How do you balance risk against cost?

Answers 4:  It needs to be done, but it is very challenging to attempt.  As an example, retro rockets were developed for the Apollo vehicles, to reduce touchdown impact pressure, but were not used due to cost.  It was determined that restructuring the vehicle’s heat shield was a much less costly but reasonable alternative solution for withstanding touchdown pressure.  On a more personal level, we need to come to terms with the fact that we can’t afford to eliminate every risk.  One of the panelists provided this example:  “I was asked if I thought it was possible that I might suffer a stroke someday.  I said that, yes, that was possible.  I was then asked if I thought it was a good idea to have paramedics around, who could respond, if I did have a stroke, to possibly save my life.  I said, yes, I supposed it was.  I was then asked why I didn’t have paramedics on duty, 24 hours a day, right outside my front door.  The answer was that I couldn’t possibly afford that.  So it came down to a trade off between my accepting a certain level of risk because I couldn’t afford the alternative. 

Closing Advice

In their closing remarks, the panelists were asked to share their advice to those just beginning or considering a career in space transportation.  Here is what they said:

1.  Have a morning meeting for at least a ½ hour each day and force people to talk about immediate problems that need to be solved.  Do not let it turn into a status meeting.  Focus on eliminating obstacles and moving forward. 

2.  Demand written requirements, not just from customer outside of the organization, but from your direct customers within the organization.  Make sure it is written on paper or electronically.  Do not let requirements be held only in somebody’s mind.  

3.  Challenge requirements that cross the line between need and want.  Do not let requirements based upon speculation (“what if”) drive up the cost and time it takes to complete a mission.  Use cost to push back on speculative requirements.  Risk is negotiable.

4.  Do not be discouraged by those who demand there be an economic benefit to them.  The manned space program NEEDS to go on.  If every decision was based upon economic benefit, who would have kids?  Asking about the economic payoff is not the right question, because the payoff is not in dollars.  INSPIRATION is the payoff. 

I think that, off all the very valuable information that I was honored to be able to enjoy at this event, these last words of advice were the most valuable:  “Risk is negotiable” and “INSPIRATION is the payoff.”  I can apply this advice to so very many aspects of my life.  I believe you can, too.

Inspiration was definitely MY payoff, at last night’s event.  Thank you, again, to all of those who have lived their lives so well as to be able to share these insights with me and others who may read this.  I hope I captured enough of what was said to spread some of that inspiration around a bit further.


Posted in Day-to-Day Observations, Tools and Methods, Science and Technology | Tagged: , , , , , , , , , , , , , | 6 Comments »