[NYTr] NASA Releases Heavily Censored Airline Safety Survey Data

Tue Jan 1 11:57:53 EST 2008

See also the strange explanation for banning lithium batteries in
check-in luggage (effective today, Jan 1, 2008) because they are a
"fire hazard" but allowing two per passenger in carry-on luggage:

"Lithium Batteries Banned from Checked Baggage as "Fire Risk" 12/31/07
https://blythe-systems.com/pipermail/nytr/Week-of-Mon-20071231/073597.html

The New York Times - Jan 1, 2008
http://www.nytimes.com/2008/01/01/us/01nasa.html

NASA Offers Airline Safety Data

By MATTHEW L. WALD

WASHINGTON — NASA on Monday released an intentionally scrambled, partly
deleted version of the safety data it gathered from 24,000 interviews
with airline pilots, making good on a promise to Congress to make
public information that it said earlier this year would shake public
confidence in the airlines and threaten their commercial interest.

But the agency released the data from the $11.5 million program in a
format that made it difficult if not impossible for outsiders to
analyze in search of trends, presenting the reports as documents rather
than spreadsheets. And the NASA administrator, Michael Griffin, said
his agency had no plans to do additional work with the material, which
he sought to disown in a conference call with reporters.

“It’s hard for me to see any data here that the traveling public would
care about or ought to care about,” he said. “But it’s also not for me
to prescribe what others may care about. We were asked to release the
data and I said that we would, and I’ve done that.”

As released, the survey data no longer linked pilot reports to the type
of plane the pilot flew, the pilot’s experience level or other
particulars. NASA said the reason was to maintain the anonymity of
respondents.

In a sometimes testy exchange with reporters, Dr. Griffin said that it
was never NASA’s intention to do anything more than test methods of
data gathering. The project, called the National Aviation Operations
Monitoring System, sought to uncover safety problems by surveying
pilots, rather than waiting for them to make anonymous reports, or
gathering information from “black boxes” or similar sources.

But Representative Bart Gordon, Democrat of Tennessee and chairman of
the House Committee on Science and Technology, who extracted the
promise from Dr. Griffin to release the information, said that the
thousands of pages of “redacted” and “disaggregated” data put on the
Internet on Monday was “a start but not a satisfactory start.”

Mr. Gordon said the database, which also includes 5,000 interviews with
pilots other than airline pilots, was so extensive that there must be
safety insights to be gained from analyzing it.

“Like penicillin or other types of discoveries, it’s not what you went
in looking for,” he said. “I think we’re going to find some things that
will help us.”

Mr. Gordon and Representative Brad Miller, Democrat of North Carolina,
who is chairman of the investigation and oversight subcommittee,
pledged to push NASA further. Mr. Miller said that “if 80 percent of
the pilots they ask agree to sit still for a half-hour survey,
voluntarily, my conclusion is the pilots had something they wanted
others to know about.”

“This is now 3 years old, and it’s been dumped, unanalyzed and scrubbed
of much of the useful information,” Mr. Miller said.

Dr. Griffin did not say whether the project had yielded any useful
insight into how to conduct surveys, which NASA had expected at one
point would be extended to mechanics, flight attendants and others. He
expressed frustration about his agency’s problems in bringing the
project to a close. It was, he said, “intended to have a beginning, a
middle and an end, and we seem to be unable to end it, which is a bit
frustrating because we don’t have the money to continue it.”

“Researchers being funded by the United States government will always
have a strong belief that their research work should be extended ad
infinitum,” he said.

But Jon A. Krosnick, a professor at Stanford who helped with the survey
design, said in a letter to the Science Committee on Dec. 17 that the
project had been terminated before it could be expanded beyond pilots
to other personnel important to safety.

NASA’s discomfort with the study it paid for appeared to arise partly
from the differing approaches of engineers, who run NASA and the
Federal Aviation Administration, and the social scientists who
conducted the survey. The engineers tend toward gathering numbers from
computers; the social scientists interview people. At the F.A.A.,
officials have said that the reports in the survey were not
sufficiently detailed to be useful. Laura J. Brown, a spokeswoman, said
Monday that the survey had gathered “hangar talk, perceptual data” that
her agency would probably find hard to integrate with its other data
sources. But, she said, “We do intend to see how it can fit with the
rest of our data.”

                                  ***

AP via Yahoo - Dec 31, 2007
http://news.yahoo.com/s/ap/20071231/ap_on_go_ca_st_pe/air_safety_secrets

NASA gives glimpse of air safety survey

By RITA BEAMISH
Associated Press Writer

NASA grudgingly released some results Monday from an $11.3 million
federal air safety study it previously withheld from the public over
concerns it would upset travelers and hurt airline profits. The data
reflects hundreds of cases where pilots flew too close to other planes,
plunged from altitude or landed at airports without clearance.

NASA published the findings — contained in 16,208 pages — but did not
provide a roadmap to understand them, making it cumbersome for any
thorough analysis by outsiders. Released on New Year's Eve, the
unprecedented research conducted over nearly four years relates to
safety problems identified by some 25,000 commercial pilots and more
than 4,000 private pilots interviewed by telephone.

The results appeared to reflect in part at least 1,266 incidents in
which aircraft flew within 500 feet of each other, generally considered
a near miss; at least 1,312 cases where pilots suddenly dropped or
climbed inadvertently more than 300 feet in flight; and 166 reports of
pilots landing without clearance at an airport with an active control
tower. The Associated Press matched the data to the questionnaire that
was used to interview pilots and was obtained separately by the AP.

The data also reflected 513 reports of hard landings and 4,267 cases of
aircraft hitting birds.

Because NASA scrambled the data, it was impossible to determine whether
multiple pilots might be reporting the same incidents, and a key expert
said the numbers appeared inflated. NASA also did not present the data
so researchers could project survey results to overall safety trends.

The data that NASA released was "intentionally designed to prevent
people from analyzing the rates properly and are designed to entrap
analysts into computing rates that are much higher than the survey
really shows," said Jon Krosnick, a Stanford University professor and
survey expert who helped design the project for NASA. He urged NASA to
release more of the data needed for a better analysis.

Citing people familiar with the research, the AP reported earlier that
the data showed events like near-collisions and runway interference
occur far more frequently than previously recognized.

The data was based on interviews with about 8,000 pilots per year from
2001 until the end of 2004. NASA Administrator Michael Griffin said
Monday the survey was poorly managed and told reporters the traveling
public shouldn't care about the data.

"It's hard for me ... to see any data here that the traveling public
would care about or ought to care about," Griffin said.

Griffin dismissed suggestions NASA chose to release the data late on
New Year's Eve, when the public is distracted by holidays and news
organizations are thinly staffed.

"We didn't deliberately choose to release on the slowest news day of
the year," Griffin said.

NASA drew harsh criticism from Congress and news organizations for
keeping the information secret. Rejecting an AP request under the
Freedom of Information Act, NASA explained that it did not want to
undermine public confidence in the airlines or hurt airline fortunes.

Griffin later overruled his staff and promised Congress he would
release at least some data by the end of the year.

NASA's survey, the National Aviation Operations Monitoring System, was
intended to see whether it could help identify problems and prevent
accidents. Survey planners said it was unique because it was a random
survey with an 80 percent response rate and it did not rely on pilots
to voluntarily report safety incidents.

Griffin said NASA never intended to analyze the data it collected, but
planned to pass its methodology to the aviation community.

Pilots were asked how many times they encountered safety incidents in
flight and on the ground, such as near-collisions, equipment failure,
runway interference, unruly passengers or trouble communicating with
the tower.

Griffin outraged some NASA employees by criticizing the project and
saying its methodology was not properly verified. Survey experts who
worked on it said they used state-of-the-art industry techniques and
carefully validated it.

NASA's handling of the matter prompted a congressional investigation
and separate investigations by its inspector general and by a union
representing NASA workers.

The FAA has questioned the project's results showing more safety
incidents than the FAA's own data, saying it reflected pilots'
subjective opinions over time.

"It's just something that we're going to have to try and understand,"
said Peggy Gilligan, a senior FAA official, in a recent telephone
interview. "We are always interested in any kind of safety data, but we
always want to look at it in terms of its quality, its quantity and how
we're going to use it and what assumptions underlie it."

She noted NASA's interview questions didn't track specifically with FAA
report language and said pilot responses were their subjective views
over 30- to 90-day time frames.

Rep. Bart Gordon, D-Tenn., chairman of the House Science and Technology
Committee, urged NASA to finish reviewing the data for further release
as soon as possible.

[AP Director of Surveys Trevor Tompson in Washington contributed to
this report.]