cryptogon.com

Director of National Intelligence Mike McConnell has launched a sweeping technology program to knit together the thousands of databases across all 16 spy agencies. After years of bureaucratic snafus, intelligence analysts will be able to search through secret intelligence files the same way they can search public data on the Internet.

Mr. McConnell’s new technology program is also addressing a more basic problem: Spies often have trouble emailing colleagues in other U.S. intelligence agencies, because email addresses aren’t readily accessible, and messages sometimes get eaten by security filters. Mr. McConnell aims to solve that by uniting the agencies’ email systems into a single system with a full directory that links names, expertise and addresses.

Linking up the 16 agencies is the challenge at the heart of the job of director of national intelligence, created after 9/11. Dennis Blair, nominated by President Barack Obama to succeed Mr. McConnell, faces a confirmation hearing Thursday where senators are likely to ask how he will make agencies with different histories and missions work together.

The new information program also is designed to include Facebook-like social-networking programs and classified news feeds. It includes enhanced security measures to ensure that only appropriately cleared people can access the network. The price tag is expected to be in the billions of dollars, but much of that money will be reallocated from existing technology programs.

The impact for analysts, Mr. McConnell says, “will be staggering.” Not only will analysts have vastly more data to examine, potentially inaccurate intelligence will stand out more clearly, he said.

Today, an analyst’s query might scan only 5% of the total intelligence data in the U.S. government, said a senior intelligence official. Even when analysts find documents, they sometimes can’t read them without protracted negotiations to gain access. Under the new system, an analyst would likely search about 95% of the data, the official said.

Several similar efforts have been aborted in the past decade, because cultural divides couldn’t be bridged between rival agencies. Some of those efforts predated 9/11, and many intelligence agencies have botched their own technology programs since 2001.

Mr. McConnell’s team says this effort, called the Information Integration Program, has experienced officials working on it full-time and is designed to deliver tangible products every few months. “There really is a very different spirit about doing all these things than there was, I think, in the past,” said Prescott Winter, a senior National Security Agency official who is directing the program for Mr. McConnell.

The program is likely to get a review from Mr. Blair. The new administration is expected to make sure it is adequately funded, effective and protects privacy.

The initiative grew out of discussions more than a year ago between the Pentagon’s intelligence chief and Mr. McConnell’s top deputy, who were concerned that military and civilian intelligence data couldn’t be easily tapped. They asked the chief information officers at the six largest intelligence agencies to develop a solution.

Over the summer, the officers began to sketch out the technology and policy problems to be solved, including protecting sources and connecting systems at different levels of classification. They also assembled case studies, which showed that the typical analyst is using technology that is about a decade old, a senior intelligence official said.

In September, all 16 agencies agreed to the goal of creating one searchable data and email system, and Mr. McConnell borrowed Mr. Winter from the NSA to get the program under way.

The first stage of the initiative is to merge the email systems of the six largest intelligence agencies, including the Federal Bureau of Investigation, the Central Intelligence Agency and the NSA. Mr. Winter said that is on track to be largely completed by the end of the month. Then, they will expand to the other 10 agencies. By 2010, the intelligence agencies and the Pentagon would have a single email system.

With the Google-like system, intelligence officials would like to connect the bulk of the databases by the end of the year, though no firm date has been set. The system would search all intelligence data, and quickly determine which data an analyst is permitted to see. Someone focused on one corner of the world may be allowed to see everything available on the countries in the region, but not other regions.