- NSA Secrets

- Copyright

- Introduction

- U.S., British intelligence mining data from nine U.S. Internet companies in broad secret program

- From obscurity to notoriety, Snowden took an unusual path

- Edward Snowden, Bradley Manning and the risk of the low-level, tech-savvy leaker

- U.S. surveillance architecture includes collection of revealing Internet, phone metadata

- Metadata reveals secrets of social position, company hierarchy, terrorist cells

- New documents reveal parameters of NSA’s secret surveillance programs

- President’s surveillance program worked with private sector to collect data after Sept. 11, 2001

- For NSA chief, terrorist threat drives passion to ‘collect it all, observers say

- NSA broke privacy rules thousands of times per year, audit finds

- FISA court: Ability to police U.S. spying program limited

- NSA gathered thousands of Americans’ e-mails before court ordered it to revise its tactics

- U.S. spy network’s successes, failures and objectives detailed in ‘black budget’ summary

- NSA paying U.S. companies for access to communications networks

- To hunt Osama bin Laden, satellites watched over Abbottabad, Pakistan, and Navy SEALs

- U.S. spy agencies mounted 231 offensive cyber-operations in 2011, documents show

- Spy agencies spend millions to hunt for insider threats, document shows

- Top-secret files show new levels of distrust of Pakistan

- U.S. documents detail al-Qaeda’s efforts to fight back against drones

- Obama administration had NSA restrictions reversed in 2011

- Declassified court documents highlight NSA violations in data collection

- FISA court releases opinion upholding NSA phone program

- U.S. officials dodge questions on scope of surveillance

- NSA report on the Tor Encrypted Network

- Dual-leadership role at NSA and Cyber Command stirs debate

- NSA tries to regain industry’s trust to work cooperatively against cyber-threats

- Effort underway to declassify document that is legal foundation for NSA phone program

- NSA collects millions of e-mail address books globally

- Documents reveal NSA’s extensive involvement in targeted killing program

- NSA infiltrates links to Yahoo, Google data centers worldwide, Snowden documents say

- NSA tracking cellphone locations worldwide, Snowden documents show

- Judge: NSA’s collecting of phone records is probably unconstitutional

- More from the Washington Post

- Connect with Diversion Books

Government Spying in the Internet Age

The Washington Post

Diversion Books
A Division of Diversion Publishing Corp.
443 Park Avenue South, Suite 1004
New York, New York 10016

www.DiversionBooks.com

Copyright © 2013 by The Washington Post

All rights reserved, including the right to reproduce this book or portions thereof in any form whatsoever.

For more information, email info@diversionbooks.com

First Diversion Books edition December 2013

ISBN: 978-1-62681-212-3

By Barton Gellman

December 23, 2013

An undated handout photo by the National Security Agency (NSA) shows the NSA headquarters in Fort Meade, Maryland, USA. A secret intelligence program called ‘Prism’ run by the NSA has been collecting data from millions of communication service subscribers through access to many of the top Internet companies, including Google, Facebook, Apple and Verizon. (EPA/National Security Agency/Handout)

MOSCOW — The familiar voice on the hotel room phone did not waste words.

“What time does your clock say, exactly?” he asked.

He checked the reply against his watch and described a place to meet.

“I’ll see you there,” he said.

Edward Joseph Snowden emerged at the appointed hour, alone, blending into a light crowd of locals and tourists. He cocked his arm for a handshake, then turned his shoulder to indicate a path. Before long he had guided his visitor to a secure space out of public view.

During more than 14 hours of interviews, the first he has conducted in person since arriving here in June, Snowden did not part the curtains or step outside. Russia granted him temporary asylum on Aug. 1, but Snowden remains a target of surpassing interest to the intelligence services whose secrets he spilled on an epic scale.

Late this spring, Snowden supplied three journalists, including this one, with caches of top-secret documents from the National Security Agency, where he worked as a contractor. Dozens of revelations followed, and then hundreds, as news organizations around the world picked up the story. Congress pressed for explanations, new evidence revived old lawsuits and the Obama administration was obliged to declassify thousands of pages it had fought for years to conceal.

Taken together, the revelations have brought to light a global surveillance system that cast off many of its historical restraints after the attacks of Sept. 11, 2001. Secret legal authorities empowered the NSA to sweep in the telephone, Internet and location records of whole populations. One of the leaked presentation slides described the agency’s “collection philosophy” as “Order one of everything off the menu.”

Six months after the first revelations appeared in The Washington Post and Britain’s Guardian newspaper, Snowden agreed to reflect at length on the roots and repercussions of his choice. He was relaxed and animated over two days of nearly unbroken conversation, fueled by burgers, pasta, ice cream and Russian pastry.

Snowden offered vignettes from his intelligence career and from his recent life as “an indoor cat” in Russia. But he consistently steered the conversation back to surveillance, democracy and the meaning of the documents he exposed.

“For me, in terms of personal satisfaction, the mission’s already accomplished,” he said. “I already won. As soon as the journalists were able to work, everything that I had been trying to do was validated. Because, remember, I didn’t want to change society. I wanted to give society a chance to determine if it should change itself.”

“All I wanted was for the public to be able to have a say in how they are governed,” he said. “That is a milestone we left a long time ago. Right now, all we are looking at are stretch goals.”

‘Going in blind’

Snowden is an orderly thinker, with an engineer’s approach to problem-solving. He had come to believe that a dangerous machine of mass surveillance was growing unchecked. Closed-door oversight by Congress and the Foreign Intelligence Surveillance Court was a “graveyard of judgment,” he said, manipulated by the agency it was supposed to keep in check. Classification rules erected walls to prevent public debate.

Toppling those walls would be a spectacular act of transgression against the norms that prevailed inside them. Someone would have to bypass security, extract the secrets, make undetected contact with journalists and provide them with enough proof to tell the stories.

The NSA’s business is “information dominance,” the use of other people’s secrets to shape events. At 29, Snowden upended the agency on its own turf.

“You recognize that you’re going in blind, that there’s no model,” Snowden said, acknowledging that he had no way to know whether the public would share his views.

“But when you weigh that against the alternative, which is not to act,” he said, “you realize that some analysis is better than no analysis. Because even if your analysis proves to be wrong, the marketplace of ideas will bear that out. If you look at it from an engineering perspective, an iterative perspective, it’s clear that you have to try something rather than do nothing.”

By his own terms, Snowden succeeded beyond plausible ambition. The NSA, accustomed to watching without being watched, faces scrutiny it has not endured since the 1970s, or perhaps ever.

The cascading effects have made themselves felt in Congress, the courts, popular culture, Silicon Valley and world capitals. The basic structure of the Internet itself is now in question, as Brazil and members of the European Union consider measures to keep their data away from U.S. territory and U.S. technology giants including Google, Microsoft and Yahoo take extraordinary steps to block the collection of data by their government.

For months, Obama administration officials attacked Snowden’s motives and said the work of the NSA was distorted by selective leaks and misinterpretations.

On Dec. 16, in a lawsuit that could not have gone forward without the disclosures made possible by Snowden, U.S. District Judge Richard J. Leon described the NSA’s capabilities as “almost Orwellian” and said its bulk collection of U.S. domestic telephone records was probably unconstitutional.

The next day, in the Roosevelt Room, an unusual delegation of executives from old telephone companies and young Internet firms told President Obama that the NSA’s intrusion into their networks was a threat to the U.S. information economy. The following day, an advisory panel appointed by Obama recommended substantial new restrictions on the NSA, including an end to the domestic call-records program.

“This week is a turning point,” said the Government Accountability Project’s Jesselyn Radack, who is one of Snowden’s legal advisers. “It has been just a cascade.”

‘They elected me’

On June 22, the Justice Department unsealed a criminal complaint charging Snowden with espionage and felony theft of government property. It was a dry enumeration of statutes, without a trace of the anger pulsing through Snowden’s former precincts.

In the intelligence and national security establishments, Snowden is widely viewed as a reckless saboteur, and journalists abetting him little less so.

At the Aspen Security Forum in July, a four-star military officer known for his even keel seethed through one meeting alongside a reporter he knew to be in contact with Snowden. Before walking away, he turned and pointed a finger.

“We didn’t have another 9/11,” he said angrily, because intelligence enabled warfighters to find the enemy first. “Until you’ve got to pull the trigger, until you’ve had to bury your people, you don’t have a clue.”

It is commonly said of Snowden that he broke an oath of secrecy, a turn of phrase that captures a sense of betrayal. NSA Director Keith B. Alexander and Director of National Intelligence James R. Clapper Jr., among many others, have used that formula.

In his interview with The Post, Snowden noted matter-of-factly that Standard Form 312, the classified-information nondisclosure agreement, is a civil contract. He signed it, but he pledged his fealty elsewhere.

“The oath of allegiance is not an oath of secrecy,” he said. “That is an oath to the Constitution. That is the oath that I kept that Keith Alexander and James Clapper did not.”

People who accuse him of disloyalty, he said, mistake his purpose.

“I am not trying to bring down the NSA, I am working to improve the NSA,” he said. “I am still working for the NSA right now. They are the only ones who don’t realize it.”

What entitled Snowden, now 30, to take on that responsibility?

“That whole question — who elected you? — inverts the model,” he said. “They elected me. The overseers.”

He named the chairmen of the Senate and House intelligence committees.

“Dianne Feinstein elected me when she asked softball questions” in committee hearings, he said. “Mike Rogers elected me when he kept these programs hidden. … The FISA court elected me when they decided to legislate from the bench on things that were far beyond the mandate of what that court was ever intended to do. The system failed comprehensively, and each level of oversight, each level of responsibility that should have addressed this, abdicated their responsibility.”

“It wasn’t that they put it on me as an individual — that I’m uniquely qualified, an angel descending from the heavens — as that they put it on someone, somewhere,” he said. “You have the capability, and you realize every other [person] sitting around the table has the same capability but they don’t do it. So somebody has to be the first.”

‘Front-page test’

Snowden grants that NSA employees by and large believe in their mission and trust the agency to handle the secrets it takes from ordinary people — deliberately, in the case of bulk records collection, and “incidentally,” when the content of American phone calls and e-mails are swept into NSA systems along with foreign targets.

But Snowden also said acceptance of the agency’s operations was not universal. He began to test that proposition more than a year ago, he said, in periodic conversations with co-workers and superiors that foreshadowed his emerging plan.

Beginning in October 2012, he said, he brought his misgivings to two superiors in the NSA’s Technology Directorate and two more in the NSA Threat Operations Center’s regional base in Hawaii. For each of them, and 15 other co-workers, Snowden said he opened a data query tool called BOUNDLESSINFORMANT, which used color-coded “heat maps” to depict the volume of data ingested by NSA taps.

His colleagues were often “astonished to learn we are collecting more in the United States on Americans than we are on Russians in Russia,” he said. Many of them were troubled, he said, and several said they did not want to know any more.

“I asked these people, ‘What do you think the public would do if this was on the front page?’ ” he said. He noted that critics have accused him of bypassing internal channels of dissent. “How is that not reporting it? How is that not raising it?” he said.

By last December, Snowden was contacting reporters, although he had not yet passed along any classified information. He continued to give his colleagues the “front-page test,” he said, until April.

Asked about those conversations, NSA spokeswoman Vanee Vines sent a prepared statement to The Post: “After extensive investigation, including interviews with his former NSA supervisors and co-workers, we have not found any evidence to support Mr. Snowden’s contention that he brought these matters to anyone’s attention.”

Snowden recounted another set of conversations that he said took place three years earlier, when he was sent by the NSA’s Technology Directorate to support operations at a listening post in Japan. As a system administrator, he had full access to security and auditing controls. He said he saw serious flaws with information security.

“I actually recommended they move to two-man control for administrative access back in 2009,” he said, first to his supervisor in Japan and then to the directorate’s chief of operations in the Pacific. “Sure, a whistleblower could use these things, but so could a spy.”

That precaution, which requires a second set of credentials to perform risky operations such as copying files onto a removable drive, has been among the principal security responses to the Snowden affair.

Vines, the NSA spokeswoman, said there was no record of those conversations, either.

U.S. ‘would cease to exist’

Just before releasing the documents this spring, Snowden made a final review of the risks. He had overcome what he described at the time as a “selfish fear” of the consequences for himself.

“I said to you the only fear [left] is apathy — that people won’t care, that they won’t want change,” he recalled this month.

The documents leaked by Snowden compelled attention because they revealed to Americans a history they did not know they had.

Internal briefing documents reveled in the “Golden Age of Electronic Surveillance.” Brawny cover names such as MUSCULAR, TUMULT and TURMOIL boasted of the agency’s prowess.

With assistance from private communications firms, the NSA had learned to capture enormous flows of data at the speed of light from fiber-optic cables that carried Internet and telephone traffic over continents and under seas. According to one document in Snowden’s cache, the agency’s Special Source Operations group, which as early as 2006 was said to be ingesting “one Library of Congress every 14.4 seconds,” had an official seal that might have been parody: an eagle with all the world’s cables in its grasp.

Each year, NSA systems collected hundreds of millions of e-mail address books, hundreds of billions of cellphone location records and trillions of domestic call logs.

Most of that data, by definition and intent, belonged to ordinary people suspected of nothing. But vast new storage capacity and processing tools enabled the NSA to use the information to map human relationships on a planetary scale. Only this way, its leadership believed, could the NSA reach beyond its universe of known intelligence targets.

In the view of the NSA, signals intelligence, or electronic eavesdropping, was a matter of life and death, “without which America would cease to exist as we know it,” according to an internal presentation in the first week of October 2001 as the agency ramped up its response to the al-Qaeda attacks on the World Trade Center and the Pentagon.

With stakes such as those, there was no capability the NSA believed it should leave on the table. The agency followed orders from President George W. Bush to begin domestic collection without authority from Congress and the courts. When the NSA won those authorities later, some of them under secret interpretations of laws passed by Congress between 2007 and 2012, the Obama administration went further still.

Using PRISM, the cover name for collection of user data from Google, Yahoo, Microsoft, Apple and five other U.S.-based companies, the NSA could obtain all communications to or from any specified target. The companies had no choice but to comply with the government’s request for data.

But the NSA could not use PRISM, which was overseen once a year by the surveillance court, for the collection of virtually all data handled by those companies. To widen its access, it teamed up with its British counterpart, Government Communications Headquarters, or GCHQ, to break into the private fiber-optic links that connected Google and Yahoo data centers around the world.

That operation, which used the cover name MUSCULAR, tapped into U.S. company data from outside U.S. territory. The NSA, therefore, believed it did not need permission from Congress or judicial oversight. Data from hundreds of millions of U.S. accounts flowed over those Google and Yahoo links, but classified rules allowed the NSA to presume that data ingested overseas belonged to foreigners.

‘Persistent threat’

Disclosure of the MUSCULAR project enraged and galvanized U.S. technology executives. They believed the NSA had lawful access to their front doors — and had broken down the back doors anyway.

Microsoft general counsel Brad Smith took to his company’s blog and called the NSA an “advanced persistent threat” — the worst of all fighting words in U.S. cybersecurity circles, generally reserved for Chinese state-sponsored hackers and sophisticated criminal enterprises.

“For the industry as a whole, it caused everyone to ask whether we knew as much as we thought,” Smith recalled in an interview. “It underscored the fact that while people were confident that the U.S. government was complying with U.S. laws for activity within U.S. territory, perhaps there were things going on outside the United States … that made this bigger and more complicated and more disconcerting than we knew.”

They wondered, he said, whether the NSA was “collecting proprietary information from the companies themselves.”

Led by Google and then Yahoo, one company after another announced expensive plans to encrypt its data traffic over tens of thousands of miles of cable. It was a direct — in some cases, explicit — blow to NSA collection of user data in bulk. If the NSA wanted the information, it would have to request it or circumvent the encryption one target at a time.

As these projects are completed, the Internet will become a less friendly place for the NSA to work. The agency can still collect data from virtually anyone, but collecting from everyone will be harder.

The industry’s response, Smith acknowledged, was driven by a business threat. U.S. companies could not afford to be seen as candy stores for U.S. intelligence. But the principle of the thing, Smith said, “is fundamentally about ensuring that customer data is turned over to governments pursuant to valid legal orders and in accordance with constitutional principles.”

‘Warheads on foreheads’

Snowden has focused on much the same point from the beginning: Individual targeting would cure most of what he believes is wrong with the NSA.

Six months ago, a reporter asked him by encrypted e-mail why Americans would want the NSA to give up bulk data collection if that would limit a useful intelligence tool.

“I believe the cost of frank public debate about the powers of our government is less than the danger posed by allowing these powers to continue growing in secret,” he replied, calling them “a direct threat to democratic governance.”

In the Moscow interview, Snowden said, “What the government wants is something they never had before,” adding: “They want total awareness. The question is, is that something we should be allowing?”

Snowden likened the NSA’s powers to those used by British authorities in Colonial America, when “general warrants” allowed for anyone to be searched. The FISA court, Snowden said, “is authorizing general warrants for the entire country’s metadata.”

“The last time that happened, we fought a war over it,” he said.

Technology, of course, has enabled a great deal of consumer surveillance by private companies, as well. The difference with the NSA’s possession of the data, Snowden said, is that government has the power to take away life or freedom.

At the NSA, he said, “there are people in the office who joke about, ‘We put warheads on foreheads.’ Twitter doesn’t put warheads on foreheads.”

Privacy, as Snowden sees it, is a universal right, applicable to American and foreign surveillance alike.

“I don’t care whether you’re the pope or Osama bin Laden,” he said. “As long as there’s an individualized, articulable, probable cause for targeting these people as legitimate foreign intelligence, that’s fine. I don’t think it’s imposing a ridiculous burden by asking for probable cause. Because, you have to understand, when you have access to the tools the NSA does, probable cause falls out of trees.”

‘Everybody knows’

On June 29, Gilles de Kerchove, the European Union’s counterterrorism coordinator, awoke to a report in Der Spiegel that U.S. intelligence had broken into E.U. offices, including his, to implant surveillance devices.

The 56-year-old Belgian, whose work is often classified, did not consider himself naive. But he took the news personally, and more so when he heard unofficial explanations from Washington.

“ ‘Everybody knows. Everybody does’ — Keith Alexander said that,” de Kerchove said in an interview. “I don’t like the idea that the NSA will put bugs in my office. No. I don’t like it. No. Between allies? No. I’m surprised that people find that noble.”

Comparable reactions, expressed less politely in private, accompanied revelations that the NSA had tapped the cellphones of German Chancellor Angela Merkel and Brazilian President Dilma Rousseff. The blowback roiled relations with both allies, among others. Rousseff canceled a state dinner with Obama in September.

When it comes to spying on allies, by Snowden’s lights, the news is not always about the target.

“It’s the deception of the government that’s revealed,” Snowden said, noting that the Obama administration offered false public assurances after the initial reports about NSA surveillance in Germany “The U.S. government said: ‘We follow German laws in Germany. We never target German citizens.’ And then the story comes out and it’s: ‘What are you talking about? You’re spying on the chancellor.’ You just lied to the entire country, in front of Congress.”

In private, U.S. intelligence officials still maintain that spying among friends is routine for all concerned, but they are giving greater weight to the risk of getting caught.

“There are many things we do in intelligence that, if revealed, would have the potential for all kinds of blowback,” Clapper told a House panel in October.

‘They will make mistakes’

U.S. officials say it is obvious that Snowden’s disclosures will do grave harm to intelligence gathering, exposing methods that adversaries will learn to avoid.

“We’re seeing al-Qaeda and related groups start to look for ways to adjust how they communicate,” said Matthew Olsen, director of the National Counterterrorism Center and a former general counsel at the NSA.

Other officials, who declined to speak on the record about particulars, said they had watched some of their surveillance targets, in effect, changing channels. That evidence can be read another way, they acknowledged, given that the NSA managed to monitor the shift.

Clapper has said repeatedly in public that the leaks did great damage, but in private he has taken a more nuanced stance. A review of early damage assessments in previous espionage cases, he said in one closed-door briefing this fall, found that dire forecasts of harm were seldom borne out.

“People must communicate,” he said, according to one participant who described the confidential meeting on the condition of anonymity. “They will make mistakes, and we will exploit them.”

According to senior intelligence officials, two uncertainties feed their greatest concerns. One is whether Russia or China managed to take the Snowden archive from his computer, a worst-case assumption for which three officials acknowledged there is no evidence.

In a previous assignment, Snowden taught U.S. intelligence personnel how to operate securely in a “high-threat digital environment,” using a training scenario in which China was the designated threat. He declined to discuss the whereabouts of the files, but he said that he is confident he did not expose them to Chinese intelligence in Hong Kong. And he said he did not bring them to Russia.

“There’s nothing on it,” he said, turning his laptop screen toward his visitor. “My hard drive is completely blank.”

The other big question is how many documents Snowden took. The NSA’s incoming deputy director, Rick Ledgett, said on CBS’s “60 Minutes” recently that the number may approach 1.7 million, a huge and unexplained spike over previous estimates. Ledgett said he would favor trying to negotiate an amnesty with Snowden in exchange for “assurances that the remainder of the data could be secured.”

Obama’s national security adviser, Susan E. Rice, later dismissed the possibility.

“The government knows where to find us if they want to have a productive conversation about resolutions that don’t involve Edward Snowden behind bars,” said the American Civil Liberties Union’s Ben Wizner, the central figure on Snowden’s legal team.

Some news accounts have quoted U.S. government officials as saying Snowden has arranged for the automated release of sensitive documents if he is arrested or harmed. There are strong reasons to doubt that, beginning with Snowden’s insistence, to this reporter and others, that he does not want the documents published in bulk.

If Snowden were fool enough to rig a “dead man’s switch,” confidants said, he would be inviting anyone who wants the documents to kill him.

Asked about such a mechanism in the Moscow interview, Snowden made a face and declined to reply. Later, he sent an encrypted message. “That sounds more like a suicide switch,” he wrote. “It wouldn’t make sense.”

‘It’s not about me’

By temperament and circumstance, Snowden is a reticent man, reluctant to discuss details about his personal life.

Over two days his guard never dropped, but he allowed a few fragments to emerge. He is an “ascetic,” he said. He lives off ramen noodles and chips. He has visitors, and many of them bring books. The books pile up, unread. The Internet is an endless library and a window on the progress of his cause.

“It has always been really difficult to get me to leave the house,” he said. “I just don’t have a lot of needs. … Occasionally there’s things to go do, things to go see, people to meet, tasks to accomplish. But it’s really got to be goal-oriented, you know. Otherwise, as long as I can sit down and think and write and talk to somebody, that’s more meaningful to me than going out and looking at landmarks.”

In hope of keeping focus on the NSA, Snowden has ignored attacks on himself.

“Let them say what they want,” he said. “It’s not about me.”

Former NSA and CIA director Michael V. Hayden predicted that Snowden will waste away in Moscow as an alcoholic, like other “defectors.” To this, Snowden shrugged. He does not drink at all. Never has.

But Snowden knows his presence here is easy ammunition for critics. He did not choose refuge in Moscow as a final destination. He said that once the U.S. government voided his passport as he tried to change planes en route to Latin America, he had no other choice.

It would be odd if Russian authorities did not keep an eye on him, but no retinue accompanied Snowden and his visitor saw no one else nearby. Snowden neither tried to communicate furtively nor asked that his visitor do so. He has had continuous Internet access and has talked to his attorneys and to journalists daily, from his first day in the transit lounge at Sheremetyevo airport.

“There is no evidence at all for the claim that I have loyalties to Russia or China or any country other than the United States,” he said. “I have no relationship with the Russian government. I have not entered into any agreements with them.”

“If I defected at all,” Snowden said, “I defected from the government to the public.”

Julie Tate contributed to this report.

By Barton Gellman and Laura Poitras

June 6, 2013

The National Security Agency and the FBI are tapping directly into the central servers of nine leading U.S. Internet companies, extracting audio and video chats, photographs, e-mails, documents, and connection logs that enable analysts to track foreign targets, according to a top-secret document obtained by The Washington Post.

The program, code-named PRISM, has not been made public until now. It may be the first of its kind. The NSA prides itself on stealing secrets and breaking codes, and it is accustomed to corporate partnerships that help it divert data traffic or sidestep barriers. But there has never been a Google or Facebook before, and it is unlikely that there are richer troves of valuable intelligence than the ones in Silicon Valley.

Equally unusual is the way the NSA extracts what it wants, according to the document: “Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.”

London’s Guardian newspaper reported Friday that GCHQ, Britain’s equivalent of the NSA, also has been secretly gathering intelligence from the same internet companies through an operation set up by the NSA.

According to documents obtained by The Guardian, PRISM would appear to allow GCHQ to circumvent the formal legal process required in Britain to seek personal material such as emails, photos and videos from an internet company based outside of the country.

PRISM was launched from the ashes of President George W. Bush’s secret program of warrantless domestic surveillance in 2007, after news media disclosures, lawsuits and the Foreign Intelligence Surveillance Court forced the president to look for new authority.

Congress obliged with the Protect America Act in 2007 and the FISA Amendments Act of 2008, which immunized private companies that cooperated voluntarily with U.S. intelligence collection. PRISM recruited its first partner, Microsoft, and began six years of rapidly growing data collection beneath the surface of a roiling national debate on surveillance and privacy. Late last year, when critics in Congress sought changes in the FISA Amendments Act, the only lawmakers who knew about PRISM were bound by oaths of office to hold their tongues.

The court-approved program is focused on foreign communications traffic, which often flows through U.S. servers even when sent from one overseas location to another. Between 2004 and 2007, Bush administration lawyers persuaded federal FISA judges to issue surveillance orders in a fundamentally new form. Until then the government had to show probable cause that a particular “target” and “facility” were both connected to terrorism or espionage.

In four new orders, which remain classified, the court defined massive data sets as “facilities” and agreed to certify periodically that the government had reasonable procedures in place to minimize collection of “U.S. persons” data without a warrant.

In a statement issue late Thursday, Director of National Intelligence James R. Clapper said “information collected under this program is among the most important and valuable foreign intelligence information we collect, and is used to protect our nation from a wide variety of threats. The unauthorized disclosure of information about this important and entirely legal program is reprehensible and risks important protections for the security of Americans.”

Clapper added that there were numerous inaccuracies in reports about PRISM by The Post and the Guardian newspaper, but he did not specify any.

Jameel Jaffer, deputy legal director of the American Civil Liberties Union, said: “I would just push back on the idea that the court has signed off on it, so why worry? This is a court that meets in secret, allows only the government to appear before it, and publishes almost none of its opinions. It has never been an effective check on government.”

Several companies contacted by The Post said they had no knowledge of the program, did not allow direct government access to their servers and asserted that they responded only to targeted requests for information.

“We do not provide any government organization with direct access to Facebook servers,” said Joe Sullivan, chief security officer for Facebook. “When Facebook is asked for data or information about specific individuals, we carefully scrutinize any such request for compliance with all applicable laws, and provide information only to the extent required by law.”

“We have never heard of PRISM,” said Steve Dowling, a spokesman for Apple. “We do not provide any government agency with direct access to our servers, and any government agency requesting customer data must get a court order.”

It is possible that the conflict between the PRISM slides and the company spokesmen is the result of imprecision on the part of the NSA author. In another classified report obtained by The Post, the arrangement is described as allowing “collection managers [to send] content tasking instructions directly to equipment installed at company-controlled locations,” rather than directly to company servers.

Government officials and the document itself made clear that the NSA regarded the identities of its private partners as PRISM’s most sensitive secret, fearing that the companies would withdraw from the program if exposed. “98 percent of PRISM production is based on Yahoo, Google and Microsoft; we need to make sure we don’t harm these sources,” the briefing’s author wrote in his speaker’s notes.

An internal presentation of 41 briefing slides on PRISM, dated April 2013 and intended for senior analysts in the NSA’s Signals Intelligence Directorate, described the new tool as the most prolific contributor to the President’s Daily Brief, which cited PRISM data in 1,477 items last year. According to the slides and other supporting materials obtained by The Post, “NSA reporting increasingly relies on PRISM” as its leading source of raw material, accounting for nearly 1 in 7 intelligence reports.

That is a remarkable figure in an agency that measures annual intake in the trillions of communications. It is all the more striking because the NSA, whose lawful mission is foreign intelligence, is reaching deep inside the machinery of American companies that host hundreds of millions of American-held accounts on American soil.

The technology companies, whose cooperation is essential to PRISM operations, include most of the dominant global players of Silicon Valley, according to the document. They are listed on a roster that bears their logos in order of entry into the program: “Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” PalTalk, although much smaller, has hosted traffic of substantial intelligence interest during the Arab Spring and in the ongoing Syrian civil war.

Dropbox, the cloud storage and synchronization service, is described as “coming soon.”

Sens. Ron Wyden (D-Ore.) and Mark Udall (D-Colo.), who had classified knowledge of the program as members of the Senate Intelligence Committee, were unable to speak of it when they warned in a Dec. 27, 2012, floor debate that the FISA Amendments Act had what both of them called a “back-door search loophole” for the content of innocent Americans who were swept up in a search for someone else.

“As it is written, there is nothing to prohibit the intelligence community from searching through a pile of communications, which may have been incidentally or accidentally been collected without a warrant, to deliberately search for the phone calls or e-mails of specific Americans,” Udall said.

Wyden repeatedly asked the NSA to estimate the number of Americans whose communications had been incidentally collected, and the agency’s director, Lt. Gen. Keith B. Alexander, insisted there was no way to find out. Eventually Inspector General I. Charles McCullough III wrote Wyden a letter stating that it would violate the privacy of Americans in NSA data banks to try to estimate their number.

Roots in the ’70s

PRISM is an heir, in one sense, to a history of intelligence alliances with as many as 100 trusted U.S. companies since the 1970s. The NSA calls these Special Source Operations, and PRISM falls under that rubric.

The Silicon Valley operation works alongside a parallel program, code-named BLARNEY, that gathers up “metadata” — technical information about communications traffic and network devices — as it streams past choke points along the backbone of the Internet. BLARNEY’s top-secret program summary, set down in the slides alongside a cartoon insignia of a shamrock and a leprechaun hat, describes it as “an ongoing collection program that leverages IC [intelligence community] and commercial partnerships to gain access and exploit foreign intelligence obtained from global networks.”

But the PRISM program appears to more nearly resemble the most controversial of the warrantless surveillance orders issued by President George W. Bush after the al-Qaeda attacks of Sept. 11, 2001. Its history, in which President Obama presided over exponential growth in a program that candidate Obama criticized, shows how fundamentally surveillance law and practice have shifted away from individual suspicion in favor of systematic, mass collection techniques.

The Obama administration points to ongoing safeguards in the form of “extensive procedures, specifically approved by the court, to ensure that only non-U.S. persons outside the U.S. are targeted, and that minimize the acquisition, retention and dissemination of incidentally acquired information about U.S. persons.”

And it is true that the PRISM program is not a dragnet, exactly. From inside a company’s data stream the NSA is capable of pulling out anything it likes, but under current rules the agency does not try to collect it all.

Analysts who use the system from a Web portal at Fort Meade, Md., key in “selectors,” or search terms, that are designed to produce at least 51 percent confidence in a target’s “foreignness.” That is not a very stringent test. Training materials obtained by The Post instruct new analysts to make quarterly reports of any accidental collection of U.S. content, but add that “it’s nothing to worry about.”

Even when the system works just as advertised, with no American singled out for targeting, the NSA routinely collects a great deal of American content. That is described as “incidental,” and it is inherent in contact chaining, one of the basic tools of the trade. To collect on a suspected spy or foreign terrorist means, at minimum, that everyone in the suspect’s inbox or outbox is swept in. Intelligence analysts are typically taught to chain through contacts two “hops” out from their target, which increases “incidental collection” exponentially. The same math explains the aphorism, from the John Guare play, that no one is more than “six degrees of separation” from any other person.

A ‘directive’

In exchange for immunity from lawsuits, companies such as Yahoo and AOL are obliged to accept a “directive” from the attorney general and the director of national intelligence to open their servers to the FBI’s Data Intercept Technology Unit, which handles liaison to U.S. companies from the NSA. In 2008, Congress gave the Justice Department authority for a secret order from the Foreign Surveillance Intelligence Court to compel a reluctant company “to comply.”

In practice, there is room for a company to maneuver, delay or resist. When a clandestine intelligence program meets a highly regulated industry, said a lawyer with experience in bridging the gaps, neither side wants to risk a public fight. The engineering problems are so immense, in systems of such complexity and frequent change, that the FBI and NSA would be hard pressed to build in back doors without active help from each company.

Apple demonstrated that resistance is possible when it held out for more than five years, for reasons unknown, after Microsoft became PRISM’s first corporate partner in May 2007. Twitter, which has cultivated a reputation for aggressive defense of its users’ privacy, is still conspicuous by its absence from the list of “private sector partners.”

Google, like the other companies, denied that it permitted direct government access to its servers.

“Google cares deeply about the security of our users’ data,” a company spokesman said. “We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government ‘back door’ into our systems, but Google does not have a ‘back door’ for the government to access private user data.”

Microsoft also provided a statement: “We provide customer data only when we receive a legally binding order or subpoena to do so, and never on a voluntary basis. In addition we only ever comply with orders for requests about specific accounts or identifiers. If the government has a broader voluntary national security program to gather customer data we don’t participate in it.”

Yahoo also issued a denial.

“Yahoo! takes users’ privacy very seriously,” the company said in a statement. “We do not provide the government with direct access to our servers, systems, or network.”

Like market researchers, but with far more privileged access, collection managers in the NSA’s Special Source Operations group, which oversees the PRISM program, are drawn to the wealth of information about their subjects in online accounts. For much the same reason, civil libertarians and some ordinary users may be troubled by the menu available to analysts who hold the required clearances to “task” the PRISM system.

There has been “continued exponential growth in tasking to Facebook and Skype,” according to the PRISM slides. With a few clicks and an affirmation that the subject is believed to be engaged in terrorism, espionage or nuclear proliferation, an analyst obtains full access to Facebook’s “extensive search and surveillance capabilities against the variety of online social networking services.”

According to a separate “User’s Guide for PRISM Skype Collection,” that service can be monitored for audio when one end of the call is a conventional telephone and for any combination of “audio, video, chat, and file transfers” when Skype users connect by computer alone. Google’s offerings include Gmail, voice and video chat, Google Drive files, photo libraries, and live surveillance of search terms.

Firsthand experience with these systems, and horror at their capabilities, is what drove a career intelligence officer to provide PowerPoint slides about PRISM and supporting materials to The Washington Post in order to expose what he believes to be a gross intrusion on privacy. “They quite literally can watch your ideas form as you type,” the officer said.

Poitras is a documentary filmmaker and MacArthur Fellow. Julie Tate, Robert O’Harrow Jr., Cecilia Kang and Ellen Nakashima contributed to this report.

Additional Content:

The NSA problem? Too much data.

NSA leak: Source believes exposure, consequences inevitable (VIDEO)

An excerpt from NSA’s Wikipedia

By Ellen Nakashima

June 9, 2013

Edward Snowden said the internet “is a TV watching you.” (Photo courtesy of The Guardian)

Edward Snowden, the 29-year-old National Security Agency contractor who admitted that he was behind recent leaks of classified intelligence, has vaulted from obscurity to international notoriety, joining the ranks of high-profile leakers such as Daniel Ellsberg of Pentagon Papers fame.

The fact that Snowden stepped forward to acknowledge his leaks to The Washington Post and the Guardian newspapers rather than wait for the FBI to find him impressed others who have disclosed government secrets.

“I consider it a magnificent act of civil disobedience,” said Thomas Drake, a former NSA official who was prosecuted for leaking classified information to a journalist but wound up serving no prison time after the government’s case fell apart. “He’s a whistleblower.”

Ellsberg was similarly impressed. He said in an interview: “There’s no American official or former official that I admire more at this point. There’s never been a more important disclosure to the American people than the leak [by Snowden] — and I include the Pentagon Papers in that. . . . He’s clearly ready to give his life or his freedom for the interests of his country.”

Others view Snowden’s leaks of NSA and other documents to the The Post and the Guardian as a breach of his oath to protect classified information that may have harmed national security.

“The government is not going to hold back on this case,” said Michael Vatis, a former associate deputy attorney general for national security in the Clinton administration. “This is a huge one.”

Vatis, now a partner at Steptoe & Johnson in Washington, asserted that Snowden’s breach was not comparable to Ellsberg’s because Ellsberg revealed how the government was lying about the prospects for success and the scale of casualties in the Vietnam War.

Snowden’s leak, Vatis said, revealed details of electronic surveillance programs operating within the law, authorized by Congress and overseen by a federal court. One program related to NSA’s gathering of records of customers of U.S. telephone companies for counterterrorism purposes. Another involves NSA’s collection of the communications of foreigners through U.S. Internet companies.

Snowden said he was motivated by his belief that the government, particularly the ultra-secret NSA, had built an enterprise “that allows it to intercept almost everything” in a way that violates privacy rights.

His revelations of the inner workings of the two programs have reignited a debate over whether the government’s secret surveillance programs are sufficiently protective of civil liberties and privacy in the name of protecting the country against terrorism.

Since the attacks of Sept. 11, 2001, the NSA has been at the forefront in collecting electronic information worldwide as part of the nation’s counterterrorism efforts. The material exposed by Snowden provided an unprecedented look at the scope of the surveillance carried out by the agency. President Obama defended the surveillance as essential to U.S. security and said he welcomes an extended debate.

“As far as I’m able to judge, the public policy value [of his leaks] far exceeds the potential risk involved to national security,” said Steven Aftergood, director of the Federation of American Scientists’ Project on Government Secrecy. “It has triggered an intense public debate. These stories have really enriched public discourse.”

Aftergood said Snowden’s leaks were targeted and selective, unlike those of Army Pfc. Bradley E. Manning, the former Army intelligence analyst on trial for orchestrating the leak of more than 700,000 government documents to the antisecrecy WikiLeaks Web site.

“There were lots of records put out by WikiLeaks that had no particular significance other than the fact that they are restricted,” Aftergood said. “It was an anti-secrecy campaign. This [Snowden’s action] seemed to aim at a much larger target — namely, the scope of intelligence surveillance activity and government overreaching and violation of policy.”

Snowden is the first leaker to step forward and publicly admit his act before he was even named by the government. “There’s a certain dignity in that,” Aftergood said. “Among leakers, as among others who defy accepted norms, that is the exception, not the rule.”

Ellsberg surrendered to authorities in 1971 after it became known that an arrest warrant had been issued for him. He said he wanted to take responsibility for his actions. But unlike Snowden, he said, it did not occur to him to flee - in Snowden’s case, to Hong Kong. “But his being in Hong Kong gives him the chance to speak out openly in a way that he could not do here, to explain his motives,” Ellsberg said.

U.S. officials and national security experts, however, say the leaker should be prosecuted.

“Taking a very sensitive classified program that targets foreign persons on foreign lands, and putting just enough out there to be dangerous, is dangerous to us. It’s dangerous to our national security. And it violates the oath which that person took,” Rep. Mike Rogers (R-Mich.), chairman of the House intelligence committee, said in a television interview Sunday. Dianne Feinstein (D-Calif.), his Senate counterpart, agreed that the leaks demanded prosecution.

The most likely statute under which Snowden would be prosecuted is the 1917 Espionage Act, a law used to prosecute Manning, Drake and Ellsberg. Provisions under that law make it a crime to disclose national defense information to persons unauthorized to receive it and to transmit classified data on communication intelligence.

Additional Content:

A timeline of Edward Snowden’s life

Tracking Edward Snowden, from a Maryland classroom to a Hong Kong hotel

By Greg Miller

June 11, 2013

In the span of three years, the United States has developed two gaping holes in its national security hull, punctures caused by leakers who worked at the lowest levels of the nation’s intelligence ranks but gained access to large caches of classified material.

The parallels between Edward Snowden, who has declared himself the source of leaks on National Security Agency surveillance programs, and Bradley Manning, a U.S. Army private on trial for sending hundreds of thousands of secret files to the WikiLeaks Web site, go beyond generational ties.

Both enlisted in the Army during the war in Iraq only to later say they were disillusioned by that conflict. Neither has a college degree or extensive academic training in computer science. And yet both were technically savvy, able to navigate sensitive computer networks and smuggle classified files.

The back-to-back breaches — seen by many as the most significant in decades — have forced U.S. intelligence officials to examine whether the cases are isolated in scope or part of a new category of exposure emerging at the edges of classified U.S. networks.

U.S. officials said counterintelligence teams are already looking beyond the details of the Snowden case at any vulnerabilities it has exposed.

“At this point, we’re still looking at this as an anomaly,” a senior U.S. intelligence official said. But he said a damage assessment ordered by Director of National Intelligence James R. Clapper Jr. is broad in scope, focused largely on the implications of a case in which highly sensitive materials were somehow within the grasp of a contractor who moved through a series of low-ranking jobs for the CIA and the NSA.

Snowden and Manning both took advantage of access to computer systems that expanded exponentially in the aftermath of the attacks of Sept. 11, 2001, partly in an effort to make critical information available across agencies.

Since the disclosures by Wiki Leaks in 2010, the Pentagon has taken steps to better protect its classified networks. It has banned the use of thumb drives unless special permission is given, mandated that users have special smart cards that authenticate their identities and required analysts to review computer logs to identify suspicious behavior on the network.

Despite such measures, it remains impossible to guarantee security on any classified network. Since going public, Snow den has attributed his extensive access to his role as an ordinary systems administrator for the NSA, although he offered few specific details about where he found or how he removed such sensitive files.

In an e-mail to a Washington Post reporter last month, he said he wanted “to embolden others to step forward,” suggesting that he hoped his leak would trigger follow-up disclosures.

Among the files he obtained were a secret order issued by the Foreign Intelligence Surveillance Court and 41 briefing slides on a highly classified program, called PRISM, in which mainstream technology companies including Microsoft, Google and Facebook have given the NSA extensive access to e-mails, videos and other content.

The records released by Snow den are fewer in number but more sensitive and of higher levels of classification than the U.S. diplomatic cables and military reports Manning sent to WikiLeaks after he downloaded them while serving in Iraq.

Snowden has praised Manning but also sought to differentiate himself from his predecessor. In particular, Snowden has indicated that he sought to be more responsible, withholding records he had that might put U.S. intelligence operatives in jeopardy, unlike Manning, who is accused of turning over thousands of pages, some of which contained the names of informants.

But their motivations and backgrounds have extensive overlaps. Both entered the military or government jobs during an era of massive hiring binges, controversial wars and repeated disclosures of alleged abuses by the U.S. military, the CIA and the NSA.

Both have said they wanted to call attention to abuses by the U.S. government, although critics have called them traitors who acted out of an exaggerated sense of self-importance.

Manning was gifted with computers and had begun learning the C++ programming language by the time he was 8. He never found a path to college and instead enlisted in the Army, where he was groomed for a career as an intelligence analyst.

Snowden’s plans to pursue a Special Forces career were derailed, he said, by leg injuries he sustained during training that ended his military stint after just three months.

It is less clear how Snowden came by his computer skills. But, as with Manning, his technology savvy appears to have helped him obtain a string of jobs that enabled him to snoop on and eventually steal sensitive files.

Their actions and motivations bear little resemblance to high-profile security breaches of the 1980s and ‘90s. The best-known cases involved mid-career officials such as the CIA’s Aldrich Ames and the FBI’s Robert Hanssen, who were convicted of selling secrets to the Soviet and Russian governments for financial gain.

The closest parallel for Snow den and Manning may be Daniel Ellsberg, who in 1971 was the New York Times’ and The Washington Post’s source for the Pentagon Papers, a secret assessment of the Vietnam War that eroded the credibility of U.S. government’s more-optimistic public claims about the conflict. Ellsberg in recent days has praised Snowden and described the material Snowden disclosed as more significant than the documents he leaked four decades ago.

There are differences, however. Ellsberg was a senior military analyst working at the Pentagon who had a direct role in drafting the Pentagon Papers. The document was largely a record of U.S. decision-making rather than a blueprint of ongoing operations. Drafts were undoubtedly stored in safes, not on networks where they might be read by low-level employees at distant military or intelligence outposts.

A former senior NSA official recalled procedures in the 1970s that were archaic but secure. “When hot documents would go around they’d be in a double-sealed envelope, and some person would wait while you read it, re-envelope it and leave,” the former official said. “By contrast, now you bring it up on your computer screen.”

Some U.S. officials question whether there is a generational gap in views on privacy and government transparency. Manning and Snowden, who are in their 20s, grew up with technology and the Internet as fixtures in their lives.

“We are recruiting Americans from a culture that has a deeper desire for absolute transparency than any previous cohort of people entering the service,” said Michael V. Hayden, former CIA and NSA director. “They are coming from a culture in which, for many, transparency is an absolute good, and it appears that in these two cases it influenced these people.”

Snowden appears to have left fewer online footprints than many of his generation, with no evidence of Facebook or Twitter accounts. In his comments to The Post, he indicated that was in part because of what he had learned.

The Internet “is a TV that watches you,” he said, a technology “governments are abusing … to extend their powers beyond what is necessary and appropriate.”

Ellen Nakashima contributed to this report.

By Barton Gellman

June 15, 2013

On March 12, 2004, acting attorney general James B. Comey and the Justice Department’s top leadership reached the brink of resignation over electronic surveillance orders that they believed to be illegal.

President George W. Bush backed down, halting secret foreign-intelligence-gathering operations that had crossed into domestic terrain. That morning marked the beginning of the end of STELLARWIND, the cover name for a set of four surveillance programs that brought Americans and American territory within the domain of the National Security Agency for the first time in decades. It was also a prelude to new legal structures that allowed Bush and then President Obama to reproduce each of those programs and expand their reach.

What exactly STELLARWIND did has never been disclosed in an unclassified form. Which parts of it did Comey approve? Which did he shut down? What became of the programs when the crisis passed and Comey, now Obama’s expected nominee for FBI director, returned to private life?

Authoritative new answers to those questions, drawing upon a classified NSA history of STELLARWIND and interviews with high-ranking intelligence officials, offer the clearest map yet of the Bush-era programs and the NSA’s contemporary U.S. operations.

STELLARWIND was succeeded by four major lines of intelligence collection in the territorial United States, together capable of spanning the full range of modern telecommunications, according to the interviews and documents.

Foreigners, not Americans, are the NSA’s “targets,” as the law defines that term. But the programs are structured broadly enough that they touch nearly every American household in some way. Obama administration officials and career intelligence officers say Americans should take comfort that privacy protections are built into the design and oversight, but they are not prepared to discuss the details.

The White House, the NSA and the Office of the Director of National Intelligence declined to comment on the record for this article. A senior intelligence official agreed to answer questions if not identified.

“We have rich oversight across three branches of government. I’ve got an [inspector general] here, a fairly robust legal staff here . . . and there’s the Justice Department’s national security division,” the official said. “For those things done under court jurisdiction, the courts are intrusive in my business, appropriately so, and there are two congressional committees. It’s a belts-and-suspenders-and-Velcro approach, and inside there’s rich auditing.”

But privacy advocates, such as Sen. Ron Wyden (D-Ore.), said the intelligence committee on which he serves needs “straight answers” to do vigorous oversight.

He added: “The typical person says, ‘If I am law-abiding and the government is out there collecting lots of information about me — who I call, when I call, where I call from’ . . . I think the typical person is going to say, ‘That sure sounds like it could have some effect on my privacy.’ ”

Two of the four collection programs, one each for telephony and the Internet, process trillions of “metadata” records for storage and analysis in systems called MAINWAY and MARINA, respectively. Metadata includes highly revealing information about the times, places, devices and participants in electronic communication, but not its contents. The bulk collection of telephone call records from Verizon Business Services, disclosed this month by the British newspaper the Guardian, is one source of raw intelligence for MAINWAY.

The other two types of collection, which operate on a much smaller scale, are aimed at content. One of them intercepts telephone calls and routes the spoken words to a system called NUCLEON.

For Internet content, the most important source collection is the PRISM project reported on June 6 by The Washington Post and the Guardian. It draws from data held by Google, Yahoo, Microsoft and other Silicon Valley giants, collectively the richest depositories of personal information in history.

Former NSA contractor Edward Snowden, 29, who unmasked himself as the source behind the PRISM and Verizon revelations, said he hoped for a systematic debate about the “danger to our freedom and way of life” posed by a surveillance apparatus “kept in check by nothing more than policy.”

For well over a week, he has had his wish. Startling disclosures have poured out of the nation’s largest and arguably tightest-lipped spy agency at an unprecedented pace. Snowden’s disclosures have opened a national conversation about the limits of secret surveillance in a free society and anoutcry overseas against U.S. espionage.

The debate has focused on two of the four U.S.-based collection programs: PRISM, for Internet content, and the comprehensive collection of telephone call records, foreign and domestic, that the Guardian revealed by posting a classified order from the Foreign Intelligence Surveillance Court to Verizon Business Services.

The Post has learned that similar orders have been renewed every three months for other large U.S. phone companies, including Bell South and AT&T, since May 24, 2006. On that day, the surveillance court made a fundamental shift in its approach to Section 215 of the Patriot Act, which permits the FBI to compel production of “business records” that are relevant to a particular terrorism investigation and to share those in some circumstances with the NSA. Henceforth, the court ruled, it would define the relevant business records as the entirety of a telephone company’s call database.

The Bush administration, by then, had been taking “bulk metadata” from the phone companies under voluntary agreements for more than four years. The volume of information overwhelmed the MAINWAY database, according to a classified report from the NSA inspector general in 2009. The agency spent $146 million in supplemental counterterrorism funds to buy new hardware and contract support — and to make unspecified payments to the phone companies for “collaborative partnerships.”

When the New York Times revealed the warrantless surveillance of voice calls, in December 2005, the telephone companies got nervous. One of them, unnamed in the report, approached the NSA with a request. Rather than volunteer the data, at a price, the “provider preferred to be compelled to do so by a court order,” the report said. Other companies followed suit. The surveillance court order that recast the meaning of business records “essentially gave NSA the same authority to collect bulk telephony metadata from business records that it had” under Bush’s asserted authority alone.

Telephone metadata was not the issue that sparked a rebellion at the Justice Department, first by Jack Goldsmith of the Office of Legal Counsel and then by Comey, who was acting attorney general because John D. Ashcroft was in intensive care with acute gallstone pancreatitis. It was Internet metadata.

At Bush’s direction, in orders prepared by David Addington, the counsel to Vice President Richard B. Cheney, the NSA had been siphoning e-mail metadata and technical records of Skype calls from data links owned by AT&T, Sprint and MCI, which later merged with Verizon.

For reasons unspecified in the report, Goldsmith and Comey became convinced that Bush had no lawful authority to do that.

MARINA and the collection tools that feed it are probably the least known of the NSA’s domestic operations, even among experts who follow the subject closely. Yet they probably capture information about more American citizens than any other, because the volume of e-mail, chats and other Internet communications far exceeds the volume of standard telephone calls.

The NSA calls Internet metadata “digital network information.” Sophisticated analysis of those records can reveal unknown associates of known terrorism suspects. Depending on the methods applied, it can also expose medical conditions, political or religious affiliations, confidential business negotiations and extramarital affairs.

What permits the former and prevents the latter is a complex set of policies that the public is not permitted to see. “You could do analyses that give you more information, but the law and procedures don’t allow that,” a senior U.S. intelligence lawyer said.

In the urgent aftermath of Sept. 11, 2001, with more attacks thought to be imminent, analysts wanted to use “contact chaining” techniques to build what the NSA describes as network graphs of people who represented potential threats.

The legal challenge for the NSA was that its practice of collecting high volumes of data from digital links did not seem to meet even the relatively low requirements of Bush’s authorization, which allowed collection of Internet metadata “for communications with at least one communicant outside the United States or for which no communicant was known to be a citizen of the United States,” the NSA inspector general’s report said.

Lawyers for the agency came up with an interpretation that said the NSA did not “acquire” the communications, a term with formal meaning in surveillance law, until analysts ran searches against it. The NSA could “obtain” metadata in bulk, they argued, without meeting the required standards for acquisition.

Goldsmith and Comey did not buy that argument, and a high-ranking U.S. intelligence official said the NSA does not rely on it today.

As soon as surveillance data “touches us, we’ve got it, whatever verbs you choose to use,” the official said in an interview. “We’re not saying there’s a magic formula that lets us have it without having it.”

When Comey finally ordered a stop to the program, Bush signed an order renewing it anyway. Comey, Goldsmith, FBI Director Robert S. Mueller III and most of the senior Bush appointees in the Justice Department began drafting letters of resignation.

Then-NSA Director Michael V. Hayden was not among them. According to the inspector general’s classified report, Cheney’s lawyer, Addington, placed a phone call and “General Hayden had to decide whether NSA would execute the Authorization without the Attorney General’s signature.” He decided to go along.

The following morning, when Mueller told Bush that he and Comey intended to resign, the president reversed himself.

Three months later, on July 15, the secret surveillance court allowed the NSA to resume bulk collection under the court’s own authority. The opinion, which remains highly classified, was based on a provision of electronic surveillance law, known as “pen register, trap and trace,” that was written to allow law enforcement officers to obtain the phone numbers of incoming and outgoing calls from a single telephone line.

When the NSA aims for foreign targets whose communications cross U.S. infrastructure, it expects to sweep in some American content “incidentally” or “inadvertently,” which are terms of art in regulations governing the NSA. Contact chaining, because it extends to the contacts of contacts of targets, inevitably collects even more American data.

Current NSA director Keith B. Alexanderand Director of National Intelligence James R. Clapper Jr. have resolutely refused to offer an estimate of the number of Americans whose calls or e-mails have thus made their way into content databases such as NUCLEON.

The agency and its advocates maintain that its protection of that data is subject to rigorous controls and oversight by Congress and courts. For the public, it comes down to a question of unverifiable trust.

“The constraints that I operate under are much more remarkable than the powers that I enjoy,” said the senior intelligence official who declined to be named.

When asked why the NSA could not release an unclassified copy of its “minimization procedures,” which are supposed to strip accidentally collected records of their identifying details, the official suggested a reporter submit a freedom-of-information request.

As for bulk collection of Internet metadata, the question that triggered the crisis of 2004, another official said the NSA is no longer doing it. When pressed on that question, he said he was speaking only of collections under authority of the surveillance court.

“I’m not going to say we’re not collecting any Internet metadata,” he added. “We’re not using this program and these kinds of accesses to collect Internet metadata in bulk.”

Julie Tate and Ellen Nakashima contributed to this report.

Additional Content:

Electronic surveillance under Presidents Bush and Obama: A timeline of surveillance in the United States from 2001 to 2013

By Ellen Nakashima

June 15, 2013

Upstream program. This slide shows PRISM as only one part of the NSA’s system for electronic eavesdropping. The “Upstream” program collects from the fiber-optic cable networks that carry much of the world’s Internet and phone data. The underlying map depicts the undersea cables that connect North America to the rest of the world.

The general’s mistress thought she was being clever by using anonymous e-mail accounts and sending messages using hotel WiFi networks. But metadata — in this case the Internet protocol addresses pointing to network locations — gave her away.

The IP addresses of the networks Paula Broadwell logged into this past fall to send threatening messages to a woman she perceived as a rival for the affection of Gen. David H. Petraeus traced back to the hotels. There, records corresponding to the dates the e-mails were sent revealed one common guest: Broadwell.

Petraeus resigned as CIA director over the affair, and the episode has since receded from the public’s attention. But it is instructive as one simple but powerful way in which metadata — or data about communications — can reveal so much about who we are, where we go and whom we associate with.

Metadata is so rich with clues that entities from Google and eBay to the world’s largest spy agency, the National Security Agency, are collecting and mining this deceptively innocuous information: e-mail addresses to and from, times of e-mails, phone numbers dialed and received, lengths of calls, unique device serial numbers.

A week and a half ago, U.S. officials acknowledged for the first time that the NSA since 2006 has been amassing a database of metadata on the phone-call records of tens of millions of U.S. customers.

And, according to new documents obtained by The Washington Post, the NSA until 2011 gathered e-mail and other digital metadata from major Internet data links, presumably to detect and thwart terrorist plots.

But the government has resisted explaining its legal justification for gathering such massive amounts of data, which hold the potential to permit vast intrusions into the personal lives of Americans.

“When you can get it all in one place and analyze the patterns, you can learn an enormous amount about the behavior of people,” said Daniel J. Weitzner, a principal research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory.

Analysts can gain clues to sleep patterns (when people are asleep, they send no e-mails and make no calls), religion (based on locations of calls made or the absence of communications on the Sabbath) or even social position (based on how often people get calls and e-mails and how quickly they receive responses).

In 2007, researchers at Columbia University were able to identify the senior-most company officers at the bankrupt Enron Corp. by studying individual e-mail volume and average response time in 620,000 company e-mails. The highest-ranking officers got the most e-mail and the quickest responses.

Similarly, federal agents use software and social-network analysis to map out terrorist cells and criminal groups. They look, for instance, at who calls whom most frequently, in a technique known as “link analysis.”

“It’s remarkable how just the phone-call data can give you at least a preliminary picture of how the organization operates and who its members are,” said Jason Weinstein, a former deputy assistant attorney general for the Justice Department’s criminal division. “It’s by no means the whole picture, but it’s a critical piece of the puzzle to solve the most serious crimes people can commit.”

Sometimes, metadata patterns can be tip-offs — a driver or courier in a terrorist cell or criminal group may be the one to receive short phone calls from several different operatives just before and after an operation.

Cellular-tower location data can help place criminals at the scene if they are using their phones just before they commit a robbery, murder or attack.

“Every day, law enforcement officers are using this data to place suspects at the scene of murders and other crimes,” said Weinstein, now a partner at Steptoe & Johnson.

Data about a communication may be just as revealing as the content itself, said Christopher Soghoian, principal technologist with the American Civil Liberties Union.

“If you call an abortion clinic and make an appointment, the fact that you’re making the appointment is far more sensitive than what time your appointment is,” he said. “If you’re calling Alcoholics Anonymous or a suicide counselor, what you’re saying will certainly be sensitive. But the fact that you’re calling Al Anon or a suicide counselor is extremely sensitive, too.”

Under U.S. law, it’s easier for the government to obtain metadata than content. Authorities generally need to show probable cause for a wiretap or intercept of communications.

Telephone records, but not e-mail metadata, can be obtained by law enforcement agencies without any kind of court order.

Weitzner said metadata is “arguably more revealing because it’s actually much easier to analyze the patterns in a large universe of metadata and correlate them with real-world events than it is to go through a semantic analysis of all of someone’s e-mail and all of someone’s telephone calls, if you could get that.

“Metadata is objective: I called you. You called me.”

Cellphone data helped Italian authorities identify CIA agents who abducted an Egyptian cleric suspected of terrorist involvement in Milan in 2003. The investigators pulled the records and identified the agents by their aliases, where they had stayed and whom they had called — including each other. Similarly, in 2011, Hezbollah identified a half-dozen CIA informants through analysis of their cellphone records and calling patterns.

Critical as metadata is, Weinstein said, it does not give you the subject’s words and thoughts. “Only the content,” he said, “will provide you with the evidence you need that the conversations are about terrorism or other crimes.”

By Ellen Nakashima, Barton Gellman and Greg Miller

June 20, 2013

President Obama said after the disclosures that NSA domestic activities “do not involve listening to people’s phone calls, do not involve reading the emails of U.S. citizens.” (Marvin Joseph/The Washington Post)

The National Security Agency may keep the e-mails and telephone calls of citizens and legal residents if the communications contain “significant foreign intelligence” or evidence of a crime, according to classified documents that lay out procedures for targeting foreigners and for guarding Americans’ privacy.

Newly disclosed documents describe a series of steps the world’s largest spy agency is supposed to take to keep Americans from being caught in its massive surveillance net. They suggest that the NSA has latitude to keep and use citizens’ communications under certain conditions.

The papers, made available to The Washington Post and Britain’s Guardian newspaper, are the first public written documentation of procedures governing a far-reaching NSA surveillance program authorized by Congress in 2008 to gather the e-mails and phone calls of targets who are supposed to be foreigners located overseas.

In recent days, the Obama administration has defended the program as critical to national security, saying it has helped foil more than 50 terrorist plots in the United States and abroad.

President Obama said after the disclosures that NSA domestic activities “do not involve listening to people’s phone calls, do not involve reading the e-mails of U.S. citizens or U.S. residents, absent further action by a federal court, that is entirely consistent with what we would do, for example, in a criminal investigation.”

The new documents show that the NSA collects, processes, retains and disseminates the contents of Americans’ phone calls and e-mails under a wide range of circumstances.

NSA Director Keith B. Alexander described the program as “limited, focused and subject to rigorous oversight.” Testifying before Congress, he said “the disciplined operation” of this and a related surveillance program “protects the privacy and civil liberties of the American people.”

A spokesman for the Office of the Director of National Intelligence declined to comment on the documents Thursday.

Privacy advocates expressed concern about what they viewed as rules that leave much wiggle room for NSA analysts to monitor Americans’ communications.

“These documents confirm what we have feared all along, that the NSA believes it can collect Americans’ international communications with little, if any, restriction,” said Alex Abdo, a staff lawyer with the American Civil Liberties Union. “Its procedures allow it to target for surveillance essentially any foreigner located abroad — whether or not they’re suspected of any wrongdoing, let alone terrorism.”

Administration officials say the surveillance program does not target Americans anywhere without a warrant. Still, said Gregory Nojeim, senior counsel for the Center for Democracy and Technology, “there’s a lot of leeway to use ‘inadvertently’ acquired domestic communications,” for instance, for criminal inquiries.

And the rules show that the communications of lawyers and their clients may be retained if they contain foreign intelligence information, although dissemination must be approved by the NSA general counsel.

Congress authorized the collection program amid a great debate about the degree to which the government was expanding its surveillance authority without sufficient protection for Americans’ privacy.

Authorized by Section 702 of the amended Foreign Intelligence Surveillance Act (FISA), the program did away with the traditional individual warrant for each foreign suspect whose communications would be collected in the United States. In its place, the FISA court, which oversees domestic surveillance for foreign intelligence purposes and whose proceedings are secret, would certify the government’s procedures to target people overseas and ensure citizens’ privacy.

It issues a certificate, good for one year, that allows the NSA to order a U.S. Internet or phone company to turn over over e-mails, phone calls and other communications related to a series of foreign targets, none of which the court approved individually.

“What’s most striking about the targeting procedures is the discretion they confer on the NSA,” said Elizabeth Goitein, co-director of the Brennan Center for Justice’s Liberty and National Security program.

In figuring out whether a target is “reasonably believed” to be located overseas, for example, the agency looks at the “totality of the circumstances” relating to a person’s location. In the absence of that specific information, “a person reasonably believed to be located outside the United States or whose location is not known will be presumed to be a non-United States person,” according to rules on the targeting of suspects.

Nonetheless, the documents contain a series of steps the NSA may take to determine a foreigner’s location. Agency analysts examine leads that may come from other agencies, including from human sources. They conduct research in NSA databases, scrutinize Internet protocol addresses and target “Internet links that terminate in a foreign country.”

“When NSA proposes to direct surveillance at a target, it does so because NSA has already learned something about the target,” according to the targeting rules. Often, that lead comes from the CIA or a law enforcement agency.

The NSA uses whatever details are contained in that lead to make an initial assessment of whether it is being asked to eavesdrop on an overseas target. But the agency then takes other steps depending on the circumstances, such as scanning databases “to which NSA has access but did not originate” for clues about location.

To prevent U.S. citizens and legal residents from being targeted, NSA keeps a database of phone numbers and e-mail addresses associated with people thought to be living in the country. New requests are compared to records on the list. Matches are signals to put the surveillance on hold.

The NSA then goes through a sequence of potential additional checks, according to the document. It may look at area codes and the ordinary data packets that accompany e-mails as they cross the Internet. And it may check contact lists associated with e-mail accounts, as well as massive “knowledge databases” that contain CIA intelligence reports.

After it begins intercepting calls or e-mails, the NSA is supposed to continue to look for signs that the person it is monitoring has entered the United States, which would prompt a halt in surveillance and possibly a notification to the FBI.

The document on “minimization” spells out rules for protecting privacy, some of which have been described publicly. The rules protect not just citizens, but foreigners in the United States.

If domestic communications lack significant foreign intelligence information, they must be promptly destroyed. Communications concerning Americans may not be kept more than five years.

If a target who was outside the United States enters the country, the monitoring must stop immediately.

Additional Content:

The NSA slide you haven’t seen

By Robert O’Harrow Jr. and Ellen Nakashima

June 27, 2013

LEFT: Firefighters fight fires inside the Pentagon the day after a plane hit it on Sept. 11, 2001. (Photo by Tracy A. Woodward / The Washington Post). RIGHT: Firefighters unfurl a large flag on the Pentagon the day after the attack. (Photo by John McDonnell/The Washington Post)

In the days after the Sept. 11, 2001, terror attacks, American telephone companies offered the National Security Agency access to domestic calling records and their own analysis of calling patterns, a top-secret document shows.

At first, the NSA could not accept the help because it did not have the legal authority to collect data domestically. That changed on Oct. 4, when President George W. Bush gave his approval for what became known in classified circles as the President’s Surveillance Program, or PSP.

The NSA immediately began cultivating an array of “private sector partners,” including telephone companies, Internet service providers and Web services, according to a top-secret report by the NSA inspector general’s office obtained by The Washington Post.

“Private sector partners began to send telephony and Internet content to NSA in October 2001. They began to send telephony and Internet metadata to NSA as early as November 2001,” the IG report said.

The 57-page document, a working draft dated March 24, 2009, offers a short history of one of the most sweeping domestic surveillance efforts in American history. It was first posted by the Guardian newspaper in England.

The document contains new information about how the decade-long program came to be, including details about legal matters, funding and the fact that 60 lawmakers were briefed about it. The report also contains much that has already been disclosed by The Post and the Guardian, based on documents leaked by former NSA contractor Edward Snowden.

It depicts a program fashioned virtually from scratch in a time of crisis, by a handful of individuals, including Gen. Michael Hayden, the head of the NSA and Vice President Dick Cheney. Given the code name “Stellar Wind,” the PSP was a set of four surveillance programs that brought Americans and U.S. territory within the domain of the NSA for the first time in decades. The PSP, which initially operated outside the restrictions of the Foreign Intelligence Surveillance Act, was eventually put under full FISA court control by 2007.

The report also offers new fodder for critics of domestic spying about the proper limits on domestic intelligence. In recent interviews with The Post, some former senior NSA officials said they had misgivings at the time.

“It was not something that I felt we needed to do or should do,” said one former NSA official who spoke on the condition of anonymity in order to discuss top-secret matters. “I thought there was a way to do it, which was to put this under FBI control, using FBI authorities, and just let the FBI use our [NSA] tools. I was just thinking, what kind of precedent does this set?”

The NSA inspector general’s office issued the classified report under a mandate from the FISA Amendments Act of 2008. The report also shows that NSA officials believed that there were no constitutional limits on the collection of digital metadata, including such details as the origin, destination and timing of calls and e-mails.

More significantly, the inspector general’s report underscores the NSA’s fundamental reliance on private-sector companies.

“According to General [Keith B.] Alexander, General Hayden’s replacement as Director of NSA/CSS, if the relationships with these companies were ever terminated, the U.S. SIGINT system would be irrevocably damaged, because NSA would have sacrificed America’s home field advantage as the primary hub for worldwide telecommunications,” the report said.

Hayden, now a security consultant, was out of the country and unavailable for comment.

The four programs under the PSP involved the collection of Internet and telephone metadata and content. The Internet metadata program “was terminated in 2011 because it didn’t have the operational impact that we needed,” Alexander said Thursday while speaking at a cyber conference in Baltimore. Alexander said that the administration and Congress supported the shutdown and the “data was purged.”

Greg Nojeim, senior counsel and civil liberties specialist at the Center for Democracy and Technology (CDT), said such a program was not supposed to happen under FISA.

“The document shows that the president authorized, and the NSA conducted, the very electronic surveillance that Congress enacted FISA to preclude,” Nojeim said. “No court orders. No finding of probable cause. In fact, the FISA court doesn’t even get briefed about it until four months later. Due process under this program at inception was just a handshake between the head of the NSA and the vice president’s counsel.”

The program was born at a time when the nation was experiencing its most acute security crisis since the Pearl Harbor attacks 60 years earlier. In the hours after Sept. 11, 2001, the NSA scrambled to determine how to use its surveillance tools within a legal framework that tightly proscribed domestic intelligence collection.

“General Hayden was operating in a unique environment in which it was a widely held belief that additional attacks on U.S. soil were imminent,” the report said.

NSA officials found “collection gaps” that left the country vulnerable. “NSA believed that the FISA process was unable to accommodate the number of terrorist targets or the speed with which they changed their communication,” the report said, adding that the average wait for FISA approval was four to six weeks.

Three days after the attacks, Hayden approved the targeting of terrorist-associated foreign telephone numbers on communication links between the United States and foreign countries, the report said.

On Oct. 2, Hayden briefed members of the House Permanent Select Committee on Intelligence and Senate Select Committee on Intelligence about his decision to allow expanded data collection.

The president issued a memo on Oct. 4 that authorized “specified electronic surveillance” for a “limited period” to detect and thwart terrorism inside the United States. The order delegated authority to the secretary of defense, who passed on the authority to Hayden.

Under the initial authorization, as long as the NSA had probable cause to think one of the people communicating was in Afghanistan or was engaged in planning an act of international terrorism, the agency could collect communications data outside the FISA process. “The majority of known terrorist email addresses that NSA has tracked are hosted on U.S.-based providers,” the report said.

Days later, the PSP’s metadata analysis center was a round-the-clock operation with 20 analysts and software developers. Many of them were “former Russian traffic analysts,” the report said. Within a week, 90 NSA employees were clear to view the PSP material.

The “PSP content collection” ultimately targeted 3,018 people in the United States from Oct. 4, 2001, to Jan. 17, 2007, the report said. Targeting of foreign nationals was broadened beyond Afghanistan shortly after the initial authorization. Of the targets, 34,646, or 92 percent, were foreign.

The report notes: “NSA leadership considered selectors for targets located in the United States to be extremely sensitive. As such, processes were set up to ensure strict compliance with the terms of the Authorization.”

But the sweep of the PSP went beyond content and included the collection of vast amounts of metadata. Among the legal assumptions was that “metadata was not constitutionally protected” and not as sensitive as content.

“Nevertheless, processes were set up to document requests for metadata analysis and justifications for conducting such analysis under Program authority,” the report said.

The report does not specify how much metadata was collected, but it pointed out that 37 billion minutes in telephone conversations originated or ended in the United States in 2003 alone.

“NSA determined that under the Authorization it could gain access to approximately 81% of the international calls into and out of the United States through three corporate partners,” the report said.

Barton Gellman, Carol Leonnig, Kimbriell Kelly and Sari Horwitz contributed to this report.

By Ellen Nakashima and Joby Warrick

July 14, 2013

NSA chief Keith Alexander has vigorously defended the agency’s data collection program. (Bill O’Leary/The Washington Post)

In late 2005, as Iraqi roadside bombings were nearing an all-time peak, the National Security Agency’s newly appointed chief began pitching a radical plan for halting the attacks that were killing or wounding a dozen Americans a day.

At the time, more than 100 teams of U.S. analysts were scouring Iraq for snippets of electronic data that might lead to the bomb-makers and their hidden factories. But the NSA director, Gen. Keith B. Alexander, wanted more than mere snippets. He wanted everything: Every Iraqi text message, phone call and e-mail that could be vacuumed up by the agency’s powerful computers.

“Rather than look for a single needle in the haystack, his approach was, ‘Let’s collect the whole haystack,’ ” said one former senior U.S. intelligence official who tracked the plan’s implementation. “Collect it all, tag it, store it. . . . And whatever it is you want, you go searching for it.”

The unprecedented data collection plan, dubbed Real Time Regional Gateway, would play a role in breaking up Iraqi insurgent networks and significantly reducing the monthly death toll from improvised explosive devices by late 2008. It also encapsulated Alexander’s controversial approach to safeguarding Americans from what he sees as a host of imminent threats, from terrorism to devastating cyberattacks.

In his eight years at the helm of the country’s electronic surveillance agency, Alexander, 61, has quietly presided over a revolution in the government’s ability to scoop up information in the name of national security. And, as he did in Iraq, Alexander has pushed hard for everything he can get: tools, resources and the legal authority to collect and store vast quantities of raw information on American and foreign communications.

His successes have won accolades from political leaders of both parties as well as from counterterrorism and intelligence professionals who say the NSA chief’s efforts have helped foil dozens of terrorist attacks. His approach also has drawn attack from civil rights groups and a bipartisan group of lawmakers. One Democrat who confronted Alexander at a congressional hearing last month accused the NSA of crossing a line by collecting the cellphone records of millions of Americans.

“What authorization gave you the grounds for acquiring my cellphone data?” demanded Sen. Jeff Merkley (D-Ore.), waving his mobile phone at the four-star general.

New details of the spy agency’s vast reach were brought to light last month by former NSA contractor Edward Snowden, who leaked classified information on government programs that sweep up “metadata” on phone calls and e-mails by Americans. Those revelations in turn have spotlighted the role played by Alexander, the NSA’s avuncular leader and, by all accounts, a driving force behind a post-Sept. 11, 2001, quest to transform an agency inundated by the data revolution into one that can exploit it to defend the nation.

As portrayed by supporters, Alexander is animated by a spymaster’s awareness of serious, overlapping threats arrayed against the United States. They include foreign and homegrown terrorists. They also include a host of adversaries who are constantly probing the country’s cyberdefenses, looking for opportunities to steal secrets or unleash mayhem by shutting down critical infrastructure. Like many national security officials of his generation, Alexander’s sensibilities were shaped by a series of painful intelligence lapses leading up to the Sept. 11 attacks.

To some of Alexander’s most vociferous critics, Snowden’s disclosures confirm their image of an agency and a director so enamored of technological prowess that they have sacrificed privacy rights.

“He is absolutely obsessed and completely driven to take it all, whenever possible,” said Thomas Drake, a former NSA official and whistleblower. The continuation of Alexander’s policies, Drake said, would result in the “complete evisceration of our civil liberties.”

Alexander frequently points out that collection programs are subject to oversight by Congress as well as the U.S. Foreign Intelligence Surveillance Court, although the proceedings of both bodies are shrouded in secrecy. But even his defenders say Alexander’s aggressiveness has sometimes taken him to the outer edge of his legal authority.

Some in Congress complain that Alexander’s NSA is sometimes slow to inform the oversight committees of problems, particularly when the agency’s eavesdroppers inadvertently pick up communications that fall outside the NSA’s legal mandates. Others are uncomfortable with the extraordinarily broad powers vested in the NSA chief. In 2010, he became the first head of U.S. Cyber Command, set up to defend Defense Department networks against hackers and, when authorized, conduct attacks on adversaries. Pentagon officials and Alexander say the command’s mission is also to defend the nation against cyberattacks.

“He is the only man in the land that can promote a problem by virtue of his intelligence hat and then promote a solution by virtue of his military hat,” said one former Pentagon official, voicing a concern that the lines governing the two authorities are not clearly demarcated and that Alexander can evade effective public oversight as a result. The former official spoke on the condition of anonymity to be able to talk freely.

Alexander himself has expressed unease about secrecy constraints that he says prohibit him from fully explaining what the NSA does. But just as in Iraq, he remains fiercely committed to the belief that “we need to get it all,” said Timothy Edgar, a former privacy officer at the Office of the Director of National Intelligence and at the White House.

“He certainly believes you need to collect everything you can under the law,” Edgar said, “and that includes pushing for pretty aggressive interpretations of the law.”

Alexander maintained in a speech last month that he is mindful to “do everything you can to protect civil liberties and privacy.”

He then added a warning: “Everyone also understands,” he said, “that if we give up a capability that is critical to the defense of this nation, people will die.”

The NSA’s 5,000-acre campus at Fort Meade in suburban Maryland contains more than 1,300 heavily guarded buildings and an array of computers and gadgetry sometimes described as “the most wonderful electronic toybox in the world.” But even before his arrival there in 2005 as director, Alexander was regarded as a leading apostle for harnessing technology’s awesome power in the service of national security.

In the mid-1990s, he had pioneered work in battlefield visualization — helping the soldier see on a laptop screen where friendly and enemy forces were on a detailed terrain map. He later led the push for technological innovation as head of the Army’s Intelligence and Security Command.

At the NSA, he has presided over the agency’s physical expansion while moving quickly to speed up the collection, analysis and sharing of massive amounts of raw data.

In the process, the West Point graduate has risen to become arguably the most powerful U.S. general most Americans have never heard of — or at least not until the Snowden affair propelled him into the public spotlight. He runs the nation’s largest, costliest and most secretive intelligence agency — an organization whose very existence was not officially acknowledged in its early years. The NSA holds primacy over all electronic eavesdropping overseas, from routine spying on foreign governments to sweeping up communications between terrorists overseas. Most of it is now done through hacking into networks in other countries.

Three years ago, Alexander’s responsibilities ballooned with his appointment to lead the Pentagon’s U.S. Cyber Command.

Even close allies have fretted about the concentration of so much responsibility — not to mention influence — in a single individual. Some worry that the job was tailored to the skills of one man and not to the mission itself.

“Where do you find another person like General Alexander who possesses the intel qualifications to run NSA and the operational experience to command” the military’s cyber operations, said retired Lt. Gen. Robert J. Elder, who launched the Air Force’s first cyber command in 2007.

No one questions Alexander’s qualifications for the dual role. With a trio of master’s degrees in electronic warfare, physics and national security strategy, the Syracuse, N.Y., native is endowed with technical smarts and ambition as well as a folksy, even goofy, charm that can disarm critics.

He routinely makes references — in speeches and in person with colleagues — to his four daughters and his 14, soon-to-be 15, grandchildren, including how the youngest, a toddler, Skypes with Grandma.

In January, he was on a cybersecurity panel in Munich when someone brought up Twitter’s announcement that 250,000 of its accounts had been hacked. “I didn’t do it,” Alexander said. “I was here. I have an alibi.” Then, turning to a fellow panelist, an official from the Chinese tech company Huawei, he quipped: “Do you?”

Last summer, Alexander waded into the lion’s den when, wearing jeans and a black T-shirt, he delivered a keynote address at the Def Con hackers convention in Las Vegas and mugged for the cameras. At the time, he told the crowd that “the story that we [at the NSA] have millions or hundreds of millions of dossiers on people is absolutely false.”

That line irked lawmakers who were aware of the NSA’s secret collection of phone data. And this year, after Snowden’s revelations, Def Con organizers said federal officials were not welcome at the event.

Alexander’s outsize commitment to his mission has propelled him to seek new capabilities with an impatience that has sometimes rankled superiors, while also raising concerns about privacy and how far he wants to push into offensive uses of cybertools. Alexander has argued for covert action authority, which is traditionally the domain of the CIA, individuals familiar with the matter say.

He has been credited as a key supporter of the development of Stuxnet, the computer worm that infected Iran’s main uranium enrichment facility in 2009 and 2010 and is the most aggressive known use to date of offensive cyberweaponry. U.S. officials have never publicly acknowledged involvement in what has been described by experts as the first known, industrial-scale cyberattack on a sovereign nation, one that is estimated to have set back Iran’s uranium production by as much as a year.

Alexander also pushed hard for expanded authority to see into U.S. private sector networks to help defend them against foreign cyberattacks.

Quiet concerns also have been voiced by some of the private companies that would potentially benefit from government protection against cyberattack.

At a private meeting with financial industry officials a few years ago, Alexander spoke about the proliferation of computer malware aimed at siphoning data from networks, including those of banks. The meeting was described by a participant who spoke on the condition of anonymity because the discussion was off the record.

His proposed solution: Private companies should give the government access to their networks so it could screen out the harmful software. The NSA chief was offering to serve as an all-knowing virus-protection service, but at the cost, industry officials felt, of an unprecedented intrusion into the financial institutions’ databases.

The group of financial industry officials, sitting around a table at the Office of the Director of National Intelligence, were stunned, immediately grasping the privacy implications of what Alexander was politely but urgently suggesting. As a group, they demurred.

“He’s an impressive person,” the participant said, recalling the group’s collective reaction to Alexander. “You feel very comfortable with him. He instills a high degree of trust.”

But he was proposing something they thought was high-risk.

“Folks in the room looked at each other like, ‘Wow. That’s kind of wild.’ ”

By Barton Gellman

August 15, 2013

The National Security Agency has broken privacy rules or overstepped its legal authority thousands of times each year since Congress granted the agency broad new powers in 2008, according to an internal audit and other top-secret documents.

Most of the infractions involve unauthorized surveillance of Americans or foreign intelligence targets in the United States, both of which are restricted by statute and executive order. They range from significant violations of law to typographical errors that resulted in unintended interception of U.S. e-mails and telephone calls.

The documents, provided earlier this summer to The Washington Post by former NSA contractor Edward Snowden, include a level of detail and analysis that is not routinely shared with Congress or the special court that oversees surveillance. Inone of the documents, agency personnel are instructed to remove details and substitute more generic language in reports to the Justice Department and the Office of the Director of National Intelligence.

In one instance, the NSA decided that it need not report the unintended surveillance of Americans. A notable example in 2008 was the interception of a “large number” of calls placed from Washington when a programming error confused the U.S. area code 202 for 20, the international dialing code for Egypt, according to a “quality assurance” review that was not distributed to the NSA’s oversight staff.

In another case, the Foreign Intelligence Surveillance Court, which has authority over some NSA operations, did not learn about a new collection method until it had been in operation for many months. The court ruled it unconstitutional.

The Obama administration has provided almost no public information about the NSA’s compliance record. In June, after promising to explain the NSA’s record in “as transparent a way as we possibly can,” Deputy Attorney General James Cole described extensive safeguards and oversight that keep the agency in check. “Every now and then, there may be a mistake,” Cole said in congressional testimony.

The NSA audit obtained by The Post, dated May 2012, counted 2,776 incidents in the preceding 12 months of unauthorized collection, storage, access to or distribution of legally protected communications. Most were unintended. Many involved failures of due diligence or violations of standard operating procedure. The most serious incidents included a violation of a court order and unauthorized use of data about more than 3,000 Americans and green-card holders.

In a statement in response to questions for this article, the NSA said it attempts to identify problems “at the earliest possible moment, implement mitigation measures wherever possible, and drive the numbers down.” The government was made aware of The Post’s intention to publish the documents that accompany this article online.

“We’re a human-run agency operating in a complex environment with a number of different regulatory regimes, so at times we find ourselves on the wrong side of the line,” a senior NSA official said in an interview, speaking with White House permission on the condition of anonymity.

“You can look at it as a percentage of our total activity that occurs each day,” he said. “You look at a number in absolute terms that looks big, and when you look at it in relative terms, it looks a little different.”

There is no reliable way to calculate from the number of recorded compliance issues how many Americans have had their communications improperly collected, stored or distributed by the NSA.

The causes and severity of NSA infractions vary widely. One in 10 incidents is attributed to a typographical error in which an analyst enters an incorrect query and retrieves data about U.S phone calls or e-mails.

But the more serious lapses include unauthorized access to intercepted communications, the distribution of protected content and the use of automated systems without built-in safeguards to prevent unlawful surveillance.

The May 2012 audit, intended for the agency’s top leaders, counts only incidents at the NSA’s Fort Meade headquarters and other facilities in the Washington area. Three government officials, speaking on the condition of anonymity to discuss classified matters, said the number would be substantially higher if it included other NSA operating units and regional collection centers.

Senate Intelligence Committee Chairman Dianne Feinstein (D-Calif.), who did not receive a copy of the 2012 audit until The Post asked her staff about it, said in a statement late Thursday that the committee “can and should do more to independently verify that NSA’s operations are appropriate, and its reports of compliance incidents are accurate.”

Despite the quadrupling of the NSA’s oversight staff after a series of significant violations in 2009, the rate of infractions increased throughout 2011 and early 2012. An NSA spokesman declined to disclose whether the trend has continued since last year.

One major problem is largely unpreventable, the audit says, because current operations rely on technology that cannot quickly determine whether a foreign mobile phone has entered the United States.

In what appears to be one of the most serious violations, the NSA diverted large volumes of international data passing through fiber-optic cables in the United States into a repository where the material could be stored temporarily for processing and selection.

The operation to obtain what the agency called “multiple communications transactions” collected and commingled U.S. and foreign e-mails, according to an article in SSO News, a top-secret internal newsletter of the NSA’s Special Source Operations unit. NSA lawyers told the court that the agency could not practicably filter out the communications of Americans.

In October 2011, months after the program got underway, the Foreign Intelligence Surveillance Court ruled that the collection effort was unconstitutional. The court said that the methods used were “deficient on statutory and constitutional grounds,” according to a top-secret summary of the opinion, and it ordered the NSA to comply with standard privacy protections or stop the program.

James R. Clapper Jr., the director of national intelligence, has acknowledged that the court found the NSA in breach of the Fourth Amendment, which prohibits unreasonable searches and seizures, but the Obama administration has fought a Freedom of Information lawsuit that seeks the opinion.

Generally, the NSA reveals nothing in public about its errors and infractions. The unclassified versions of the administration’s semiannual reports to Congress feature blacked-out pages under the headline “Statistical Data Relating to Compliance Incidents.”

Members of Congress may read the unredacted documents, but only in a special secure room, and they are not allowed to take notes. Fewer than 10 percent of lawmakers employ a staff member who has the security clearance to read the reports and provide advice about their meaning and significance.

The limited portions of the reports that can be read by the public acknowledge “a small number of compliance incidents.”

Under NSA auditing guidelines, the incident count does not usually disclose the number of Americans affected.

“What you really want to know, I would think, is how many innocent U.S. person communications are, one, collected at all, and two, subject to scrutiny,” said Julian Sanchez, a research scholar and close student of the NSA at the Cato Institute.

The documents provided by Snowden offer only glimpses of those questions. Some reports make clear that an unauthorized search produced no records. But a single “incident” in February 2012 involved the unlawful retention of 3,032 files that the surveillance court had ordered the NSA to destroy, according to the May 2012 audit. Each file contained an undisclosed number of telephone call records.

One of the documents sheds new light on a statement by NSA Director Keith B. Alexander last year that “we don’t hold data on U.S. citizens.”

Some Obama administration officials, speaking on the condition of anonymity, have defended Alexander with assertions that the agency’s internal definition of “data” does not cover “metadata” such as the trillions of American call records that the NSA is now known to have collected and stored since 2006. Those records include the telephone numbers of the parties and the times and durations of conversations, among other details, but not their content or the names of callers.

The NSA’s authoritative definition of data includes those call records. “Signals Intelligence Management Directive 421,” which is quoted in secret oversight and auditing guidelines, states that “raw SIGINT data . . . includes, but is not limited to, unevaluated and/or unminimized transcripts, gists, facsimiles, telex, voice, and some forms of computer-generated data, such as call event records and other Digital Network Intelligence (DNI) metadata as well as DNI message text.”

In the case of the collection effort that confused calls placed from Washington with those placed from Egypt, it is unclear what the NSA meant by a “large number” of intercepted calls. A spokesman declined to discuss the matter.

The NSA has different reporting requirements for each branch of government and each of its legal authorities. The “202” collection was deemed irrelevant to any of them. “The issue pertained to Metadata ONLY so there were no defects to report,” according to the author of the secret memo from March 2013.

The large number of database query incidents, which involve previously collected communications, confirms long-standing suspicions that the NSA’s vast data banks — with code names such as MARINA, PINWALE and XKEYSCORE — house a considerable volume of information about Americans. Ordinarily the identities of people in the United States are masked, but intelligence “customers” may request unmasking, either one case at a time or in standing orders.

In dozens of cases, NSA personnel made careless use of the agency’s extraordinary powers, according to individual auditing reports. One team of analysts in Hawaii, for example, asked a system called DISHFIRE to find any communications that mentioned both the Swedish manufacturer Ericsson and “radio” or “radar” — a query that could just as easily have collected on people in the United States as on their Pakistani military target.

The NSA uses the term “incidental” when it sweeps up the records of an American while targeting a foreigner or a U.S. person who is believed to be involved in terrorism. Official guidelines for NSA personnel say that kind of incident, pervasive under current practices, “does not constitute a . . .violation” and “does not have to be reported” to the NSA inspector general for inclusion in quarterly reports to Congress. Once added to its databases, absent other restrictions, the communications of Americans may be searched freely.

In one required tutorial, NSA collectors and analysts are taught to fill out oversight forms without giving “extraneous information” to “our FAA overseers.” FAA is a reference to the FISA Amendments Act of 2008, which granted broad new authorities to the NSA in exchange for regular audits from the Justice Department and the Office of the Director of National Intelligence and periodic reports to Congress and the surveillance court.

Using real-world examples, the “Target Analyst Rationale Instructions” explain how NSA employees should strip out details and substitute generic descriptions of the evidence and analysis behind their targeting choices.

“I realize you can read those words a certain way,” said the high-ranking NSA official who spoke with White House authority, but the instructions were not intended to withhold information from auditors. “Think of a book of individual recipes,” he said. Each target “has a short, concise description,” but that is “not a substitute for the full recipe that follows, which our overseers also have access to.”

Julie Tate and Carol D. Leonnig contributed to this report.

Additional Content:

NSA report on privacy violations in the first quarter of 2012

First direct evidence of illegal surveillance found by the FISA court

What’s a ‘violation’?

What to say, and not to say, to ‘our overseers’

By Carol Leonnig

August 15, 2013

The leader of the secret court that is supposed to provide critical oversight of the government’s vast spying programs said that its ability to do so is limited and that it must trust the government to report when it improperly spies on Americans.

The chief judge of the Foreign Intelligence Surveillance Court said the court lacks the tools to independently verify how often the government’s surveillance breaks the court’s rules that aim to protect Americans’ privacy. Without taking drastic steps, it also cannot check the veracity of the government’s assertions that the violations its staff members report are unintentional mistakes.

“The FISC is forced to rely upon the accuracy of the information that is provided to the Court,” its chief, U.S. District Judge Reggie B. Walton, said in a written statement to The Washington Post. “The FISC does not have the capacity to investigate issues of noncompliance, and in that respect the FISC is in the same position as any other court when it comes to enforcing [government] compliance with its orders.”

Walton’s comments came in response to internal government records obtained by The Post showing that National Security Agency staff members in Washington overstepped their authority on spy programs thousands of times per year. The records also show that the number of violations has been on the rise.

The court’s description of its practical limitations contrasts with repeated assurances from the Obama administration and intelligence agency leaders that the court provides central checks and balances on the government’s broad spying efforts. They have said that Americans should feel comfortable that the secret intelligence court provides robust oversight of government surveillance and protects their privacy from rogue intrusions.

President Obama and other government leaders have emphasized the court’s oversight role in the wake of revelations this year that the government is vacuuming up “metadata” on Americans’ telephone and Internet communications.

“We also have federal judges that we’ve put in place who are not subject to political pressure,” Obama said at a news conference in June. “They’ve got lifetime tenure as federal judges, and they’re empowered to look over our shoulder at the executive branch to make sure that these programs aren’t being abused.”

Privacy advocates and others in government have voiced concerns about the ability of overseers to police secret programs of immense legal and technological complexity. Several members of the House and Senate intelligence committees told The Post last week that they face numerous obstacles and constraints in questioning spy agency officials about their work.

In 2009, for example, a Justice Department review uncovered a major operational glitch that had led to a series of significant violations of the court’s order and notified the court, according to records that were declassified July 31 by the Office of the Director of National Intelligence.

The government described the problem as one of “over-collection” of metadata records for U.S. phone calls.

In September 2009, NSA Director Keith B. Alexander made a presentation to the FISA court about the agency’s effort to remedy the problem.

“FISA Court placed several restrictions on aspects of the business records collection program until the compliance processes were improved to its satisfaction,” the memo stated.

The public summaries of the violations do not say how long the problem went undetected and unreported to the court, or what information was improperly gathered by the agency’s automated collection systems.

“The problems generally involved the implementation of highly sophisticated technology in a complex and ever-changing communications environment which, in some instances, results in the automated tools operating in a manner that was not completely consistent with the specific terms of the Court’s orders,” according to unredacted portions of a December 2009 memo provided to the Senate and House intelligence committees.

Two people familiar with the 2009 flaw said that the agency was collecting more “fields” of information from the customer records of telephone companies than the court had approved. The NSA declined to answer questions about the event.

One senior intelligence official, who was authorized by the White House to speak on the condition of anonymity, described the 2009 incident as a “major event” that prompted the agency to dramatically increase its compliance staff.

“We uncovered some disconnects between us and our overseers, disconnects between what we had put in documentation, the way we had described things in documentation,” the official said.

Although the violation was unintentional, the official said, “it wasn’t always the easiest of discussions” with the court.

The agency paused, “got ourselves with our overseers back into fair territory,” and has since made “substantial improvement” in compliance, the official said.

Privacy advocates say they fear that some violations are never reported to the court.

In January 2008, the NSA appeared to have mistakenly collected data on numerous phone calls from the Washington area code 202, thinking they were foreign phone calls from Egypt, whose country code is 20. According to a 2013 “quality assurance” review of the incident, a communications switch misread the coding of the calls and presumed they were international. The NSA has broad authority that is not subject to the FISA court to collect and monitor foreign communications under certain circumstances.

The description of the 2008 problem suggests that the inadvertent collection of U.S. phone calls was not reported to the FISA court.

“However, the issue pertained to Metadata ONLY so there were no defects to report,” the review stated.

Under FISA rules, the government is required to immediately notify the court if it believes it has violated any of its orders on surveillance.

The government does not typically provide the court with case-specific detail about individual compliance cases, such as the names of people it later learned it was improperly searching in its massive phone or e-mail databases, according to the two people familiar with the court’s work.

In contrast to the dozens of staff available to Congress’s intelligence and judiciary committees, the FISA court has five lawyers to review compliance violation reports.

A staff lawyer can elevate a concern about a significant compliance issue to a judge on the court, according to a letter Walton recently sent to the Senate describing the court’s role.

The court can always demand and obtain more details about cases, but it is unclear how often that occurs. In the past, while grappling with rules for implementing the surveillance programs, judges on the court have requested a visit to NSA headquarters to inspect the operations, the officials said.

Last week, the president said that he recognizes that some Americans may lack trust in the oversight process — in which the secret court approves the rules for collecting Americans’ communications — and that he will work with Congress on reforms, which could include a privacy advocate to the court.

“In other words, it’s not enough for me as president to have confidence in these programs,” Obama said in his news conference. “The American people need to have confidence in them, as well.”

Barton Gellman, Peter Wallsten and Alice Crites contributed to this report.

By Ellen Nakashima

August 21, 2013

Sen. Ron Wyden (D-Ore.) got the Office of the Director of National Intelligence to acknowledge in July 2012 that the NSA’s surveillance had at least once violated the Constitution. (Photo by Melina Mara/The Washington Post)

For several years, the National Security Agency unlawfully gathered tens of thousands of e-mails and other electronic communications between Americans as part of a now-revised collection method, according to a 2011 secret court opinion.

The redacted 85-page opinion, which was declassified by U.S. intelligence officials on Wednesday, states that, based on NSA estimates, the spy agency may have been collecting as many as 56,000 “wholly domestic” communications each year.

In a strongly worded opinion, the chief judge of the Foreign Intelligence Surveillance Court expressed consternation at what he saw as a pattern of misleading statements by the government and hinted that the NSA possibly violated a criminal law against spying on Americans.

“For the first time, the government has now advised the court that the volume and nature of the information it has been collecting is fundamentally different from what the court had been led to believe,” John D. Bates, then the surveillance court’s chief judge, wrote in his Oct. 3, 2011, opinion.

The court, which meets in secret, oversees the Foreign Intelligence Surveillance Act, the law authorizing such surveillance in the United States. It has been criticized by some as a “rubber stamp” for the government, but the opinion makes clear the court does not see itself that way.

Bates’s frustration with the government’s lack of candor extended beyond the program at issue to other NSA surveillance efforts.

“The court is troubled that the government’s revelations regarding NSA’s acquisition of Internet transactions mark the third instance in less than three years in which the government has disclosed a substantial misrepresentation regarding the scope of a major collection program,” Bates wrote in a scathing footnote.

The Washington Post reported last week that the court had ruled the collection method unconstitutional. The declassified opinion sheds new light on the volume of Americans’ communications that were obtained by the NSA and the nature of the violations, as well as the FISA court’s interpretation of the program.

The release marks the first time the government has disclosed a FISA court opinion in response to a Freedom of Information Act lawsuit. The lawsuit was brought a year ago by the Electronic Frontier Foundation, a privacy group.

“It’s unfortunate it took a year of litigation and the most significant leak in American history to finally get them to release this opinion,” said foundation staff attorney Mark Rumold, “but I’m happy that the administration is beginning to take this debate seriously.”

The pressure to release the opinion was heightened by a series of recent revelations about government surveillance based on documents leaked to The Washington Post and Britain’s Guardian newspaper by former NSA contractor Edward Snowden.

Over the past 21 / 2 months, those revelations have reignited a national debate on the balance between privacy and security, and President Obama has promised to assuage concerns about government overreach, in part through more transparency.

The document was released along with several others related to a controversial collection program approved by Congress in 2008 under Section 702 of the FISA Amendments Act. Through that program, the NSA may target for collection the e-mails and phone calls of foreigners “reasonably believed” to be overseas.

Under Section 702, the NSA collects more than 250 million Internet communications each year, the opinion said. The vast majority — 91 percent — are obtained from Internet providers such as Google, Yahoo and AOL through a program code-named PRISM.

At issue here was the less voluminous “upstream” collection that takes place as communications flow across Internet hubs — not from service providers such as Google. Under that program, the NSA diverted international data passing through fiber-optic cables in the United States into a repository where the material could be stored temporarily for processing and for the selection of foreign communications, rather than domestic ones. But in practice, because of technological difficulties, the NSA was unable to filter out the “wholly domestic” communications between Americans.

Officials stressed that it was the NSA that brought the collection method to the court’s attention as part of its regular reporting process. “This was not in any respect an intentional or wholesale breach of privacy of American persons,” Robert S. Litt III, the general counsel for the Office of the Director of National Intelligence, told reporters Wednesday.

Still, Bates noted that it was not until May 2011 — several years after Section 702 was approved — that the NSA told the court that its upstream collection of Internet communications may contain entire Internet “transactions” not related to the target. In other words, the agency may be collecting e-mails between two Americans or people inside the United States in violation of FISA.

In June 2011, the NSA informed Bates that an Internet transaction may be a single communication or it may include “multiple discrete communications,” including those that are not to, from or about a target. That means instead of one e-mail, a string of Americans’ e-mails could be inadvertently picked up. “That revelation fundamentally alters the Court’s understanding of the scope of the collection conducted pursuant to Section 702,” Bates said.

“By expanding its Section 702 acquisitions to include the acquisition of Internet transactions through its upstream collection, NSA has, as a practical matter, circumvented the spirit of [the law],” Bates wrote. “NSA’s knowing acquisition of tens of thousands of wholly domestic communications through its upstream collection is a cause of concern for the court.”

He ordered the collection to stop until the NSA could propose an acceptable remedy. In November 2011, Bates signed an order approving the fix, which included a new technical means to segregate transactions most likely to contain U.S. persons’ communications and reducing the retention period from five to two years.

In April 2012, the NSA decided to conduct a purge of all upstream data collected since Section 702’s inception in 2008, senior intelligence officials said. They could not estimate the quantity, but one official said it was “lots.” Said another: “It would have been everything.”

The newly released opinion also reflects Bates’s frustration with the court’s inability to independently verify the NSA’s assertions, a sentiment underscored in a recent statement made to The Post by the current chief judge, Reggie B. Walton.

Because of the “sheer volume” of transactions acquired by the NSA, “any meaningful review of the entire body of the transactions” was not feasible, Bates wrote. “As a result, the court cannot know for certain the exact number” of wholly domestic communications but was reliant on the NSA’s samples of data. “Even if the court accepts the validity of conclusions derived from statistical analyses, there are significant hurdles in assessing NSA’s upstream collection,” he wrote.

He also stated in a footnote that the government’s revelations about the scope of the NSA’s upstream collection “implicate” a law that criminalizes unauthorized electronic surveillance. He said that he would address that issue in a separate order.

In another footnote, he also noted that in March 2009 the court concluded that its authorization of the NSA’s bulk collection of Americans’ phone-call records was “premised on a flawed depiction of how the NSA” uses the data. He also wrote: “This misperception by the FISC existed from the inception of its authorized collection in May 2006, buttressed by repeated inaccurate statements made in the government’s submissions, and despite a government-devised and court-mandated oversight regime.”

In that program, which was disclosed through a document leaked by Snowden to the Guardian, the NSA amasses a database of hundreds of millions of Americans’ phone-call records. That includes numbers dialed and the time and duration of calls — also known as metadata — but no content.

Bates continued: “Contrary to the government’s repeated assurances, NSA had been routinely running queries of the metadata using querying terms that did not meet the required standard. . . . The Court concluded that this requirement had been ‘so frequently and systematically violated that it can fairly be said that this critical element of the overall . . . regime has never functioned effectively.’ ”

The Electronic Frontier Foundation sued after Sen. Ron Wyden (D-Ore.) got the Office of the Director of National Intelligence to acknowledge in July 2012 that the NSA’s surveillance had at least once violated the Constitution.

“The FISA Court has noted that this collection violates the spirit of the law, but the government has failed to address this concern in the two years since this ruling was issued,” Wyden said Wednesday. “This ruling makes it clear that FISA Section 702, as written, is insufficient to adequately protect the civil liberties and privacy rights of law-abiding Americans and should be reformed.”

By Barton Gellman and Greg Miller

August 29, 2013

U.S. spy agencies have built an intelligence-gathering colossus since the attacks of Sept. 11, 2001, but remain unable to provide critical information to the president on a range of national security threats, according to the government’s top-secret budget.

The $52.6 billion “black budget” for fiscal 2013, obtained by The Washington Post from former intelligence contractor Edward Snowden, maps a bureaucratic and operational landscape that has never been subject to public scrutiny. Although the government has annually released its overall level of intelligence spending since 2007, it has not divulged how it uses the money or how it performs against the goals set by the president and Congress.

The 178-page budget summary for the National Intelligence Program details the successes, failures and objectives of the 16 spy agencies that make up the U.S. intelligence community, which has 107,035 employees.

The summary describes cutting-edge technologies, agent recruiting and ongoing operations. The Post is withholding some information after consultation with U.S. officials who expressed concerns about the risk to intelligence sources and methods. Sensitive details are so pervasive in the documents that The Post is publishing only summary tables and charts online.

“The United States has made a considerable investment in the Intelligence Community since the terror attacks of 9/11, a time which includes wars in Iraq and Afghanistan, the Arab Spring, the proliferation of weapons of mass destruction technology, and asymmetric threats in such areas as cyber-warfare,” Director of National IntelligenceJames R. Clapper Jr. wrote in response to inquiries from The Post.

“Our budgets are classified as they could provide insight for foreign intelligence services to discern our top national priorities, capabilities and sources and methods that allow us to obtain information to counter threats,” he said.

Among the notable revelations in the budget summary:

- Spending by the CIA has surged past that of every other spy agency, with $14.7 billion in requested funding for 2013. The figure vastly exceeds outside estimates and is nearly 50 percent above that of the National Security Agency, which conducts eavesdropping operations and has long been considered the behemoth of the community.

- The CIA and the NSA have begun aggressive new efforts to hack into foreign computer networks to steal information or sabotage enemy systems, embracing what the budget refers to as “offensive cyber operations.”

- Long before Snowden’s leaks, the U.S. intelligence community worried about “anomalous behavior” by employees and contractors with access to classified material. The NSA planned to ward off a “potential insider compromise of sensitive information” by re-investigating at least 4,000 people this year who hold high-level security clearances.

- U.S. intelligence officials take an active interest in friends as well as foes. Pakistan is described in detail as an “intractable target,” and counterintelligence operations “are strategically focused against [the] priority targets of China, Russia, Iran, Cuba and Israel.” The latter is a U.S. ally but has a history of espionage attempts against the United States.

- In words, deeds and dollars, intelligence agencies remain fixed on terrorism as the gravest threat to national security, which is listed first among five “mission objectives.” Counterterrorism programs employ one in four members of the intelligence workforce and account for one-third of the intelligence program’s spending.

- The governments of Iran, China and Russia are difficult to penetrate, but North Korea’s may be the most opaque. There are five “critical” gaps in U.S. intelligence about Pyongyang’s nuclear and missile programs, and analysts know virtually nothing about the intentions of North Korean leader Kim Jong Un.

Formally known as the Congressional Budget Justification for the National Intelligence Program, the “top-secret” blueprint represents spending levels proposed to the House and Senate intelligence committees in February 2012. Congress may have made changes before the fiscal year began on Oct 1. Clapper is expected to release the actual total spending figure after the fiscal year ends on Sept. 30.

The document describes a constellation of spy agencies that track millions of surveillance targets and carry out operations that include hundreds of lethal strikes. They are organized around five priorities: combating terrorism, stopping the spread of nuclear and other unconventional weapons, warning U.S. leaders about critical events overseas, defending against foreign espionage, and conducting cyber-operations.

In an introduction, Clapper said the threats facing the United States “virtually defy rank-ordering.” He warned of “hard choices” as the intelligence community — sometimes referred to as the “IC” — seeks to rein in spending after a decade of often double-digit budget increases.

The current budget proposal envisions that spending will remain roughly level through 2017 and amounts to a case against substantial cuts.

“Never before has the IC been called upon to master such complexity and so many issues in such a resource-constrained environment,” Clapper wrote.

An espionage empire

The summary provides a detailed look at how the U.S. intelligence community has been reconfigured by the massive infusion of resources that followed the 2001 attacks. The United States has spent more than $500 billion on intelligence during that period, an outlay that U.S. officials say has succeeded in its main objective: preventing another catastrophic terrorist attack in the United States.

The result is an espionage empire with resources and a reach beyond those of any adversary, sustained even now by spending that rivals or exceeds the levels at the height of the Cold War.

The current total budget request was2.4 percent below that of fiscal 2012. In constant dollars, it was about twice the estimated size of the 2001 budget and 25 percent above that of 2006, five years into what was then known as the “global war on terror.”

Historical data on U.S. intelligence spending is largely nonexistent. Through extrapolation, experts have estimated that Cold War spending probably peaked in the late 1980s at an amount that would be the equivalent of $71 billion today.

Spending in the most recent cycle surpassed that amount, based on the $52.6 billion detailed in documents obtained by The Post plus a separate $23 billion devoted to intelligence programs that more directly support the U.S. military.

Lee H. Hamilton, an Indiana Democrat who chaired the House Intelligence Committee and co-chaired the commission that investigated the Sept. 11 attacks, said that access to budget details will enable an informed public debate on intelligence spending for the first time, much as Snowden’s disclosures of NSA surveillance programs brought attention to operations that had assembled data on nearly every U.S. citizen.

“Much of the work that the intelligence community does has a profound impact on the life of ordinary Americans, and they ought not to be excluded from the process,” Hamilton said.

“Nobody is arguing that we should be so transparent as to create dangers for the country,” he said. But, he added, “there is a mind-set in the national security community: ‘Leave it to us, we can handle it, the American people have to trust us.’ They carry it to quite an extraordinary length so that they have resisted over a period of decades transparency. . . . The burden of persuasion as to keeping something secret should be on the intelligence community, the burden should not be on the American public.”

Experts said that access to such details about U.S. spy programs is without precedent.

“It was a titanic struggle just to get the top-line budget number disclosed, and that has only been done consistently since 2007,” said Steven Aftergood, an expert at the Federation of American Scientists, a Washington-based organization that provides analyses of national security issues. “But a real grasp of the structure and operations of the intelligence bureaucracy has been totally beyond public reach. This kind of material, even on a historical basis, has simply not been available.”

The only meaningful frame of reference came in 1994, when a congressional subcommittee inadvertently published a partial breakdown of the National Intelligence Program. At the time, the CIA accounted for just $4.8 billion of a budget that totaled $43.4 billion in 2012 dollars. The NSA and the National Reconnaissance Office, which operates satellites and other sensors, commanded far larger shares of U.S. intelligence budgets until years after the Cold War ended.

During the past decade, they have taken a back seat to the CIA.

The NSA was in line to receive $10.5 billion in 2013, and the NRO was to get $10.3 billion — both far below the CIA, whose share had surged to 28 percent of the total budget.

Overall, the U.S. government spends 10 times as much on the Defense Department as it does on spy agencies.

“Today’s world is as fluid and unstable as it has been in the past half century,” Clapper said in his statement to The Post. “Even with stepped up spending on the IC over the past decade, the United States currently spends less than one percent of GDP on the Intelligence Community.”

Dominant position

The CIA’s dominant position is likely to stun outside experts. It represents a remarkable recovery for an agency that seemed poised to lose power and prestige after acknowledging intelligence failures leading up to the 2001 attacks and the 2003 U.S.-led invasion of Iraq.

The surge in resources for the agency funded secret prisons, a controversial interrogation program, the deployment of lethal drones and a huge expansion of its counterterrorism center. The agency was transformed from a spy service struggling to emerge from the Cold War into a paramilitary force.

The CIA has devoted billions of dollars to recruiting and training a new generation of case officers, with the workforce growing from about 17,000 a decade ago to 21,575 this year.

The agency’s budget allocates $2.3 billion for human intelligence operations and $2.5 billion to cover the cost of supporting the security, logistics and other needs of those missions around the world. A relatively small amount of that total, $68.6 million, was earmarked for creating and maintaining “cover,” the false identities employed by operatives overseas.

There is no specific entry for the CIA’s fleet of armed drones in the budget summary, but a broad line item hints at the dimensions of the agency’s expanded paramilitary role, providing more than $2.6 billion for “covert action programs” that would include drone operations in Pakistan and Yemen, payments to militias in Afghanistan and Africa, and attempts to sabotage Iran’s nuclear program.

The black budget illuminates for the first time the intelligence burden of the wars in Afghanistan and Iraq. For 2013, U.S. spy agencies were projected to spend $4.9 billion on “overseas contingency operations.” The CIA accounted for about half of that figure, a sum factored into its overall $14.7 billion budget.

Those war expenditures are projected to shrink as the United States withdraws forces from Afghanistan. The budget also indicates that the intelligence community has cut the number of contractors it hires over the past five years by about 30 percent.

Critical gaps

Despite the vast outlays, the budget blueprint catalogues persistent and in some cases critical blind spots.

Throughout the document, U.S. spy agencies attempt to rate their efforts in tables akin to report cards, generally citing progress but often acknowledging that only a fraction of their questions could be answered — even on the community’s foremost priority, counterterrorism.

In 2011, the budget assessment says intelligence agencies made at least “moderate progress” on 38 of their 50 top counterterrorism gaps, the term used to describe blind spots. Several concern Lebanon’s Hezbollah movement, an enemy of Israel that has not attacked U.S. interests directly since the 1990s.

Other blank spots include questions about the security of Pakistan’s nuclear components when they are being transported, the capabilities of China’s next-generation fighter aircraft, and how Russia’s government leaders are likely to respond to “potentially destabilizing events in Moscow, such as large protests and terrorist attacks.”

A chart outlining efforts to address key questions on biological and chemical weapons is particularly bleak. U.S. agencies set annual goals for at least five categories of intelligence collection related to these weapons. In 2011, the agencies made headway on just two gaps; a year earlier, the mark was zero.

The documents describe expanded efforts to “collect on Russian chemical warfare countermeasures” and assess the security of biological and chemical laboratories in Pakistan.

A table of “critical” gaps listed five for North Korea, more than for any other country that has pursued or is pursuing a nuclear bomb.

The intelligence community seems particularly daunted by the emergence of “homegrown” terrorists who plan attacks in the United States without direct support or instruction from abroad, a threat realized this year, after the budget was submitted, in twin bombings at the Boston Marathon.

The National Counterterrorism Center has convened dozens of analysts from other agencies in attempts to identify “indicators” that could help law enforcement officials understand the path from religious extremism to violence. The FBI was in line for funding to increase the number of agents who surreptitiously track activity on jihadist Web sites.

But a year before the bombings in Boston, the search for meaningful insight into the stages of radicalization was described as one of the “more challenging intelligence gaps.”

High-tech surveillance

The documents make clear that U.S. spy agencies’ long-standing reliance on technology remains intact. If anything, their dependence on high-tech surveillance systems to fill gaps in human intelligence has intensified.

A section on North Korea indicates that the United States has all but surrounded the nuclear-armed country with surveillance platforms. Distant ground sensors monitor seismic activity and scan the country for signs that might point to construction of new nuclear sites. U.S. agencies seek to capture photos, air samples and infrared imagery “around the clock.”

In Iran, new surveillance techniques and technologies have enabled analysts to identify suspected nuclear sites that had not been detected in satellite images, according to the document.

In Syria, NSA listening posts were able to monitor unencrypted communications among senior military officials at the outset of the civil war there, a vulnerability that President Bashar al-Assad’s forces apparently later recognized. One of the NRO’s functions is to extract data from sensors placed on the ground near suspected illicit weapons sites in Syria and other countries.

Across this catalogue of technical prowess, one category is depicted as particularly indispensable: signals intelligence, or SIGINT.

The NSA’s ability to monitor e-mails, phone calls and Internet traffic has come under new scrutiny in recent months as a result of disclosures by Snowden, who worked as a contract computer specialist for the agency before stockpiling secret documents and then fleeing, first to Hong Kong and then Moscow.

The NSA was projected to spend $48.6 million on research projects to assist in “coping with information overload,” an occupational hazard as the volumes of intake have increased sharply from fiber-optic cables and Silicon Valley Internet providers.

The agency’s ability to monitor the communications of al-Qaeda operatives is described in the documents as “often the best and only means to compromise seemingly intractable targets.”

Signals intercepts also have been used to direct the flight paths of drones, gather clues to the composition of North Korea’s leadership and evaluate the response plans of Russia’s government in the event of a terrorist attack in Moscow.

The resources devoted to signals intercepts are extraordinary.

Nearly 35,000 employees are listed under a category called the Consolidated Cryptologic Program, which includes the NSA as well as the surveillance and code-breaking components of the Air Force, Army, Navy and Marines.

The NSA is planning high-risk covert missions, a lesser-known part of its work, to plant what it calls “tailored radio frequency solutions” — close-in sensors to intercept communications that do not pass through global networks.

Even the CIA devotes $1.7 billion, or nearly 12 percent of its budget, to technical collection efforts, including a joint program with the NSA called “CLANSIG,” a covert program to intercept radio and telephone communications from hostile territory.

The agency also is pursuing tracking systems “that minimize or eliminate the need for physical access and enable deep concealment operations against hard targets.”

The CIA has deployed new biometric sensors to confirm the identities and locations of al-Qaeda operatives. The system has been used in the CIA’s drone campaign.

Spending on satellite systems and almost every other category of collection is projected to shrink or remain stagnant in coming years, as Washington grapples with budget cuts across the government. But the 2013 intelligence budget called for increased investment in SIGINT.

Counterintelligence

The budget includes a lengthy section on funding for counterintelligence programs designed to protect against the danger posed by foreign intelligence services as well as betrayals from within the U.S. spy ranks.

The document describes programs to “mitigate insider threats by trusted insiders who seek to exploit their authorized access to sensitive information to harm U.S. interests.”

The agencies had budgeted for a major counterintelligence initiative in fiscal 2012, but most of those resources were diverted to an all-hands emergency response to successive floods of classified data released by the anti-secrecy group WikiLeaks.

For this year, the budget promised a renewed “focus . . . on safeguarding classified networks” and a strict “review of high-risk, high-gain applicants and contractors” — the young, nontraditional computer coders with the skills the NSA needed.

Among them was Snowden, then a 29-year-old contract computer specialist whom the NSA trained to circumvent computer network security. He was copying thousands of highly classified documents at an NSA facility in Hawaii, and preparing to leak them, as the agency embarked on the new security sweep.

“NSA will initiate a minimum of 4,000 periodic reinvestigations of potential insider compromise of sensitive information,” according to the budget, scanning its systems for “anomalies and alerts.”

Julie Tate contributed to this report.

Additional Content:

Inside the 2013 U.S. intelligence ‘black budget’

The Black Budget: Top Secret U.S. Intelligence Funding

America’s secret intelligence budget, in 11 (nay 13) charts

2013 U.S. intelligence budget: additional resources

By Craig Timberg and Barton Gellman

August 29, 2013

The National Security Agency is paying hundreds of millions of dollars a year to U.S. companies for clandestine access to their communications networks, filtering vast traffic flows for foreign targets in a process that also sweeps in large volumes of American telephone calls, e-mails and instant messages.

The bulk of the spending, detailed in a multi-volume intelligence budget obtained by The Washington Post, goes to participants in a Corporate Partner Access Project for major U.S. telecommunications providers. The documents open an important window into surveillance operations on U.S. territory that have been the subject of debate since they were revealed by The Post and Britain’s Guardian newspaper in June.

New details of the corporate-partner project, which falls under the NSA’s Special Source Operations, confirm that the agency taps into “high volume circuit and packet-switched networks,” according to the spending blueprint for fiscal 2013. The program was expected to cost $278 million in the current fiscal year, down nearly one-third from its peak of $394 million in 2011.

Voluntary cooperation from the “backbone” providers of global communications dates to the 1970s under the cover name BLARNEY, according to documents provided by former NSA contractor Edward Snowden. These relationships long predate the PRISM program disclosed in June, under which American technology companies hand over customer data after receiving orders from the Foreign Intelligence Surveillance Court.

In briefing slides, the NSA described BLARNEY and three other corporate projects — OAKSTAR, FAIRVIEW and STORMBREW — under the heading of “passive” or “upstream” collection. They capture data as they move across fiber-optic cables and the gateways that direct global communications traffic.

The documents offer a rare view of a secret surveillance economy in which government officials set financial terms for programs capable of peering into the lives of almost anyone who uses a phone, computer or other device connected to the Internet.

Although the companies are required to comply with lawful surveillance orders, privacy advocates say the multimillion-dollar payments could create a profit motive to offer more than the required assistance.

“It turns surveillance into a revenue stream, and that’s not the way it’s supposed to work,” said Marc Rotenberg, executive director of the Electronic Privacy Information Center, a Washington-based research and advocacy group. “The fact that the government is paying money to telephone companies to turn over information that they are compelled to turn over is very troubling.”

Verizon, AT&T and other major telecommunications companies declined to comment for this article, although several industry officials noted that government surveillance laws explicitly call for companies to receive reasonable reimbursement for their costs.

Previous news reports have made clear that companies frequently seek such payments, but never before has their overall scale been disclosed.

The budget documents do not list individual companies, although they do break down spending among several NSA programs, listed by their code names.

There is no record in the documents obtained by The Post of money set aside to pay technology companies that provide information to the NSA’s PRISM program. That program is the source of 91 percent of the 250 million Internet communications collected through Section 702 of the FISA Amendments Act, which authorizes PRISM and the upstream programs, according to an 2011 opinion and order by the Foreign Intelligence Surveillance Court.

Several of the companies that provide information to PRISM, including Apple, Facebook and Google, say they take no payments from the government when they comply with national security requests. Others say they do take payments in some circumstances. The Guardian reported last week that the NSA had covered “millions of dollars” in costs that some technology companies incurred to comply with government demands for information.

Telecommunications companies generally do charge to comply with surveillance requests, which come from state, local and federal law enforcement officials as well as intelligence agencies.

Former telecommunications executive Paul Kouroupas, a security officer who worked at Global Crossing for 12 years, said that some companies welcome the revenue and enter into contracts in which the government makes higher payments than otherwise available to firms receiving re imbursement for complying with surveillance orders.

These contractual payments, he said, could cover the cost of buying and installing new equipment, along with a reasonable profit. These voluntary agreements simplify the government’s access to surveillance, he said.

“It certainly lubricates the [surveillance] infrastructure,” Kouroupas said. He declined to say whether Global Crossing, which operated a fiber-optic network spanning several continents and was bought by Level 3 Communications in 2011, had such a contract. A spokesman for Level 3 Communications declined to comment.

In response to questions in 2012 from then-Rep. Edward J. Markey (D-Mass.), who was elected to the Senate in June, several telecommunications companies detailed their prices for surveillance services to law enforcement agencies under individual warrants and subpoenas. AT&T, for example, reported that it charges $325 to activate surveillance of an account and also a daily rate of $5 or $10, depending on the information gathered. For providing the numbers that have accessed cell towers, meanwhile, AT&T charged $75 per tower, the company said in a letter.

No payments have been previously disclosed for mass surveillance access to traffic flowing across a company’s infrastructure.

Lawyer Albert Gidari Jr., a partner at Perkins Coie who represents technology and telecommunications companies, said that surveillance efforts are expensive, requiring teams of attorneys to sift through requests and execute the ones deemed reasonable. Government agencies, meanwhile, sometimes balk at paying the full costs incurred by companies

“They lose a ton of money,” Gidari said. “And yet the government is still unsatisfied with it.”

The budget documents obtained by The Post list $65.96 million for BLARNEY, $94.74 million for FAIRVIEW, $46.04 million for STORMBREW and $9.41 million for OAKSTAR. It is unclear why the total of these four programs amounts to less than the overall budget of $278 million.

Among the possible costs covered by these amounts are “network and circuit leases, equipment hardware and software maintenance, secure network connectivity, and covert site leases,” the documents say. They also list in a separate line item $56.6 million in payments for “Foreign Partner Access,” although it is not clear whether these are for foreign companies, foreign governments or other foreign entities.

Some privacy advocates favor payments to companies when they comply with surveillance efforts because the costs can be a brake on overly broad requests by government officials. Invoices also can provide a paper trail to help expose the extent of spying.

But if the payments are too high, they may persuade companies to go beyond legal requirements in providing information, said Chris Soghoian, a technology expert with the American Civil Liberties Union who has studied government payments related to surveillance requests.

“I’m worried that the checks might grease the wheels a little bit,” he said.

By Craig Whitlock and Barton Gellman

August 29, 2013

The U.S. commando raid that killed Osama bin Laden was guided from space by a fleet of satellites, which aimed dozens of receivers over Pakistan to collect a torrent of electronic and signals intelligence as the mission unfolded, according to a top-secret U.S. intelligence document.

The National Security Agency also was able to penetrate guarded communications among al-Qaeda operatives by tracking calls from mobile phones identified by specific calling patterns, the document shows. Analysts from the CIA pinpointed the geographic location of one of the phones and linked it to the compound in Abbottabad, Pakistan, where other evidence suggested bin Laden was hiding.

The disclosures about the hunt for the elusive founder of al-Qaeda are contained in classified documents that detail the fiscal 2013 “black budget” for U.S. intelligence agencies, including the NSA and the CIA. The documents, provided to The Washington Post by former NSA contractor Edward Snowden, make only brief references to the bin Laden operation. But the mission is portrayed as a singular example of counterterrorism cooperation among the U.S. government’s numerous intelligence agencies.

Eight hours after the raid, according to the documents, a forensic intelligence laboratory run by the Defense Intelligence Agency in Afghanistan had analyzed DNA from bin Laden’s corpse and “provided a conclusive match” confirming his identity. The budget further reveals that satellites operated by the National Reconnaissance Office performed more than 387 “collects” of high-resolution and infrared images of the Abbottabad compound in the month before the raid — intelligence that was “critical to prepare for the mission and contributed to the decision to approve execution.”

Also playing a role in the search for bin Laden was an arm of the NSA known as the Tailored Access Operations group. Among other functions, the group specializes in surreptitiously installing spyware and tracking devices on targeted computers and mobile-phone networks.

Although the budget does not provide detail, it reports that Tailored Access Operations “implants” enabled the NSA to collect intelligence from mobile phones that were used by al-Qaeda operatives and other “persons of interest” in the hunt for bin Laden.

Separately, Tailored Access Operations capabilities were used in April 2011, the month before bin Laden was killed, when U.S. forces in Afghanistan relied on signals intelligence from implants to capture 40 low- and mid-level Taliban fighters and other insurgents in that country, according to the documents.

The new details about the raid fill out an already rich public account of how the U.S. government employed virtually every tool in its enormous surveillance apparatus to locate bin Laden. For more than a decade, bin Laden had stymied all efforts to find him by making certain he did not leave a direct electronic trail. He steadfastly avoided phones and e-mail, relying on face-to-face communications with a few couriers and middlemen.

In addition to the satellites, the government flew an advanced stealth drone, the RQ-170, over Pakistan to eavesdrop on electronic transmissions. The CIA also recruited a Pakistani doctor and other public health workers to try to obtain blood samples from people living in the Abbottabad compound as part of a vaccination program to determine whether the residents might be related to bin Laden.

That doctor was convicted by a Pakistani court in May 2012 of “conspiring against the state.” A senior judicial official on Thursday overturned the 33-year prison sentence for Shakil Afridi on technical grounds and ordered a retrial.

For all their technological prowess, U.S. spy agencies were unable to identify bin Laden with confidence inside the Abbottabad compound. By the time President Obama ordered a team of Navy SEALs to storm the site in May 2011, U.S. intelligence officials told the president that, according to their best guesses, the odds that bin Laden was present were 40 percent to 60 percent.

Even after bin Laden’s death, the U.S. government kept up its relentless high-tech campaign to unlock his secrets.

Budget documents show that intelligence agencies scraped together $2.5 million in emergency money in September 2011 to sift through a backlog of computer files and other evidence recovered from bin Laden’s hideout. The money went to buy 36 computer workstations and pay overtime to forensic examiners, linguists and “triage personnel” involved in the project.

By Barton Gellman and Ellen Nakashima

August 30, 2013

U.S. intelligence services carried out 231 offensive cyber-operations in 2011, the leading edge of a clandestine campaign that embraces the Internet as a theater of spying, sabotage and war, according to top-secret documents obtained by The Washington Post.

That disclosure, in a classified intelligence budget provided by NSA leaker Edward Snowden, provides new evidence that the Obama administration’s growing ranks of cyberwarriors infiltrate and disrupt foreign computer networks.

Additionally, under an extensive effort code-named GENIE, U.S. computer specialists break into foreign networks so that they can be put under surreptitious U.S. control. Budget documents say the $652 million project has placed “covert implants,” sophisticated malware transmitted from far away, in computers, routers and firewalls on tens of thousands of machines every year, with plans to expand those numbers into the millions.

The documents provided by Snowden and interviews with former U.S. officials describe a campaign of computer intrusions that is far broader and more aggressive than previously understood. The Obama administration treats all such cyber-operations as clandestine and declines to acknowledge them.

The scope and scale of offensive operations represent an evolution in policy, which in the past sought to preserve an international norm against acts of aggression in cyberspace, in part because U.S. economic and military power depend so heavily on computers.

“The policy debate has moved so that offensive options are more prominent now,” said former deputy defense secretary William J. Lynn III, who has not seen the budget document and was speaking generally. “I think there’s more of a case made now that offensive cyberoptions can be an important element in deterring certain adversaries.”

Of the 231 offensive operations conducted in 2011, the budget said, nearly three-quarters were against top-priority targets, which former officials say includes adversaries such as Iran, Russia, China and North Korea and activities such as nuclear proliferation. The document provided few other details about the operations.

Stuxnet, a computer worm reportedly developed by the United States and Israel that destroyed Iranian nuclear centrifuges in attacks in 2009 and 2010, is often cited as the most dramatic use of a cyberweapon. Experts said no other known cyberattacks carried out by the United States match the physical damage inflicted in that case.

U.S. agencies define offensive cyber-operations as activities intended “to manipulate, disrupt, deny, degrade, or destroy information resident in computers or computer networks, or the computers and networks themselves,” according to a presidential directive issued in October 2012.

Most offensive operations have immediate effects only on data or the proper functioning of an adversary’s machine: slowing its network connection, filling its screen with static or scrambling the results of basic calculations. Any of those could have powerful effects if they caused an adversary to botch the timing of an attack, lose control of a computer or miscalculate locations.

U.S. intelligence services are making routine use around the world of government-built malware that differs little in function from the “advanced persistent threats” that U.S. officials attribute to China. The principal difference, U.S. officials told The Post, is that China steals U.S. corporate secrets for financial gain.

“The Department of Defense does engage” in computer network exploitation, according to an e-mailed statement from an NSA spokesman, whose agency is part of the Defense Department. “The department does ***not*** engage in economic espionage in any domain, including cyber.”

‘Millions of implants’

The administration’s cyber-operations sometimes involve what one budget document calls “field operations” abroad, commonly with the help of CIA operatives or clandestine military forces, “to physically place hardware implants or software modifications.”

Much more often, an implant is coded entirely in software by an NSA group called Tailored Access Operations (TAO). As its name suggests, TAO builds attack tools that are custom-fitted to their targets.

The NSA unit’s software engineers would rather tap into networks than individual computers because there are usually many devices on each network. Tailored Access Operations has software templates to break into common brands and models of “routers, switches and firewalls from multiple product vendor lines,” according to one document describing its work.

The implants that TAO creates are intended to persist through software and equipment upgrades, to copy stored data, “harvest” communications and tunnel into other connected networks. This year TAO is working on implants that “can identify select voice conversations of interest within a target network and exfiltrate select cuts,” or excerpts, according to one budget document. In some cases, a single compromised device opens the door to hundreds or thousands of others.

Sometimes an implant’s purpose is to create a back door for future access. “You pry open the window somewhere and leave it so when you come back the owner doesn’t know it’s unlocked, but you can get back in when you want to,” said one intelligence official, who was speaking generally about the topic and was not privy to the budget. The official spoke on the condition of anonymity to discuss sensitive technology.

Under U.S. cyberdoctrine, these operations are known as “exploitation,” not “attack,” but they are essential precursors both to attack and defense.

By the end of this year, GENIE is projected to control at least 85,000 implants in strategically chosen machines around the world. That is quadruple the number — 21,252 — available in 2008, according to the U.S. intelligence budget.

The NSA appears to be planning a rapid expansion of those numbers, which were limited until recently by the need for human operators to take remote control of compromised machines. Even with a staff of 1,870 people, GENIE made full use of only 8,448 of the 68,975 machines with active implants in 2011.

For GENIE’s next phase, according to an authoritative reference document, the NSA has brought online an automated system, code-named TURBINE, that is capable of managing “potentially millions of implants” for intelligence gathering “and active attack.”

‘The ROC’

When it comes time to fight the cyberwar against the best of the NSA’s global competitors, the TAO calls in its elite operators, who work at the agency’s Fort Meade headquarters and in regional operations centers in Georgia, Texas, Colorado and Hawaii. The NSA’s organizational chart has the main office as S321. Nearly everyone calls it “the ROC,” pronounced “rock”: the Remote Operations Center.

“To the NSA as a whole, the ROC is where the hackers live,” said a former operator from another section who has worked closely with the exploitation teams. “It’s basically the one-stop shop for any kind of active operation that’s not defensive.”

Once the hackers find a hole in an adversary’s defense, “[t]argeted systems are compromised electronically, typically providing access to system functions as well as data. System logs and processes are modified to cloak the intrusion, facilitate future access, and accomplish other operational goals,” according to a 570-page budget blueprint for what the government calls its Consolidated Cryptologic Program, which includes the NSA.

Teams from the FBI, the CIA and U.S. Cyber Command work alongside the ROC, with overlapping missions and legal authorities. So do the operators from the NSA’s National Threat Operations Center, whose mission is focused primarily on cyber defense. That was Snowden’s job as a Booz Allen Hamilton contractor, and it required him to learn the NSA’s best hacking techniques.

According to one key document, the ROC teams give Cyber Command “specific target related technical and operational material (identification/recognition), tools and techniques that allow the employment of U.S. national and tactical specific computer network attack mechanisms.”

The intelligence community’s cybermissions include defense of military and other classified computer networks against foreign attack, a task that absorbs roughly one-third of a total cyber operations budget of $1.02 billion in fiscal 2013, according to the Cryptologic Program budget. The ROC’s breaking-and-entering mission, supported by the GENIE infrastructure, spends nearly twice as much: $651.7 million.

Most GENIE operations aim for “exploitation” of foreign systems, a term defined in the intelligence budget summary as “surreptitious virtual or physical access to create and sustain a presence inside targeted systems or facilities.” The document adds: “System logs and processes are modified to cloak the intrusion, facilitate future access, and accomplish other operational goals.”

The NSA designs most of its own implants, but it devoted $25.1 million this year to “additional covert purchases of software vulnerabilities” from private malware vendors, a growing gray-market industry based largely in Europe.

‘Most challenging targets’

The budget documents cast U.S. attacks as integral to cyber defense — describing them in some cases as “active defense.”

“If you’re neutralizing someone’s nuclear command and control, that’s a huge attack,” said one former defense official. The greater the physical effect, officials said, the less likely it is that an intrusion can remain hidden.

“The United States is moving toward the use of tools short of traditional weapons that are unattributable — that cannot be easily tied to the attacker — to convince an adversary to change their behavior at a strategic level,” said another former senior U.S. official, who also spoke on the condition of anonymity to discuss sensitive operations.

China and Russia are regarded as the most formidable cyber threats, and it is not always easy to tell who works for whom. China’s offensive operations are centered in the Technical Reconnaissance Bureau of the People’s Liberation Army, but U.S. intelligence has come to believe that those state-employed hackers by day return to work at night for personal profit, stealing valuable U.S. defense industry secrets and selling them.

Iran is a distant third in capability but is thought to be more strongly motivated to retaliate for Stuxnet with an operation that would not only steal information but erase it and attempt to damage U.S. hardware.

The “most challenging targets” to penetrate are the same in cyber-operations as for all other forms of data collection described in the intelligence budget: Iran, North Korea, China and Russia. GENIE and ROC operators place special focus on locating suspected terrorists “in Afghanistan, Pakistan, Yemen, Iraq, Somalia, and other extremist safe havens,” according to one list of priorities.

The growth of Tailored Access Operations at the NSA has been accompanied by a major expansion of the CIA’s Information Operations Center, or IOC.

The CIA unit employs hundreds of people at facilities in Northern Virginia and has become one of the CIA’s largest divisions. Its primary focus has shifted in recent years from counterterrorism to cybersecurity, according to the budget document.

The military’s cyber-operations, including U.S. Cyber Command, have drawn much of the public’s attention, but the IOC undertakes some of the most notable offensive operations, including the recruitment of several new intelligence sources, the document said.

Military cyber-operations personnel grouse that the actions they can take are constrained by the legal authorities that govern them. The presidential policy directive on cyber-operations issued in October made clear that military cyber-operations that result in the disruption or destruction or even manipulation of computers must be approved by the president. But the directive, the existence of which was first reported last fall by The Post and leaked in June by Snowden, largely does not apply to the intelligence community.

Given the “vast volumes of data” pulled in by the NSA, storage has become a pressing question. The NSA is nearing completion of a massive new data center in Utah. A second one will be built at Fort Meade “to keep pace with cyber processing demands,” the budget document said.

According to the document, a high-performance computing center in Utah will manage “storage, analysis, and intelligence production.” This will allow intelligence agencies “to evaluate similarities among intrusions that could indicate the presence of a coordinated cyber attack, whether from an organized criminal enterprise or a nation-state.”

By Carol Leonnig, Julie Tate and Barton Gellman

September 1, 2013

The U.S. government suspects that individuals with connections to al-Qaeda and other hostile groups have repeatedly sought to obtain jobs in the intelligence community, and it reinvestigates thousands of employees a year to reduce the threat that one of its own may be trying to compromise closely held secrets, according to a classified budget document.

The CIA found that among a subset of job seekers whose backgrounds raised questions, roughly one out of every five had “significant terrorist and/or hostile intelligence connections,” according to the document, which was provided to The Washington Post by former National Security Agency contractor Edward Snowden.

The groups cited most often were Hamas, Hezbollah, and al-Qaeda and its affiliates, but the nature of the connections was not described in the document.

So sharp is the fear of threats from within that last year the NSA planned to launch at least 4,000 probes of potentially suspicious or abnormal staff activity after scrutinizing trillions of employee keystrokes at work. The anomalous behavior that sent up red flags could include staffers downloading multiple documents or accessing classified databases they do not normally use for their work, said two people familiar with the software used to monitor employee activity.

This shrouded, multimillion-dollar hunt for insider threats has suffered from critical delays in recent years and uneven implementation across agencies, the budget records show. And the spy agencies’ detection systems never noticed that Snowden was copying highly classified documents from different parts of the NSA’s networks.

He subsequently fled to Hong Kong and then Moscow, where he remains after being granted temporary asylum.

Contractors like Snowden, an NSA spokeswoman said, were not included in the plans to reinvestigate 4,000 security clearances.

CIA officials said the number of applicants ultimately tied to terrorist networks or hostile foreign governments was “small” but declined to provide an exact number or the reasons the broader group of applicants initially raised concerns.

“Over the last several years, a small subset of CIA’s total job applicants were flagged due to various problems or issues,” one official said in response to questions. “During this period, one in five of that small subset were found to have significant connections to hostile intelligence services and or terrorist groups.” The official, like others interviewed for this article, spoke on the condition of anonymity to discuss classified material.

The intelligence community’s dramatic emphasis on insider threats came in the wake of disclosures by WikiLeaks in 2010. The anti-secrecy group received hundreds of thousands of military and diplomatic documents from Army Pfc. Bradley Manning, now known as Chelsea Manning.

Congress made security a top priority and in 2011 ordered Director of National Intelligence James R. Clapper Jr. to set up “an effective automated insider threat detection program” to guard against similar security failures. The program was supposed to flag possible abuses, identify double agents and prevent leaks.

The project was delayed several times because the intelligence community was preoccupied with handling the fallout from Manning’s leaks, the budget documents show.

Congress gave Clapper an October 2012 deadline to install the automation system and until October 2013 to have it fully operating. At the Obama administration’s request, the deadlines were each pushed back a year.

Steven Aftergood, a government secrecy expert at the Federation of American Scientists, which analyzes national security policy, said he suspects the agency may respond to a lot of “false positives” — alerts for activity that is actually innocent and work-related.

“If the 4,000 cases turn up only two or three actual threats, they need to adjust their detection threshold or they’ll be using a lot of resources for no purpose,” he said.

An intelligence official knowledgeable about insider-threat programs said government agencies may need to better calibrate their software and reassess their criteria for what constitutes a realistic or likely threat.

But he added, “Insider-threat detection is an important security tool that needs to be put in place.”

NSA spokeswoman Vanee Vines said the investigations are part of an effort to try to reduce risks, not investigate known threats.

“In FY 2013, NSA planned to initiate 4,000 re-investigations on civilian employees to reduce the potential of an insider compromise of sensitive information and missions,” Vines said in a statement. “Periodic re-investigations are conducted as one due-diligence component of our multifaceted insider threat program.”

President Obama issued a national insider-threat policy in November 2012 that defines the problem as any risk that insiders will use their access to government secrets, knowingly or unknowingly, in a way that hurts U.S. security. The damage can come through “espionage, terrorism, [or] unauthorized disclosure of national security information, or through the loss or degradation of departmental resources or capabilities.”

The policy puts leakers of classified information on par with terrorists and double agents, an equivalency that critics of government secrecy find worrisome.

“It’s disturbing, because they are not the same,” Aftergood said. “There are such things as a good leak. Some classified things should be public.The official policy does not admit that distinction.”

The newly obtained budget records also reveal what some consider a new security risk in the making. The NSA is creating a massive new database, code-named WILDSAGE, to facilitate the use of sensitive intelligence. The system “provides a mechanism for cybersecurity centers to share signatures at the SECRET classification level,” the budget document said.

The NSA had not fully implemented WILDSAGE, according to the budget document. But the intelligence community also reported that it “made significant progress in FY 2011 in increasing collaboration” across cybersecurity centers and in creating architecture to share important information on risks.”

Such shared databases give government employees information they may need to track and disrupt terrorist or cyber plots, officials said.

The investigation of a 2009 plot by al-Qaeda’s affiliate in Yemen to bring down a civilian aircraft over Detroit found that government agencies failed to share information that could have flagged the would-be bomber and put him on a terrorist watch list.

A report by the Senate Intelligence Committee identified 14 failure points, including that the CIA did not disseminate some its reporting, which allowed bomber Umar Farouk Abdulmutallab to board a U.S.-bound flight from Amsterdam.

But the new NSA system has stoked fears about insiders’ access after Manning downloaded the entire contents of a similarly shared State Department database.

“It’s a valid concern,” said one intelligence official. “But the other side of this . . . is the need to share information.”

In the case of Manning’s leak, the State Department maintained a shared Net Centric diplomacy database where all secret-level cables were deposited and available to all intelligence agencies to access and search. Manning reviewed the database while creating analytic intelligence products in Iraq.

Manning was convicted last month of leaking classified information and subsequently sentenced to 35 years in prison.

A prosecutor in Manning’s case said the U.S. government ultimately has to entrust its secrets to its employees — even low-level ones.

“Military intelligence professionals go to work every day and use the information available to them to do their jobs,” said prosecutor Capt. Joe Morrow. “We can take all the mitigation steps in the world, but the bottom line is that there is no step we can take as a nation, as a military, that’s going to stop the determined insider.”

By Greg Miller, Craig Whitlock and Barton Gellman

September 2, 2013

A member of Pakistan’s border security force stands guard near Tahir. (Photo by Tim Craig/The Washington Post)

The $52.6 billion U.S. intelligence arsenal is aimed mainly at unambiguous adversaries, including al-Qaeda, North Korea and Iran. But top-secret budget documents reveal an equally intense focus on one purported ally: Pakistan.

No other nation draws as much scrutiny across so many categories of national security concern.

A 178-page summary of the U.S. intelligence community’s “black budget” shows that the United States has ramped up its surveillance of Pakistan’s nuclear arms, cites previously undisclosed concerns about biological and chemical sites there, and details efforts to assess the loyalties of counter terrorism sources recruited by the CIA.

Pakistan appears at the top of charts listing critical U.S. intelligence gaps. It is named as a target of newly formed analytic cells. And fears about the security of its nuclear program are so pervasive that a budget section on containing the spread of illicit weapons divides the world into two categories: Pakistan and everybody else.

The disclosures — based on documents provided to The Washington Post by former intelligence contractor Edward Snowden — expose broad new levels of U.S.distrust in an already unsteady security partnership with Pakistan, a politically unstable country that faces rising Islamist militancy. They also reveal a more expansive effort to gather intelligence on Pakistan than U.S. officials have disclosed.

The United States has delivered nearly $26 billion in aid to Pakistan over the past 12 years, aimed at stabilizing the country and ensuring its cooperation in counterterrorism efforts. But with Osama bin Laden dead and al-Qaeda degraded, U.S. spy agencies appear to be shifting their attention to dangers that have emerged beyond the patch of Pakistani territory patrolled by CIA drones.

“If the Americans are expanding their surveillance capabilities, it can only mean one thing,” said Husain Haqqani, who until 2011 served as Pakistan’s ambassador to the United States. “The mistrust now exceeds the trust.”

Beyond the budget files, other classified documents provided to The Post expose fresh allegations of systemic human rights abuses in Pakistan. U.S. spy agencies reported that high-ranking Pakistani military and intelligence officials had been aware of — and possibly ordered — an extensive campaign of extrajudicial killings targeting militants and other adversaries.

Public disclosure of those reports, based on communications intercepts from 2010 to 2012 and other intelligence, could have forced the Obama administration to sever aid to the Pakistani armed forces because of a U.S. law that prohibits military assistance to human rights abusers. But the documents indicate that administration officials decided not to press the issue, in order to preserve an already frayed relationship with the Pakistanis.

In a statement, a spokeswoman for the National Security Council said the United States is “committed to a long-term partnership with Pakistan, and we remain fully engaged in building a relationship that is based on mutual interests and mutual respect.”

“We have an ongoing strategic dialogue that addresses in a realistic fashion many of the key issues between us, from border management to counterterrorism, from nuclear security to promoting trade and investment,” said the spokeswoman, Caitlin Hayden. “The United States and Pakistan share a strategic interest in combating the challenging security issues in Pakistan, and we continue to work closely with Pakistan’s professional and dedicated security forces to do so.”

The Post agreed to withhold some details from the budget documents after consultations with U.S. officials, who expressed concern about jeopardizing ongoing operations and sources.

A spokesman for the Pakistani Embassy in Washington did not respond to a request for comment.

Critical ‘intelligence gaps’

Stark assessments of Pakistan contained in the budget files seem at odds with the signals that U.S. officials have conveyed in public, partly to avoid fanning Pakistani suspicions that the United States is laying contingency plans to swoop in and seize control of the country’s nuclear complex.

When Director of National Intelligence James R. Clapper Jr. was asked during congressional testimony last year whether Pakistan had appropriate safeguards for its nuclear program, he replied, “I’m reasonably confident they do.” Facing a similar question this year, Clapper declined to discuss the matter in open session.

But the classified budget overview he signed and submitted for fiscal 2013 warned that “knowledge of the security of Pakistan’s nuclear weapons and associated material encompassed one of the most critical set of . . . intelligence gaps.” Those blind spots were especially worrisome, the document said, “given the political instability, terrorist threat and expanding inventory [of nuclear weapons] in that country.”

The budget documents do not break down expenditures by country or estimate how much the U.S. government spends to spy on Pakistan. But the nation is at the center of two categories — counterterrorism and counter-proliferation — that dominate the black budget.

In their proposal for fiscal 2013, which ends Sept. 30, U.S. spy agencies sought $16.6 billion to fight al-Qaeda and other terrorist groups and asked for $6.86 billion to counter the spread of nuclear, biological and chemical weapons. Together, the two categories accounted for nearly half of the U.S. intelligence community’s budget request for this year.

Detailed spreadsheets contain dozens of line items that correspond to operations in Pakistan. The CIA, for example, was scheduled to spend $2.6 billion on “covert action” programs around the world. Among the most expensive, according to current and former U.S. intelligence officials, is the armed drone campaign against al-Qaeda fighters and other militants in Pakistan’s tribal belt.

U.S. intelligence analysts “produced hundreds of detailed and timely reports on shipments and pending deliveries of suspect cargoes” to Pakistan, Syria and Iran. Multiple U.S. agencies exploited the massive American security presence in Afghanistan — including a string of CIA bases and National Security Agency listening posts along the border mainly focused on militants — for broader intelligence on Pakistan.

Anxiety over nuclear program

After years of diplomatic conflict, significant sources of tension between the United States and Pakistan have begun to subside.

The pace of CIA drone strikes has plunged, and two years have passed since U.S. leaders infuriated Islamabad by ordering the secret raid inside Pakistani territory that killed bin Laden.

Although Pakistani anger has abated, Haqqani said the fallout from the raid had broader consequences than widely understood.

“The discovery of bin Laden [in Pakistan] made the Americans think that the Pakistani state’s ability to know what happens within the country is a lot less than had been assumed,” said Haqqani, who is an international-relations professor at Boston University.

That realization may have ratcheted up a long-standing source of concern: Pakistan’s ability to safeguard its nuclear materials and components.

U.S. intelligence agencies are focused on two particularly worrisome scenarios: the possibility that Pakistan’s nuclear facilities might come under attack by Islamist militants, as its army headquarters in Rawalpindi did in 2009, and even greater concern that Islamist militants might have penetrated the ranks of Pakistan’s military or intelligence services, putting them in a position to launch an insider attack or smuggle out nuclear material.

Pakistan has dozens of laboratories and production and storage sites scattered across the country. After developing warheads with highly enriched uranium, it has more recently tried to do the same with more-powerful and compact plutonium. The country is estimated to have as many as 120 nuclear weapons, and the budget documents indicate that U.S. intelligence agencies suspect that Pakistan is adding to that stockpile.

Little is known about how it moves materials among its facilities, an area that experts have cited as a potential vulnerability.

“Nobody knows how they truly do it,” said Feroz Khan, a retired Pakistani military officer and director of arms control who lectures at the U.S. Naval Postgraduate School in Monterey, Calif. “Vehicles move in a stealthy manner and move with security. But it’s not clear whether the cores are moved to the warheads or the warheads are moved to the core locations.”

Concerns persist that extremists could seize components of the stockpile or trigger a war with neighboring India. Pakistan also has a track record of exporting nuclear technology to countries that are on Washington’s blacklist.

Pakistan has accepted some security training from the CIA, but U.S. export restrictions and Pakistani suspicions have prevented the two countries from sharing the most sophisticated technology for safeguarding nuclear components.

U.S. anxiety over Pakistan’s nuclear program appears to be driven more by uncertainty about how it is run than specific intelligence indicating that its systems are vulnerable, according to the budget documents.

A lengthy section on counter-proliferation starts with a single goal: “Make Quantitative and Qualitative Progress against Pakistan Nuclear Gaps.” A table indicates that U.S. spy agencies have identified at least six areas in which their understanding of Pakistan’s weapons programs is deficient.

U.S. agencies reported gaining valuable information through “extensive efforts to increase understanding of the transfer and storage of the associated materials.”

The budget describes the creation of a Pakistan WMD Analysis Cell to track movements of nuclear materials. Agencies, including the CIA and the Defense Department, were able “to develop and deploy a new compartmented collection capability” that delivered a “more comprehensive understanding of strategic weapons security in Pakistan.”

Even so, “the number of gaps associated with Pakistani nuclear security remains the same,” the document said, and “the questions associated with this intractable target are more complex.”

The budget documents indicate that U.S. intelligence agencies are also focused on the security of the nuclear program in India, Pakistan’s arch-rival.

Other fields under scrutiny

U.S. surveillance of Pakistan extends far beyond its nuclear program. There are several references in the black budget to expanding U.S. scrutiny of chemical and biological laboratories. The country is not thought to be running a rogue chemical or biological weapons program, but U.S. intelligence officials fear that Islamists could seize materials from government- run laboratories.

Even American interdiction operations targeting other countries have stumbled into connections with Pakistan. In one case, a U.S. effort to block an Iranian shipment through a Turkish port “proved to be even more successful when aluminum powder destined for Pakistan was also discovered and detained,” according to the documents. Aluminum powder can be used to increase the power of explosives.

The budget documents don’t disclose CIA payments to its Pakistani counterpart, the Inter-Services Intelligence directorate, or ISI, which former officials said has totaled tens of millions of dollars. The documents do show that the CIA has developed sophisticated means of assessing the loyalties of informants who have helped the agency find al-Qaeda leaders in Pakistan’s tribal region.

Those measures, which The Post has agreed not to disclose, have allowed the CIA to “gain confidence in each asset’s authenticity, reliability and freedom from hostile control.”

Extrajudicial killings

Other classified documents given to The Post by Snowden reveal that U.S. spy agencies for years reported that senior Pakistani military and intelligence leaders were orchestrating a wave of extrajudicial killings of terrorism suspects and other militants.

In July 2011, an assessment of communications intercepts and other intelligence by the NSA concluded that the Pakistani military and intelligence services had continued over the preceding 16 months a pattern of lethally targeting perceived enemies without trial or due process. The killings, according to the NSA, occurred “with the knowledge, if not consent, of senior officers.”

The NSA cited two senior Pakistani officials who “apparently ordered some of the killings or were at least aware of them,” read a summary of the top-secret NSA report, titled “Pakistan/Human Rights: Extrajudicial Killings Conducted With Consent of Senior Intelligence Officials.”

The report summary did not provide an estimate of how many people had been killed or their identities. But it generally described the targets as people whom the Pakistani security forces viewed as “undeniably linked to terrorist activity” or responsible for attacks on Pakistan’s armed forces.

The killings “seemed to serve the purpose of dispensing what the military considered swift justice,” the intelligence assessment stated. Pakistani authorities “were conscious of not arousing suspicions. The number of victims at a given time tended to be very small. Furthermore, the military took care to make the deaths seem to occur in the course of counterinsurgency operations, from natural causes, or as the result of personal vendettas.”

Although Pakistan has been engaged for years in open warfare with Taliban factions and other domestic insurgents, the NSA placed the extrajudicial killings in a much darker category. Pakistani police forces “were reluctant to carry out the killings,” the report said.

The NSA compiled its report shortly after the public exposure of other alleged Pakistani atrocities.

In June 2010, the Human Rights Commission of Pakistan charged that Pakistani forces had carried out more than 280 summary executions during an offensive against Taliban fighters and other militants, mostly in the Swat Valley. Five months later, a video surfaced on the Internet showing Pakistani soldiers executing six blindfolded men with their hands tied behind their backs.

An international outcry over the latter incident prompted the Obama administration to withhold aid — but only to a handful of low-level Pakistani army units thought to have been involved in such incidents.

At the time, Pakistani officials dismissed the video and other reports of summary executions as Taliban propaganda, but they later reversed course and launched an internal investigation. Pakistan’s military leaders insisted publicly that they had zero tolerance for such incidents.

Human rights abuses

It was not the first time that U.S. officials sought to keep evidence of Pakistani human rights abuses out of the public eye.

A classified diplomatic cable, sent from the U.S. Embassy in Islamabad to officials in Washington in September 2009, also raised concern about the extrajudicial killings of militants by Pakistani army units. But the cable — originally released in 2010 by the anti-secrecy group WikiLeaks — advised against public disclosure of the incidents, saying it was more important to maintain support for the Pakistani armed forces.

U.S. intelligence officials have kept quiet about other signs of human rights abuses by the Pakistani military, even though their classified reporting on the subject underscores persistent concerns.

In September 2011, the summary of a top-secret report from a Defense Intelligence Agency task force cited the “systemic practice” of unlawful killings by Pakistani security forces in the tribal regions of western Pakistan.

Pakistan had recently passed a law allowing the military to detain insurgents indefinitely and make it easier to convict them in civilian courts. But the DIA concluded that because extrajudicial killings were “condoned by senior officials” in Pakistan’s security establishment, the new law was unlikely to significantly reduce the number of deaths.

Other U.S. intelligence documents indicate that Pakistani officials weren’t targeting just suspected insurgents.

In May 2012, U.S. intelligence agencies discovered evidence of Pakistani officers plotting to “eliminate” a prominent human rights activist, Asma Jahangir, according to the summary of a top-secret DIA report. Jahangir had been a leading public critic of the ISI for years.

The DIA report did not identify which officers were plotting to kill Jahangir, but it said the plan “included either tasking militants to kill her in India or tasking militants or criminals to kill her in Pakistan.”

The U.S. agency said it did not know whether the ISI had given approval for the plot to proceed. Although the report speculated that the ISI was motivated to kill Jahangir “to quiet public criticism of the military,” the DIA noted that such a plot “would result in international and domestic backlash as ISI is already under significant criticism for intimidation and extra- judicial killings.”

News of the alleged plot became public a few weeks later when Jahangir gave a round of interviews to journalists, revealing that she had learned that Pakistani intelligence officials had marked her for death. The plot was never carried out.

By Barton Gellman and Craig Whitlock

September 3, 2013

Al-Qaeda’s leadership has assigned cells of engineers to find ways to shoot down, jam or remotely hijack U.S. drones, hoping to exploit the technological vulnerabilities of a weapons system that has inflicted huge losses upon the terrorist network, according to top-secret U.S. intelligence documents.

Although there is no evidence that al-Qaeda has forced a drone crash or interfered with flight operations, U.S. intelligence officials have closely tracked the group’s persistent efforts to develop a counterdrone strategy since 2010, the documents show.

Al-Qaeda commanders are hoping a technological breakthrough can curb the U.S. drone campaign, which has killed an estimated 3,000 people over the past decade. The airstrikes have forced al-Qaeda operatives and other militants to take extreme measures to limit their movements in Pakistan, Afghanistan, Yemen, Somalia and other places. But the drone attacks have also taken a heavy toll on civilians, generating a bitter popular backlash against U.S. policies toward those countries.

Details of al-Qaeda’s attempts to fight back against the drone campaign are contained in a classified intelligence report provided to The Washington Post by Edward Snowden, the fugitive former National Security Agency contractor. The top-secret report, titled “Threats to Unmanned Aerial Vehicles,” is a summary of dozens of intelligence assessments posted by U.S. spy agencies since 2006.

U.S. intelligence analysts noted in their assessments that information about drone operational systems is available in the public realm. But The Post is withholding some detailed portions of the classified material that could shed light on specific weaknesses of certain aircraft.

Under President Obama and his predecessor, George W. Bush, drones have revolutionized warfare and become a pillar of the U.S. government’s counterterrorism strategy, enabling the CIA and the military to track down enemies in some of the remotest parts of the planet. Drone strikes have left al-Qaeda’s core leadership in Pakistan scrambling to survive.

U.S. spy agencies have concluded that al-Qaeda faces “substantial” challenges in devising an effective way to attack drones, according to the top-secret report disclosed by Snowden. Still, U.S. officials and aviation experts acknowledge that unmanned aircraft have a weak spot: the satellite links and remote controls that enable pilots to fly them from thousands of miles away.

In July 2010, a U.S. spy agency intercepted electronic communications indicating that senior al-Qaeda leaders had distributed a “strategy guide” to operatives around the world advising them how “to anticipate and defeat” unmanned aircraft. The Defense Intelligence Agency (DIA) reported that al-Qaeda was sponsoring simultaneous research projects to develop jammers to interfere with GPS signals and infrared tags that drone operators rely on to pinpoint missile targets.

Other projects in the works included the development of observation balloons and small radio-controlled aircraft, or hobby planes, which insurgents apparently saw as having potential for monitoring the flight patterns of U.S. drones, according to the report.

Al-Qaeda cell leaders in the tribal areas of northwestern Pakistan were “determining the practical application of technologies being developed for battlefield applications,” analysts from the DIA wrote. The analysts added that they believed al-Qaeda “cell leadership is tracking the progress of each project and can redirect components from one project to another.”

The technological vulnerabilities of drones are no secret. The U.S. Air Force Scientific Advisory Board issued an unclassified report two years ago warning that “increasingly capable adversaries” in countries such as Afghanistan could threaten drone operations by inventing inexpensive countermeasures.

The board said insurgents might try to use “lasers and dazzlers” to render a drone ineffective by blinding its cameras and sensors. It also predicted that insurgents might use rudimentary acoustic receivers to detect drones and “simple jammer techniques” to interfere with navigation and communications.

Researchers have since proved that the threat is not just theoretical. Last year, a research team from the University of Texas at Austin demonstrated to the Department of Homeland Security that it was possible to commandeer a small civilian drone by “spoofing” its GPS signal with a ground transmitter and charting a different navigational course.

Trained engineers

Al-Qaeda has a long history of attracting trained engineers and others with a scientific background. Khalid Sheik Mohammed, the self-proclaimed architect of the Sept. 11, 2001, attacks, holds a mechanical-engineering degree and is such an inveterate tinkerer that the CIA allowed him to fiddle around with new designs for a vacuum cleaner after he was captured a decade ago.

In 2010, the CIA noted in a secret report that al-Qaeda was placing special emphasis on the recruitment of technicians and that “the skills most in demand” included expertise in drones and missile technology. In July of that year, Atiyah Abd al-Rahman, an al-Qaeda operations chief, told a jihadist Web site that the network did not need “ordinary fighters” and that it was looking instead for “specialist staff” to join the organization.

That same year, authorities in Turkey said they arrested an al-Qaeda member who was developing plans to shoot down small NATO surveillance drones in Afghanistan. The suspect, a 23-year-old mathematics student, was using software to conduct ballistics research for drone attacks, according to Turkish officials.

Al-Qaeda leaders have become increasingly open about their anti-drone efforts. In March, a new English-language online jihadist magazine called Azan published a story titled “The Drone Chain.” The article derided drone armaments as “evil missiles designed by the devils of the world” but reassured readers that jihadists had been working on “various technologies” to hack, manipulate and destroy unmanned aircraft.

At the same time, the magazine indicated that those efforts needed a boost, and it issued an emergency plea for scientific help: “Any opinions, thoughts, ideas and practical implementations to defeat this drone technology must be communicated to us as early as possible because these would aid greatly … against the crusader- zionist enemy.”

In the absence of a high-tech silver bullet, al-Qaeda affiliates around the world have taken to sharing hard-earned lessons about the importance of basic defensive measures.

Islamist extremists in North Africa this year distributed a photocopied tipsheet with 22 recommendations for avoiding drone strikes. Among the suggestions are several ideas for camouflage as well as dubious advice on using radio or microwave transmitters to “confuse the frequencies used to control the drone.”

The Associated Press in February found a copy of the tipsheet in Mali, left behind by Islamist fighters fleeing the city of Timbuktu. It was written by a jihadist in Yemen two years earlier and has circulated among al-Qaeda franchises since then.

‘GPS jamming capability’

In January 2011, U.S. intelligence agencies detected an unusual electronic signal emanating from near Miran Shah, a jihadist haven in North Waziristan, Pakistan. The DIA called the signal “the first observed test of a new terrorist GPS jamming capability.”

The test apparently did not pose a threat to military GPS frequencies or encrypted communications links. In addition, whoever was beaming the mysterious signal mistakenly thought that jamming ground-based GPS receivers would interfere with drones’ ability to aim missiles or munitions at fixed targets, according to the DIA report.

Despite such missteps, al-Qaeda has been undeterred. In a separate 2011 report, the DIA stated that affiliates in Miran Shah and the Pakistani city of Karachi were pursuing other “R&D projects,” including one effort to shoot down drones with portable shoulder-fired missiles, known as manpads.

Army intelligence analysts uncovered similar projects, including attempts to develop laser detectors that could give warning whenever a U.S. Predator drone was about to fire a laser-guided Hellfire missile, according to a summary of a classified Army report.

In 2011, the DIA concluded that an “al-Qaeda-affiliated research and development cell currently lacks the technical knowledge to successfully integrate and deploy a counterdrone strike system.” DIA analysts added, however, that if al-Qaeda engineers were to “overcome these substantial design challenges, we believe such a system probably would be highly disruptive for U.S. operations in Afghanistan and Pakistan.”

The Air Force and CIA rely heavily on Predator and Reaper drones to hunt for al-Qaeda targets and other insurgents in several countries. Both aircraft can stay aloft for more than 20 hours to conduct surveillance missions and can be armed with Hellfire missiles.

The drones are flown by remote control via satellite data links, usually by pilots and sensor operators stationed thousands of miles away at bases in the United States. Those satellite links are encrypted, which makes the connections extremely difficult to hack.

It is only slightly less of a challenge for al-Qaeda fighters to spot a high-flying drone with the naked eye. Predators and Reapers loiter at altitudes above 20,000 feet, and their powerful cameras focus on objects several miles over the horizon, so their presence is hard to detect.

The satellite links, however, are the Achilles’ heel of drone operations. “Lost link” incidents — triggered when a satellite moves out of range or a drone drops a signal — are relatively common. The connections are usually reestablished within seconds or minutes. The aircraft are programmed to fly in a loop pattern or return to their launching spot during prolonged disruptions.

On several occasions, however, lost links have led to crashes. In September, an Air Force Predator slammed into mountainous terrain along the Iraq-Turkey border after the satellite data links were lost and the drone crew could no longer communicate with the aircraft.

In December 2011, a stealth U.S. spy drone operated by the CIA crashed in Iranian territory. Iran said it downed the advanced RQ-170 drone in an “electronic ambush.” U.S. officials said they did not believe that the drone had been hacked or jammed. They said a technical malfunction was probably to blame.

Although the navigational satellite links are encrypted, other drone transmissions are sometimes left unprotected.

In 2009, the U.S. military discovered that Iraqi insurgents had hacked into video feeds from Predator and Shadow drones using off-the-shelf software. The drones had been transmitting full-motion video to U.S. troops on the ground, but the Air Force had not encrypted those data links, leaving them vulnerable.

Air Force officials acknowledged the flaw and said they would work to encrypt all video feeds from its fleet of Predator drones by 2014. In their classified assessments, U.S. intelligence agencies sought to play down the insurgents’ hacking handiwork. Although analysts were concerned about the interceptions of the video feeds, they said there was no sign that insurgents had been able to seize control of the drone itself.

“While the ability of insurgent forces to view unencrypted or to break into encrypted data streams has been a concern for some time, indications to date are that insurgents have not been able to wrest [drone] control from its mission control ground station,” a 2010 report concluded.

The report went on to suggest that allowing insurgents to intercept video feeds might actually have “a deterrent effect” by demonstrating the extent to which U.S. forces were able to watch their movements.

Growing unease

Still, summaries of the classified reports indicate a growing unease among U.S. agencies about al-Qaeda’s determination to find a way to neutralize drones.

“Al-Qaida Engineers in Pakistan Continue Development of Laser-Warning Systems in Effort To Counter UAV Strikes,” read the headline of one report in 2011, using the military acronym for unmanned aerial vehicles.

Beyond the threat that al-Qaeda might figure out how to hack or shoot down a drone, however, U.S. spy agencies worried that their drone campaign was becoming increasingly vulnerable to public opposition.

Intelligence analysts took careful note of al-Qaeda’s efforts to portray drone strikes as cowardly or immoral, beginning in January 2011 with a report titled “Al-Qa’ida Explores Manipulating Public Opinion to Curb CT Pressure.”

Analysts also questioned whether they were losing the rhetorical battle in the media, the courts and even among “citizens with legitimate social agendas.” One 2010 report predicted that drone operations “could be brought under increased scrutiny, perceived to be illegitimate, openly resisted or undermined.”

In response, intelligence agencies floated their own ideas to influence public perceptions. One unclassified report said the phrase “drone strike” should never be uttered, calling it “a loaded term.”

“Drones connote mindless automatons with no capability for independent thought or action,” the report said. “Strikes connote a first attack, which leaves the victim unable to respond. Other phrases employed to evoke an emotional response include ‘Kill List,’ ‘Hit Squads,’ ‘Robot Warfare,’ or ‘Aerial Assassins.’ “

Instead, the report advised referring to “lethal UAV operations.” It also suggested “elevating the conversation” to more-abstract issues, such as the “Inherent Right of Self-Defense” and “Pre-emptive and Preventive Military Action.”

Greg Miller contributed to this report.

By Ellen Nakashima

September 7, 2013

Sen. Ron Wyden (D-Ore.) has been one of the most vocal critics of the NSA’s domestic spying program. (Photo by Melina Mara/The Washington Post)

The Obama administration secretly won permission from a surveillance court in 2011 to reverse restrictions on the National Security Agency’s use of intercepted phone calls and e-mails, permitting the agency to search deliberately for Americans’ communications in its massive databases, according to interviews with government officials and recently declassified material.

In addition, the court extended the length of time that the NSA is allowed to retain intercepted U.S. communications from five years to six years — and more under special circumstances, according to the documents, which include a recently released 2011 opinion by U.S. District Judge John D. Bates, then chief judge of the Foreign Intelligence Surveillance Court.

What had not been previously acknowledged is that the court in 2008 imposed an explicit ban — at the government’s request — on those kinds of searches, that officials in 2011 got the court to lift the bar and that the search authority has been used.

Together the permission to search and to keep data longer expanded the NSA’s authority in significant ways without public debate or any specific authority from Congress. The administration’s assurances rely on legalistic definitions of the term “target” that can be at odds with ordinary English usage. The enlarged authority is part of a fundamental shift in the government’s approach to surveillance: collecting first, and protecting Americans’ privacy later.

“The government says, ‘We’re not targeting U.S. persons,’ “ said Gregory T. Nojeim, senior counsel at the Center for Democracy and Technology. “But then they never say, ‘We turn around and deliberately search for Americans’ records in what we took from the wire.’ That, to me, is not so different from targeting Americans at the outset.”

The court decision allowed the NSA “to query the vast majority” of its e-mail and phone call databases using the e-mail addresses and phone numbers of Americans and legal residents without a warrant, according to Bates’s opinion.

The queries must be “reasonably likely to yield foreign intelligence information.” And the results are subject to the NSA’s privacy rules.

The court in 2008 imposed a wholesale ban on such searches at the government’s request, said Alex Joel, civil liberties protection officer at the Office of the Director of National Intelligence (ODNI). The government included this restriction “to remain consistent with NSA policies and procedures that NSA applied to other authorized collection activities,” he said.

But in 2011, to more rapidly and effectively identify relevant foreign intelligence communications, “we did ask the court” to lift the ban, ODNI general counsel Robert S. Litt said in an interview. “We wanted to be able to do it,” he said, referring to the searching of Americans’ communications without a warrant.

Joel gave hypothetical examples of why the authority was needed, such as when the NSA learns of a rapidly developing terrorist plot and suspects that a U.S. person may be a conspirator. Searching for communications to, from or about that person can help assess that person’s involvement and whether he is in touch with terrorists who are surveillance targets, he said. Officials would not say how many searches have been conducted.

The court’s expansion of authority went largely unnoticed when the opinion was released, but it formed the basis for cryptic warnings last year by a pair of Democratic senators, Ron Wyden (Ore.) and Mark Udall (Colo.), that the administration had a “back-door search loophole” that enabled the NSA to scour intercepted communications for those of Americans. They introduced legislation to require a warrant, but they were barred by classification rules from disclosing the court’s authorization or whether the NSA was already conducting such searches.

“The [surveillance] Court documents declassified recently show that in late 2011 the court authorized the NSA to conduct warrantless searches of individual Americans’ communications using an authority intended to target only foreigners,” Wyden said in a statement to The Washington Post. “Our intelligence agencies need the authority to target the communications of foreigners, but for government agencies to deliberately read the e-mails or listen to the phone calls of individual Americans, the Constitution requires a warrant.”

Senior administration officials disagree. “If we’re validly targeting foreigners and we happen to collect communications of Americans, we don’t have to close our eyes to that,” Litt said. “I’m not aware of other situations where once we have lawfully collected information, we have to go back and get a warrant to look at the information we’ve already collected.”

The searches take place under a surveillance program Congress authorized in 2008 under Section 702 of the Foreign Intelligence Surveillance Act. Under that law, the target must be a foreigner “reasonably believed” to be outside the United States, and the court must approve the targeting procedures in an order good for one year.

But — and this was the nub of the criticism — a warrant for each target would no longer be required. That means that communications with Americans could be picked up without a court first determining that there is probable cause that the people they were talking to were terrorists, spies or “foreign powers.”

That is why it is important to require a warrant before searching for Americans’ data, Udall said. “Our founders laid out a roadmap where Americans’ privacy rights are protected before their communications are seized or searched — not after the fact,” he said in a statement to The Post.

Another change approved by Bates allows the agency to keep the e-mails of or concerning Americans for up to six years, with an extension possible for foreign intelligence or counterintelligence purposes. Because the retention period begins “from the expiration date” of the one-year surveillance period, the court effectively added up to one year of shelf life for the e-mails collected at the beginning of the period.

Joel said that the change was intended to standardize retention periods across the agencies and that the more generous standard was “already in use” by another agency.

The NSA intercepts more than 250 million Internet communications each year under Section 702. Ninety-one percent are from U.S. Internet companies such as Google and Yahoo. The rest come from “upstream” companies that route Internet traffic to, from and within the United States. The expanded search authority applies only to the downstream collection.

By Ellen Nakashima, Julie Tate, Carol Leonnig

September 10, 2013

The National Security Agency for almost three years searched a massive database of Americans’ phone call records attempting to identify potential terrorists in violation of court-approved privacy rules, and the problem went unfixed because no one at the agency had a full technical understanding of how its system worked, according to new documents and senior government officials.

Moreover, it was Justice Department officials who discovered the problem and reported it to the court that oversees surveillance programs, the documents show, undermining assertions by the NSA that self-reporting is part of its culture.

The improper activity went on from May 2006 to January 2009, according to a March 2009 opinion by Judge Reggie B. Walton, who serves on the Foreign Intelligence Surveillance Court.

It was one of more than a dozen documents declassified and released Tuesday in response to lawsuits by civil liberties groups and at the direction of President Obama in the wake of the June disclosure by former NSA contractor Edward J. Snowden of the massive phone records collection.

“The documents released today are a testament to the government’s strong commitment to detecting, correcting and reporting mistakes that occur in implementing technologically complex intelligence collection activities, and to continually improving its oversight and compliance processes,” said James R. Clapper, the director of national intelligence.

A strong rebuke of the NSA by the court comes less than a month after the Office of the Director of National Intelligence released a highly critical FISA court opinion that took the agency to task for its operation of a separate surveillance program. Taken together, the documents released by the office over the past month paint a troubling picture of an agency that has sought and won far-reaching surveillance powers to run complex domestic data collection without anyone having full technical understanding of the efforts, and that has repeatedly misrepresented the programs’ scope to its court overseer.

Such revelations call into question the effectiveness of an oversight program that depends on accurate disclosure by the NSA to a court that acts in secret and says it lacks the resources to verify independently the agency’s assertions.

“It has finally come to light that the FISC’s authorizations of this vast collection program have been premised on a flawed depiction of how the NSA uses” the phone data, Walton wrote.

“This misperception by the FISC existed from the inception of its authorized collection in May 2006, buttressed by repeated inaccurate statements made in the government’s submissions,” he continued.

Privacy procedures “have been so frequently and systemically violated that it can fairly be said that this critical element of the overall [phone records] regime has never fully functioned effectively,” he said.

The “bulk records” program began without any court or congressional approval shortly after the Sept. 11, 2001, attacks but was put under court supervision in May 2006 when American phone companies balked at providing the data solely at the request of the executive branch.

Under the program, the NSA receives daily transfers of all customer records from the nation’s phone companies. Those records include numbers called, the calls’ time and duration, but not the content of conversations.

Beginning in late January 2009, Justice Department officials began notifying the court of problems, in particular that the NSA had been running an automated “alert list” on selected phone numbers without meeting the court-required standard of “reasonable and articulable suspicion” that those numbers were tied to terrorists.

Justice Department officials notified the court that the NSA had been searching the business records “in a manner directly contrary” to the court’s orders “and directly contrary to the sworn attestations of several Executive Branch officials,” Walton wrote in a Jan. 29, 2009, order.

NSA Director Keith B. Alexander suggested to the court that the violations stemmed from a belief by NSA personnel that not all the databases were covered by the same privacy rules, Walton wrote in his March opinion.

“That interpretation of the court’s orders strains credulity,” Walton said.

Walton also suggested that the NSA’s Office of General Counsel deliberately chose to approve the use of phone numbers that did not meet the court standards because such procedures were in keeping with other NSA collection activities.

In March 2009, the court took the unusual step of ordering the government to seek approval to query the database on a case-by-case basis “except where necessary to protect against an imminent threat to human life.”

Walton also expressed consternation at the NSA’s inaccurate description of its use of the database, saying it slowed efforts to fix the errors. “The government’s failure to ensure that responsible officials adequately understood the NSA’s alert list process, and to accurately report its implementation to the court, has prevented for more than two years, both the government and the FISC from taking steps to remedy daily violations,” Walton wrote.

After the problems came to light, the NSA and the Justice Department in February 2009 began a full review of the program. The review identified additional violations that needed fixing, the documents show.

For instance, the review found that “over time” more than 200 analysts from the CIA, the FBI and the National Counterterrorism Center had access to “query results” from the database that did not properly mask the identities of U.S. persons.

In September 2009, the Justice Department reported that query results were shared with NSA analysts who were not trained in how to handle the data. Walton noted that the problems came after the government’s “submission of a report intended to assure the Court that the NSA had corrected and addressed the issues.”

In November 2009, Walton also expressed concern that the NSA had searched phone numbers long after the numbers had been found to be irrelevant and said he “remained concerned” that the NSA did not meet the required standard for using the numbers, exposing information about Americans who were not the subject of FBI investigations. Walton’s March ruling also expressed skepticism about the program’s utility, noting the government cited only three FBI preliminary investigations opened as a result of tips gleaned from the database. “The time has come for the government to describe to the Court how, based on the information collected and analyzed during that time, the value of the program to the nation’s security justifies the continued collected and retention of massive quantities of US person information,” he said.

The documents were released in response to lawsuits from the Electronic Frontier Foundation and the ACLU. The EFF first sought disclosure two years ago when Congress was debating reauthorization of the statute that the government used to justify the phone collection.

“At that time, the government withheld every word, claiming that grave harm to national security would result if the court’s opinions were released,” said David Sobel, the foundation’s senior counsel. “Now that the agency has been forced to release them, we can see that the real reason for secrecy was to conceal the fact that surveillance under the Patriot Act was far broader, and less focused, than the public and Congress had been led to believe.”

By Ellen Nakashima

September 17, 2013

A federal surveillance court on Tuesday released a declassified opinion upholding the constitutionality of the National Security Agency’s sweeping collection of billions of Americans’ phone records for counterterrorism purposes.

The gathering of “all call detail records” from phone companies is justified as long as the government can show that it is relevant to an authorized investigation into known — and, significantly — unknown terrorists who may be in the United States, the Aug. 29 opinion states.

Moreover, the government need only show that there are “reasonable grounds to believe” the records will be relevant to the investigation, a lower burden than required in ordinary criminal investigations. That is justified because the goal is to prevent a terrorist attack, not solve a crime that has already taken place, the court said, affirming the government’s position.

Taken together, the argument is a bold assertion of authority that critics say is not justified by the Foreign Intelligence Surveillance Act (FISA) or the Constitution. Some elements of the court’s reasoning had been discussed in an earlier Justice Department white paper released by the government, but the concept of the “unknown” terrorist and the argument for the lower burden had not been explicitly linked to the program.

The 29-page opinion signed by Claire V. Eagan, a judge on the secretive Foreign Intelligence Surveillance Court (FISC), is the first to be released that addresses the constitutionality of the NSA’s “bulk records” collection of phone data. It is an attempt to address growing criticism about the broad surveillance since its existence was disclosed in June in a document leaked to the Guardian, a British newspaper, by former NSA contractor Edward Snowden. The program was authorized by the court in 2006 under Section 215 of the Patriot Act, which amended FISA, but that was not known until June. In the program, the NSA gathers records of phone calls, their time and duration, but not subscriber names or call content.

The opinion was released at Eagan’s will, Justice Department officials said, not at the request of the government or in response to lawsuits from civil liberties groups. Eagan, appointed to the federal bench in Oklahoma by President George W. Bush, is a fairly new member of the FISC.

A senior Justice Department official said that it is not a substitute for the release of other lengthier, significant opinions relating to the surveillance program.

Privacy advocates reacted with dismay upon reading the ruling.

“This isn’t a judicial opinion in the conventional sense,” said Jameel Jaffer, American Civil Liberties Union deputy legal director. “It’s a document that appears to have been cobbled together over the last few weeks in an effort to justify a decision that was made seven years ago. I don’t know of any precedent for that, and it raises a lot of questions.”

Jaffer added that the opinion was “completely unpersuasive” as a defense of the call-records program. The constitutional analysis fails to mention the landmark United States v. Jones privacy case decided by the Supreme Court last year, which suggested a warrant was necessary for long-term tracking of GPS data, he said. And Eagan’s analysis of the statute overemphasizes some terms while ignoring others, he said.

“On the whole, the opinion only confirms the folly of entrusting privacy rights to a court that hears argument only from the government,” said Jaffer, referring to the fact that there is no adversary in the classified proceedings.

Eagan’s ruling endorsed the government’s argument that the broad collection was necessary to find unknown terrorist operatives who may be in the United States “because it is impossible to know where in the data the connections to international terrorist organizations will be found.”

The senior Justice Department official said the “unknown” language was important. “If you know who all the people were, you would just ask [the phone companies] for those numbers,” he said. “So that’s why you need all those numbers … The bottom line is this: You have to have this larger body of data to find the needles in the haystack.”

The ruling also reaffirmed the government’s contention, upheld by the Supreme Court in 1979, that Americans have no reasonable expectation of privacy in records of their calls held by phone companies, and a warrant to collect them is not required. A warrant would be required to wiretap the calls.

Eagan asserted that to date, no company has challenged the legality of an order. She also stated that Congress ratified the program when it reauthorized the statute without change in 2010 and 2011, because it had access to information on the statute’s application to the phone program.

Kurt Opsahl, senior staff attorney at Electronic Frontier Foundation, disagreed. “The outrage of many Congress members” upon hearing about the program’s scope “shows this is not true. We should not have the legal basis of the surveillance state resting on a judicially created legal fiction.”

By Ellen Nakashima

September 26, 2013

The Senate Select Intelligence Committee holds a confirmation hearing to hear the testimony of nominee James Clapper to fill the vacancy of director of National Intelligence (DNI), on Capitol Hill Tuesday July 20, 2010. (Photo by Melina Mara/The Washington Post)

U.S. officials declined to directly answer lawmakers’ questions on Thursday about the full scope of the National Security Agency’s collection of Americans’ data, including whether it has ever sought to acquire large volumes of cellphone location information or other records.

NSA Director Keith Alexander dodged questions by a senior member of the Senate Intelligence Committee about whether the agency has ever tried to augment its broad collection of virtually all Americans’ phone-call records by gathering data that would indicate the callers’ locations. He noted that intelligence officials had given a classified answer to the question.

“If you’re responding to my question by not answering it because you think it’s a classified matter, that is your right,” Sen. Ron Wyden (D-Ore.) told Alexander at a hearing about the government’s intelligence-gathering programs.

“We will continue to explore that,” Wyden said, “because I believe this is something that the American people have a right to know — whether the NSA has ever collected or made plans to collect cell site information.”

Questions by Wyden and Sen. Mark Udall (D-Colo.) suggested that the agency has at least sought if not won permission to expand its domestic collection activities beyond what has been publicly acknowledged.

Their pointed questioning of officials stood in contrast to mostly sympathetic lines of querying from other panel members, who made clear that they support the phone-logs program as a vital counterterrorism tool and accused the media of inaccurate and misleading coverage. Committee Chairman Dianne Feinstein (D-Calif.) said such coverage “has led to an unfortunate … skepticism and distrust” of the intelligence community.

The existence of the phone-records collection, and its authorization by a federal surveillance court, was revealed in June when the Guardian newspaper in Britain published a court order to a phone company leaked by former NSA contractor Edward Snowden. Leaks regarding other NSA programs ensued and have put the administration on the defensive, forcing it, as Director of National Intelligence James R. Clapper Jr. said Thursday, to “lower … the threshold” for disclosure.

Alexander and other officials stressed that the call database contains only phone numbers and the time and duration of calls, but no conversation content, subscriber names or cell site location. But Alexander acknowledged that the NSA is interested in compiling the largest national database possible, and that there is no limit to the number of records that can be gathered. The storehouse holds billions of records, former officials have told The Washington Post.

“Is it the goal of the NSA to collect the phone records of all Americans?” Udall asked.

“I believe it is in the nation’s best interests to put all the phone records into a lockbox that we could search when the nation needs to do it, yes,” Alexander said.

The government has claimed the authority to gather the data under Section 215 of the USA Patriot Act, also known as the “business records” provision of the Foreign Intelligence Surveillance Act. The FISA court in 2006 agreed that the government could use that statute to order phone companies to hand over “all call detail records” daily to the NSA.

Asked by Udall if that statute gave NSA the authority to collect other data — such as utility bills — Deputy Attorney General James M. Cole offered a qualified answer. “It’s given them the authority to collect other bulk records if they can show that it is necessary to find something relevant to a foreign intelligence investigation of particular types … It’s not just all bulk records. But it’s also not no business records. It’s all dependent on the purpose.”

At the hearing’s outset, Clapper asserted that the leaks “have been extremely damaging” to national security. “These disclosures are threatening our ability to collect intelligence and keep our country safe,” he said. “There’s no way to erase or make up for the damage that has already been done. We anticipate more as we continue our assessment.”

Feinstein said she and Saxby Chambliss (Ga.), the ranking Republican on the panel, have begun drafting legislation that would reform the nation’s surveillance system, including a measure that would require Senate confirmation of the NSA director. Their bill also would limit access to phone records collected by the agency, and reduce the length of time the records can be held.

Wyden, Udall and other lawmakers have introduced reform legislation that would, among other things, end the phone records collection, while allowing for a more limited program.

On Thursday, Wyden accused U.S. officials of not being more forthcoming about intelligence-collection programs.

“The leadership of your agencies built an intelligence-collection system that repeatedly deceived the American people,” he said. “Time and time again, the American people were told one thing about domestic surveillance in public forums while government agencies did something else in private.”

By Barton Gellman

October 4, 2013

On Nov. 1, 2007, the National Security Agency hosted a talk by Roger Dingledine, principal designer of one of the world’s leading Internet privacy tools. It was a wary encounter, akin to mutual intelligence gathering, between a spy agency and a man who built tools to ward off electronic surveillance.

According to a top-secret NSA summary of the meeting, Dingledine told the assembled NSA staff that his service, called Tor, offered anonymity to people who needed it badly — to keep business secrets, protect their identities from oppressive political regimes or conduct research without revealing themselves. In the minds of NSA officials, Tor was offering protection to terrorists and other intelligence targets.

As he spoke to the NSA, Dingledine said in an interview Friday, he suspected the agency was attempting to break into Tor, which is used by millions of people around the world to shield their identities. Documents provided to The Washington Post by former agency contractor Edward Snowden show that he was right.

Beginning at least a year before Dingledine’s visit, the NSA has mounted increasingly successful attacks to unmask the identities and locations of users of Tor. In some cases, the agency has succeeded in blocking access to the anonymous network, diverting Tor users to insecure channels. In others, it has been able to “stain” anonymous traffic as it enters the Tor network, enabling the NSA to identify users as it exits.

Tor works by encrypting traffic repeatedly as it flows across a global network of servers, mostly run by volunteers. The traffic, which can include e-mails, information from a Web site and almost anything else on the Internet, is supposed to arrive at its destination with no identifying information about its origin or the path it took.

The Snowden documents, including a detailed PowerPoint presentation, suggest that the NSA cannot see directly inside Tor’s anonymous network but that it has repeatedly uncloaked users by circumventing Tor’s protections. The documents also illustrate the power of the NSA to at least partially penetrate what have long been considered the most secure corners of the Internet.

The U.S. Naval Research Laboratory first developed Tor more than a decade ago as a tool to allow anonymous communications and Web browsing. It was embraced by privacy advocates, including the Electronic Frontier Foundation, and continues to receive substantial federal funding. Tor is now maintained by Dingledine’s nonprofit group, the Tor Project.

The State Department trains political activists worldwide on how to use Tor to protect communications from the intelligence services of repressive governments. But the anonymity service also has become popular with criminals — especially dealers of illicit drugs, military-grade weapons and child pornography — and terrorists seeking to evade tracking by Western intelligence services.

One of the documents provided by Snowden said an NSA technique code-named EGOTISTICALGIRAFFE had succeeded in unmasking 24 Tor users in a single weekend. The same operation allowed the NSA to discover the identity of a key propagandist for al-Qaeda in the Arabian Peninsula, as the group’s offshoot in Yemen is known, after he posted information and instructions on the group’s Web site.

NSA anti-anonymity techniques are now also being used by law enforcement agencies. In August, civilian security researchers detected an FBI operation against an alleged child pornography ring that used a Tor-based Web server called Freedom Hosting. The FBI mounted a cyberattack to unmask the location and owner of that anonymous server, using precisely the technique described as EGOTISTICALGIRAFFE.

The Washington Post is not releasing certain details from the documents, including the name of the al-Qaeda operative. Documents about the NSA’s attempts to penetrate Tor were also shared with the British newspaper the Guardian, which published a report on the effort Friday.

In a statement, Director of National Intelligence James R. Clapper Jr., who oversees the NSA and other intelligence agencies, said that the intelligence community “seeks to understand” tools that facilitate anonymous communication. He added that it does so because of the “undeniable fact that these are the tools our adversaries use to communicate and coordinate attacks against the United States and our allies.”

The intelligence community “is only interested in communication related to valid foreign intelligence and counterintelligence purposes,” Clapper said.

There is no evidence that the NSA is capable of unmasking Tor traffic routinely on a global scale. But for almost seven years, it has been trying.

Since 2006, according to a 49-page research paper titled simply “Tor,” the agency has worked on several methods that, if successful, would allow the NSA to uncloak anonymous traffic on a “wide scale” — effectively by watching communications as they enter and exit the Tor system, rather than trying to follow them inside. One type of attack, for example, would identify users by minute differences in the clock times on their computers.

Dingledine expressed no surprise that the NSA has tried to defeat efforts at anonymity. In the interview, he said the weaknesses in Tor described in the PowerPoint presentation likely could be exploited only against a relatively small number of individual users. That, he said, is reassuring.

“If those documents actually represent what they can do, they are not as big an adversary as I thought,” he said.

The Tor Browser Bundle, available for free at www.torproject.org, was downloaded 40 million times last year. Until a recent security upgrade to the Firefox browser, which is incorporated in the bundle, the NSA could trick the browser into leaking the real Internet address of a targeted user. One slide described these tactics as “pretty much guaranteed to succeed.”

Mozilla, the nonprofit organization that develops Firefox, declined to comment.

One document provided by Snowden included an internal exchange among NSA hackers in which one of them said the agency’s Remote Operations Center was capable of targeting anyone who visited an al-Qaeda Web site using Tor.

“The ROC currently [operates] against certain extremist web forums at the moment,” the employee wrote. “I am under the impression that they can serve up an exploit” — hacker jargon for malicious code — “to pretty much anyone that visits the particular web forum, though.”

“Like any tool, [Tor] can be used for something good, and it can be used for something bad,” said Garth Bruen, a Boston-based investigator who studies Internet crimes. “It’s all about how people are using it, and criminals have been using it to great advantage. . . . It’s a nightmare.”

An FBI agent told an Irish court last month that Freedom Hosting, unmasked with NSA-devised techniques, was among the largest purveyors of child pornography in the world, according to news reports. Silk Road, an online marketplace some called “the eBay of illicit substances,” also relied on Tor — and was targeted by the FBI. Federal officials arrested the alleged founder and shut down the site Wednesday.

Privacy advocates, however, say Tor is valuable and should be protected even if it is sometimes used by criminals. “Tor is networking technology,” said Christopher Soghoian, an American Civil Liberties Union technology expert. “It is no different from a postage stamp or a highway. Good people use highways, and bad people use highways.”

The NSA documents portray a years-long program to defeat what the agency called “The Tor Problem,” with the agency repeatedly updating its tactics as Tor’s developers made changes to the network.

The NSA also altered tactics as Mozilla introduced new versions of Firefox. In anticipation of a new release of Firefox, one agency official wrote in January that a new exploit was under development: “I’m confident we can have it ready when they release something new, or very soon after :).”

In late 2006, when the NSA prepared a working paper on methods to defeat Tor, the anonymous network had an estimated 200,000 users and 1,000 servers. Among the secret NSA documents were lists of hundreds of servers the agency believed to be “nodes” on that network.

Along with EGOTISTICALGIRAFFE, the agency’s cover names for Tor attacks have included MJOLNIR, MOTHMONSTER and EGOTISTICALGOAT. A similar program at Britain’s Government Communications Headquarters, the NSA’s close counterpart, was called STUNT WORM.

One NSA PowerPoint presentation provided by Snowden is titled “Peeling Back the Layers of TOR with EGOTISTICALGIRAFFE.”

The agency began identifying browsers that were using Tor by noting how the encryption program reset what’s called the BuildID — a 14-digit code representing the exact date and time when that version of Firefox was released. On versions using Tor, the BuildID is reset to “0.” That feature made it hard to distinguish one Tor user from another, but it also allowed the NSA to pick out Tor-enabled browsers from among all others in use at any given moment.

“It’s easy!” a slide describing the technique said.

Mozilla issued a patch to Firefox that would protect newer versions of the browser against such an attack, though the NSA documents make clear that research into new exploits remains active.

One PowerPoint slide sums up a multistep method for learning the identity and location of Tor users and implanting NSA code in the browser. It ends with a final bullet point saying, “Win!”

Ashkan Soltani and Julie Tate contributed to this report.

Additional Content:

NSA report on the Tor encrypted network

By Ellen Nakashima

October 6, 2013

Mark Young, former executive director for the Directorate for Plans and Policy at U.S. Cyber Command. One military official oversees both the NSA and Cyber Command. (Photo by Jeffrey MacMillan/The Washington Post)

During suspected Iranian cyberattacks on the Web sites of commercial banks last year, Gen. Keith B. Alexander, who simultaneously heads the country’s largest electronic spy agency and the military’s Cyber Command, proposed a simple solution: Shut off the attacks at their source.

“We had the expertise and could have done something about it,” said one U.S. official, who like others interviewed for this store spoke on the condition of anonymity to describe sensitive discussions. “We’re sitting on their networks overseas. Why don’t we just turn it off?”

But the proposal to send a simple computer “reset” command to the attacking servers was ultimately rejected by National Security Council officials this year because the attacks were not causing enough harm to warrant an offensive response.

The episode shows the willingness — some say eagerness — of Alexander to use his authority to conduct offensive actions to fend off attacks against the private sector. If a similar proposal were on the table today, it would be the new cyber-teams that Alexander is creating to defend the nation that probably would do the job.

As he builds out U.S. Cyber Command at Fort Meade, Md., and other installations to a fighting force of 6,000 over the next three years, there are fresh questions about the wisdom of so much power residing in one“dual-hatted” official.

The debate has taken on greater significance in the wake of disclosures by former National Security Agency contractor Edward Snowden about the sweeping scope of the agency’s domestic surveillance to thwart terrorist attacks and gain foreign intelligence.

“The mashing together of the NSA and Cyber Command has blurred the lines between a military command and a national spy agency,” said Peter Singer, a Brookings Institution expert on evolving modes of warfare.

Alexander disagrees. “It’s one network,” he said in a recent interview. “We all operate on the same network. You create more problems by trying to separate them and have two people fighting over who’s in charge [of both missions] than putting it all together. I think our nation benefits from that.”

He said that other countries “do similar things.” Britain’s GCHQ, the equivalent of the NSA, is able to conduct espionage and computer-network attacks.

With Alexander expected to retire next year after eight years as the longest-serving NSA director, his successor will face the questions.

Administration officials acknowledge that there are concerns with what they call the dual-hat assignment, which includes regularly briefing the president on counterterrorism. “It is an unusual arrangement,” said one senior administration official, speaking on the condition of anonymity. “I’m sure that debate will resurface when General Alexander leaves that position — whether that’s the right mixture to have.”

When it comes to cyberattacks, much of it is done by the NSA’s Tailored Access Operations unit, officials say. Many of the operators are uniformed military personnel who spy on overseas networks for national intelligence priorities, including targets such as Iran, Russia and North Korea, among other countries.

According to interviews with U.S. officials, these same personnel, who operate under intelligence legal authorities, may switch to a military authority when they are ordered to conduct a computer attack under an execute order by the president and the defense secretary. The process is documented. “You can be doing intelligence-gathering one second and then pull the trigger on an offensive op the next,” a former intelligence official said.

“We’re allowing the same military commander to tell us how bad the problem is and propose and implement suggestions to fix it,” said Jason Healey, director of the Atlantic Council’s Cyber Statecraft Initiative. “The concentration of power at Fort Meade risks further militarizing cyberspace.”

Alexander and his supporters have long argued that the NSA and Cyber Command must be closely connected because so much of what Cyber Command was set up to do — from defend to attack — depends on intelligence that comes from the NSA. To re-create the NSA’s capabilities is neither feasible nor desirable, officials say.

“It would be folly to separate them,” a former senior intelligence official said.

Such coordination, Alexander said, is key to the United States’ cyberattack prowess.

“Cyber offense requires a deep, persistent and pervasive presence on adversary networks in order to precisely deliver effects,” Alexander told the House Armed Services Committee in March in a written response to questions.

Effects could include things such as stopping a denial-of-service attack, rerouting a jihadi Web site and disrupting an industrial control or military weapons system, actions that generally require presidential permission.

The NSA and the CIA carried out 231 offensive cyber-operations in 2011, according to classified budget documents obtained by Snowden, The Washington Post reported last month. Their exact nature was not detailed in the documents, but most offensive operations have immediate effects only on data or the proper functioning of an adversary’s machine: slowing its network connection or scrambling the results of basic calculations, which the NSA would do for foreign intelligence purposes. The NSA, officials say, does not conduct a “computer network attack.” That is done by the military under an execute order.

But it can help develop the weapons used in a covert operation, as was reportedly the case with Stuxnet, a computer worm used in an extensive U.S.-Israeli campaign to delay Iran’s nuclear program by disrupting centrifuges in an uranium enrichment plant.

The House and Senate Armed Services committees have raised concerns with the dual-hat arrangement, as well as with a proposal to elevate Cyber Command to a unified command on a par with Central Command and Pacific Command. The cyber unit falls under Strategic Command in Omaha, though it leaves considerable leeway in policy and operational matters to Alexander, officials say. The move to a unified command, once looked upon as a sure bet, has been slowed because of concerns on Capitol Hill and the Snowden disclosures.

Alexander is moving ahead with plans to build out the fledgling command. By the end of 2016, there will be 133 teams, with about 2,000 new personnel in place. About 4,000 will be coming from the services. Alexander said it will take a couple of years to train the teams.

Of the 133 teams, 27 will support the combatant organizations such as Central Command, Pacific Command and European Command. Key questions remain as to how well the teams will meet combatant commanders’ needs and priorities.

Perhaps the most controversial, though, are the 13 national mission teams, whose job it will be to defend the nation in the event of attacks on critical systems in the private sector.

Some in the military say it is not the Defense Department’s job to stave off attacks against U.S. private-sector systems. “My role is to fight wars — not protect the private sector,” was how one military official put it.

But to settle the argument, last year, then-Defense Secretary Leon E. Panetta signed a memo stating that it was the Defense Department’s role to defend the country against cyberattacks.

The national mission forces are “absolutely critical,” Alexander said in the recent interview. “It’s the future. There are only a couple of ways that people can hurt this country: terrorism and cyber. We’re doing really good against terrorism. Our cyber-force we have to build up.”

By Ellen Nakashima

October 10, 2013

In this slide from a National Security Agency presentation on “Google Cloud Exploitation,” a sketch shows where the “Public Internet” meets the internal “Google Cloud” where user data resides. Two engineers with close ties to Google exploded in profanity when they saw the drawing.

A drop in Americans’ trust in the government is making the difficult task of public-private cooperation against cyber-threats even more difficult.

And that has officials such as Gen. Keith B. Alexander, director of the National Security Agency, scrambling to shore up confidence in his agency, whose image has taken a beating in the wake of leaks about its surveillance programs by former NSA contractor Edward Snowden.

At public hearings and in speeches, Alexander, who also heads the U.S. Cyber Command, is warning that cyberattacks on such critical and technology-dependent industries as energy, finance and transportation can be prevented only if those industries work with the government. But companies are wary of partnering with an agency that has been revealed to be conducting far-reaching domestic data collection in the name of thwarting terrorism.

“Industry is critical to resolving our problems” in cybersecurity, Alexander said at the Billington Cybersecurity Summit last month at the National Press Club.

Toward that end, he said, Congress needs to pass “cyber-legislation” to encourage private companies to share data on cyber-threats. A bipartisan bill the House passed in April would provide immunity from civil lawsuits or criminal prosecution to companies that give the Department of Homeland Security network data that might contain evidence of such threats. DHS would pass the data on to relevant agencies, such as the NSA.

Alexander said the protected data would be limited to technical material indicating vulnerabilities in systems and hackers’ tracks. “We’re not talking about sharing our private information,” he assured the summit audience.

But there is wide recognition within and outside the government that the Snowden leaks, which began in June, have created a deficit of trust. “It was tough enough to [pass the bill] when the waters were calm,” Michael V. Hayden, Alexander’s predecessor as NSA director, said last week at The Washington Post’s Cyber Summit. “Now [proponents are] trying to do it in whitewater rapids, and it’s not going to happen.”

Even before the Snowden revelations, the White House threatened to veto the bill on grounds it lacked adequate safeguards for Americans’ privacy, among other things. Now, experts say, it is increasingly unlikely that the House version will emerge from the Senate.

“I don’t think anybody thinks it’s realistic to put the NSA in the middle of domestic cybersecurity at this point,” said Michelle Richardson, legislative counsel at the American Civil Liberties Union.

One of the most consequential Snowden leaks was a classified court order whose publication forced the government to acknowledge that the NSA had obtained secret court permission in 2006 to gather the phone records of virtually all Americans — billions of calls — to search for clues to terrorist plots. Another leak detailed how nine Internet companies — including Yahoo, Google and Microsoft — cooperated, under court order, with the NSA to collect e-mails and other digital data from lawful foreign targets.

The scale of the data collection stunned Americans, said Paul Tiao, former senior counselor to the FBI director who is a partner at Hunton & Williams. “I don’t think a lot of people thought they had all that information. The NSA has been trying to overcome that ever since.”

Companies have long been sensitive to the implications of sharing data with the government, fearing harm to their reputations and potential lawsuits for privacy and other violations, Tiao said. “The Snowden disclosures have made companies more careful about what they might share with the government because they know that the public is that much more concerned about it.” And restoring confidence, experts say, depends on how meaningful the government’s surveillance reforms are.

Here’s the reality: The bits and bytes that zip through computer networks each day serve many purposes. Some are innocent messages and images sent by friends, co-workers or marketers. But others contain foreign intelligence useful to the government such as indications of terrorist activity. And some may contain malicious software or signals from foreign hackers designed to disrupt U.S. industrial networks or steal secrets from companies.

The NSA is supposed to ignore the first category of messages as it intercepts the second. But Snowden’s leaks have cast doubt on how well the agency protects the privacy of Americans’ communications it “inadvertently” collects while eavesdropping on foreign targets.

And that is impeding its efforts to be more successful at seeing the third category: the data that indicate a cyberattack is brewing. Whether the agency should have that capability is the crux of that debate.

Administration officials are optimistic about another initiative: a voluntary framework — which President Obama ordered — to help companies improve their network security. The National Institute of Standards and Technology is on track to issue a preliminary framework this week that includes controls such as a company identifying its computer systems so it knows what it has. It’s a far cry from the administration’s initial goal of mandatory industry standards. But, White House cybersecurity coordinator Michael Daniel said, “it will provide companies a good way of thinking about their cyber-risk and give them a very structured way to mitigate that risk.”

The outline should make it easier to elevate the cyber-risk discussion to the boardroom, on a par with financial risk issues, Daniel said. “This structured process will really make it much easier for companies that haven’t tackled this problem as fully as they would like to.”

Rick Dakin, chief executive of Coalfire Systems in Louisville, Colo., predicted that industry would get “tremendous mileage” out of the framework. “If you’re an exec today and you know there’s a downside if you’re not complying with that industry standard, wouldn’t you want that standard defined?” he said. “It’s really to industry’s benefit to know.”

But Paul Rosenzweig, a former Department of Homeland Security deputy assistant secretary for policy who consults on cybersecurity, called the framework a “modestly useful compilation of existing standards” that “breaks no new ground.” Anybody who operates high-risk systems, he said, already knows most of what is in it. And some industry officials say the framework is a step toward regulation that will stifle innovation.

Mark Weatherford, former deputy undersecretary for cybersecurity at the Department of Homeland Security, said it’s up to industry to raise its game — because the government’s role is limited by restrictions on the sharing of classified information and other policy constraints. “The government is not going to come riding in on a white horse to rescue you when you have a security incident,” he said at a speech in August. “The government is simply unable, at least today, to provide timely and actionable information when you really need it.”

Daniel said the next step is to create incentives for industry to adopt the best practices suggested in the framework — immunity from lawsuits, for example, or contracting advantages.

The bottom line, experts say, is that cyber-legislation looks unlikely without trust in government — unless there’s a major cyberattack on the United States.

“That,” Tiao said, “could change things.”

By Ellen Nakashima and Carol D. Leonnig

October 12, 2013

In the recent stream of disclosures about National Security Agency surveillance programs, one document, sources say, has been conspicuously absent: the original — and still classified — judicial interpretation that held that the bulk collection of Americans’ data was lawful.

That document, written by Colleen Kollar-Kotelly, then chief judge of the Foreign Intelligence Surveillance Court (FISC), provided the legal foundation for the NSA amassing a database of all Americans’ phone records, say current and former officials who have read it.

Now, more officials are saying that Americans should be able to read and understand how an important precedent was established under the 2001 USA Patriot Act, which was passed after the Sept. 11, 2001, attacks.

“The original legal interpretation that said that the Patriot Act could be used to collect Americans’ records in bulk should never have been kept secret and should be declassified and released,” Sen. Ron Wyden (D-Ore) said in a statement to The Washington Post. “This collection has been ongoing for years and the public should be able to compare the legal interpretation under which it was originally authorized with more recent documents.”

Kollar-Kotelly told associates this summer that she wanted her legal argument out, according to two people familiar with what she said. Several members of the intelligence court want more transparency about the court’s role to dispel what they consider a misperception that the court acted as a rubber stamp for the administration’s top-secret spying programs. The court meets in secret to hear applications for domestic surveillance and its opinions generally are not made public.

Kollar-Kotelly, who is no longer on the FISC, declined to comment, as did the current chief judge, Reggie B. Walton.

Kollar-Kotelly’s interpretation served as the legal basis for a court authorization in May 2006 that allowed the NSA to gather on a daily basis the phone records of tens of millions of Americans, sources say. Her analysis, more than 80 pages long, was “painstakingly thorough,” said one person who read it. The date of the analysis has not been disclosed.

The broad outlines of the judge’s argument have been revealed via a Justice Department “white paper.” And last month, the administration released a 29-page opinion written in August that defended the program by asserting essentially that as long as some Americans’ phone records might be “relevant” to a terrorism investigation, the government may collect them all. But that opinion, current and former officials said, is not a substitute for Kollar-Kotelly’s original interpretation.

“If the question is, ‘How was this program authorized and what type of legal analysis first took place?’ the 2013 opinion is just not responsive,” said one former senior Obama administration official. “It’s hard for me to imagine, with all that’s already out there, that highly classified intelligence material would be so deeply entwined in the legal analysis in that original interpretation that they couldn’t somehow release it.”

Asked to comment, Justice Department spokesman Brian Fallon said, “Already so far, the department has declassified a white paper explaining the legal basis for the [phone records] program, and the court has publicly posted an opinion upholding its lawfulness.” He declined to confirm or deny the existence of Kollar-Kotelly’s original interpretation.

According to a draft NSA inspector general report obtained by The Post from former NSA contractor Edward Snowden, Kollar-Kotelly in 2004 approved the NSA’s collection of Americans’ bulk e-mail metadata records (to-from addresses, dates, time stamps, etc.) using a Patriot Act provision similar to one the court used in 2006 to authorize the phone records program. The argument made for the e-mail metadata program, analysts say, may have been used to justify the phone records collection.

The phone program logs metadata such as numbers called and the time and duration of calls but not names or content.

Fallon said that “additional materials related to the [phone] program are under review for possible declassification and release” in response to lawsuits by the American Civil Liberties Union and the Electronic Frontier Foundation. The original interpretation is one of a number of classified court documents and executive branch rulings that some lawmakers say should be declassified.and released

Government lawyers have told the ACLU that they are withholding at least two significant FISC opinions — one from 2008 and one from 2010 — relating to the Patriot Act’s Section 215, or “business records” provision.

Fallon declined to comment.

The Justice Department also is reviewing a 2006 court opinion related to the Section 215 provision to determine whether it can be released, said Alex Abdo, an ACLU staff lawyer. (A senior department official told The Post that no 2006 Kollar-Kotelly opinion is based on that provision.)

Kenneth Wainstein, a former senior Justice Department official now on the Public Interest Declassification Board, an advisory committee established by Congress, agreed that more disclosure is wise. “Especially when it comes to legal decisions about big programs,” he said at a recent panel discussion sponsored by the New York University School of Law, “we can talk about them in a sanitized way without disclosing sources and methods.”

Senate Judiciary Committee Chairman Patrick J. Leahy (D-Vt.) said in a statement to The Post that he is “particularly interested” in seeing the administration declassify and release “any additional legal analysis” related to the phone records program. “That,” he said, “is exactly the sort of transparency we need in order to have a full and open debate about whether this program is legal and appropriate or needed.”

By Barton Gellman and Ashkan Soltani

October 14, 2013

The National Security Agency is harvesting hundreds of millions of contact lists from personal e-mail and instant messaging accounts around the world, many of them belonging to Americans, according to senior intelligence officials and top-secret documents provided by former NSA contractor Edward Snowden.

The collection program, which has not been disclosed before, intercepts e-mail address books and “buddy lists” from instant messaging services as they move across global data links. Online services often transmit those contacts when a user logs on, composes a message, or synchronizes a computer or mobile device with information stored on remote servers.

Rather than targeting individual users, the NSA is gathering contact lists in large numbers that amount to a sizable fraction of the world’s e-mail and instant messaging accounts. Analysis of that data enables the agency to search for hidden connections and to map relationships within a much smaller universe of foreign intelligence targets.

During a single day last year, the NSA’s Special Source Operations branch collected 444,743 e-mail address books from Yahoo, 105,068 from Hotmail, 82,857 from Facebook, 33,697 from Gmail and 22,881 from unspecified other providers, according to an internal NSA PowerPoint presentation. Those figures, described as a typical daily intake in the document, correspond to a rate of more than 250 million a year.

Each day, the presentation said, the NSA collects contacts from an estimated 500,000 buddy lists on live-chat services as well as from the inbox displays of Web-based e-mail accounts.

The collection depends on secret arrangements with foreign telecommunications companies or allied intelligence services in control of facilities that direct traffic along the Internet’s main data routes.

Although the collection takes place overseas, two senior U.S. intelligence officials acknowledged that it sweeps in the contacts of many Americans. They declined to offer an estimate but did not dispute that the number is likely to be in the millions or tens of millions.

A spokesman for the Office of the Director of National Intelligence, which oversees the NSA, said the agency “is focused on discovering and developing intelligence about valid foreign intelligence targets like terrorists, human traffickers and drug smugglers. We are not interested in personal information about ordinary Americans.”

The spokesman, Shawn Turner, added that rules approved by the attorney general require the NSA to “minimize the acquisition, use and dissemination” of information that identifies a U.S. citizen or permanent resident.

The NSA’s collection of nearly all U.S. call records, under a separate program, has generated significant controversy since it was revealed in June. The NSA’s director, Gen. Keith B. Alexander, has defended “bulk” collection as an essential counterterrorism and foreign intelligence tool, saying, “You need the haystack to find the needle.”

Contact lists stored online provide the NSA with far richer sources of data than call records alone. Address books commonly include not only names and e-mail addresses, but also telephone numbers, street addresses, and business and family information. Inbox listings of e-mail accounts stored in the “cloud” sometimes contain content, such as the first few lines of a message.

Taken together, the data would enable the NSA, if permitted, to draw detailed maps of a person’s life, as told by personal, professional, political and religious connections. The picture can also be misleading, creating false “associations” with ex-spouses or people with whom an account holder has had no contact in many years.

The NSA has not been authorized by Congress or the special intelligence court that oversees foreign surveillance to collect contact lists in bulk, and senior intelligence officials said it would be illegal to do so from facilities in the United States. The agency avoids the restrictions in the Foreign Intelligence Surveillance Act by intercepting contact lists from access points “all over the world,” one official said, speaking on the condition of anonymity to discuss the classified program. “None of those are on U.S. territory.”

Because of the method employed, the agency is not legally required or technically able to restrict its intake to contact lists belonging to specified foreign intelligence targets, he said.

When information passes through “the overseas collection apparatus,” the official added, “the assumption is you’re not a U.S. person.”

In practice, data from Americans is collected in large volumes — in part because they live and work overseas, but also because data crosses international boundaries even when its American owners stay at home. Large technology companies, including Google and Facebook, maintain data centers around the world to balance loads on their servers and work around outages.

A senior U.S. intelligence official said the privacy of Americans is protected, despite mass collection, because “we have checks and balances built into our tools.”

NSA analysts, he said, may not search within the contacts database or distribute information from it unless they can “make the case that something in there is a valid foreign intelligence target in and of itself.”

In this program, the NSA is obliged to make that case only to itself or others in the executive branch. With few exceptions, intelligence operations overseas fall solely within the president’s legal purview. The Foreign Intelligence Surveillance Act, enacted in 1978, imposes restrictions only on electronic surveillance that targets Americans or takes place on U.S. territory.

By contrast, the NSA draws on authority in the Patriot Act for its bulk collection of domestic phone records, and it gathers online records from U.S. Internet companies, in a program known as PRISM, under powers granted by Congress in the FISA Amendments Act. Those operations are overseen by the Foreign Intelligence Surveillance Court.

Sen. Dianne Feinstein, the California Democrat who chairs the Senate Intelligence Committee, said in August that the committee has less information about, and conducts less oversight of, intelligence gathering that relies solely on presidential authority. She said she planned to ask for more briefings on those programs.

“In general, the committee is far less aware of operations conducted under 12333,” said a senior committee staff member, referring to Executive Order 12333, which defines the basic powers and responsibilities of the intelligence agencies. “I believe the NSA would answer questions if we asked them, and if we knew to ask them, but it would not routinely report these things, and, in general, they would not fall within the focus of the committee.”

Because the agency captures contact lists “on the fly” as they cross major Internet switches, rather than “at rest” on computer servers, the NSA has no need to notify the U.S. companies that host the information or to ask for help from them.

“We have neither knowledge of nor participation in this mass collection of web-mail addresses or chat lists by the government,” said Google spokeswoman Niki Fenwick.

At Microsoft, spokeswoman Nicole Miller said the company “does not provide any government with direct or unfettered access to our customers’ data,” adding that “we would have significant concerns if these allegations about government actions are true.”

Facebook spokeswoman Jodi Seth said that “we did not know and did not assist” in the NSA’s interception of contact lists.

It is unclear why the NSA collects more than twice as many address books from Yahoo than the other big services combined. One possibility is that Yahoo, unlike other service providers, has left connections to its users unencrypted by default.

Suzanne Philion, a Yahoo spokeswoman, said Monday in response to an inquiry from The Washington Post that, beginning in January, Yahoo would begin encrypting all its e-mail connections.

Google was the first to secure all its e-mail connections, turning on “SSL encryption” globally in 2010. People with inside knowledge said the move was intended in part to thwart large-scale collection of its users’ information by the NSA and other intelligence agencies.

The volume of NSA contacts collection is so high that it has occasionally threatened to overwhelm storage repositories, forcing the agency to halt its intake with “emergency detasking” orders. Three NSA documents describe short-term efforts to build an “across-the-board technology throttle for truly heinous data” and longer-term efforts to filter out information that the NSA does not need.

Spam has proven to be a significant problem for the NSA — clogging databases with information that holds no foreign intelligence value. The majority of all e-mails, one NSA document says, “are SPAM from ‘fake’ addresses and never ‘delivered’ to targets.”

In fall 2011, according to an NSA presentation, the Yahoo account of an Iranian target was “hacked by an unknown actor,” who used it to send spam. The Iranian had “a number of Yahoo groups in his/her contact list, some with many hundreds or thousands of members.”

The cascading effects of repeated spam messages, compounded by the automatic addition of the Iranian’s contacts to other people’s address books, led to a massive spike in the volume of traffic collected by the Britain intelligence service on the NSA’s behalf.

After nine days of data-bombing, the Iranian’s contact book and contact books for several people within it were “emergency detasked.”

In a briefing from the NSA’s Large Access Exploitation working group, that example was used to illustrate the need to narrow the criteria for data interception. It called for a “shifting collection philosophy”: “Memorialize what you need” vs. “Order one of everything off the menu and eat what you want.”

Julie Tate contributed to this report. Soltani is an independent security researcher and consultant.

By Greg Miller, Julie Tate and Barton Gellman

October 16, 2013

It was an innocuous e-mail, one of millions sent every day by spouses with updates on the situation at home. But this one was of particular interest to the National Security Agency and contained clues that put the sender’s husband in the crosshairs of a CIA drone.

Days later, Hassan Ghul — an associate of Osama bin Laden who provided a critical piece of intelligence that helped the CIA find the al-Qaeda leader — was killed by a drone strike in Pakistan’s tribal belt.

The U.S. government has never publicly acknowledged killing Ghul. But documents provided to The Washington Post by former NSA contractor Edward Snowden confirm his demise in October 2012 and reveal the agency’s extensive involvement in the targeted killing program that has served as a centerpiece of President Obama’s counterterrorism strategy.

An al-Qaeda operative who had a knack for surfacing at dramatic moments in the post-Sept. 11 story line, Ghul was an emissary to Iraq for the terrorist group at the height of that war. He was captured in 2004 and helped expose bin Laden’s courier network before spending two years at a secret CIA prison. Then, in 2006, the United States delivered him to his native Pakistan, where he was released and returned to the al-Qaeda fold.

But beyond filling in gaps about Ghul, the documents provide the most detailed account of the intricate collaboration between the CIA and the NSA in the drone campaign.

The Post is withholding many details about those missions, at the request of U.S. intelligence officials who cited potential damage to ongoing operations and national security.

The NSA is “focused on discovering and developing intelligence about valid foreign intelligence targets,” an NSA spokeswoman said in a statement provided to The Post on Wednesday, adding that the agency’s operations “protect the nation and its interests from threats such as terrorism and the proliferation of weapons of mass destruction.”

In the search for targets, the NSA has draped a surveillance blanket over dozens of square miles of northwest Pakistan. In Ghul’s case, the agency deployed an arsenal of cyber-espionage tools, secretly seizing control of laptops, siphoning audio files and other messages, and tracking radio transmissions to determine where Ghul might “bed down.”

The e-mail from Ghul’s wife “about her current living conditions” contained enough detail to confirm the coordinates of that household, according to a document summarizing the mission. “This information enabled a capture/kill operation against an individual believed to be Hassan Ghul on October 1,” it said.

The file is part of a collection of records in the Snowden trove that make clear that the drone campaign — often depicted as the CIA’s exclusive domain — relies heavily on the NSA’s ability to vacuum up enormous quantities of e-mail, phone calls and other fragments of signals intelligence, or SIGINT.

To handle the expanding workload, the NSA created a secret unit known as the Counter-Terrorism Mission Aligned Cell, or CT MAC, to concentrate the agency’s vast resources on hard-to-find terrorism targets. The unit spent a year tracking Ghul and his courier network, tunneling into an array of systems and devices, before he was killed. Without those penetrations, the document concluded, “this opportunity would not have been possible.”

At a time when the NSA is facing intense criticism for gathering data on Americans, the drone files may bolster the agency’s case that its resources are focused on fighting terrorism and supporting U.S. operations overseas.

“Ours is a noble cause,” NSA Director Keith B. Alexander said during a public event last month. “Our job is to defend this nation and to protect our civil liberties and privacy.”

The documents do not explain how the Ghul e-mail was obtained or whether it was intercepted using legal authorities that have emerged as a source of controversy in recent months and enable the NSA to compel technology giants including Microsoft and Google to turn over information about their users. Nor is there a reference to another NSA program facing scrutiny after Snowden’s leaks, its metadata collection of numbers dialed by nearly every person in the United States.

To the contrary, the records indicate that the agency depends heavily on highly targeted network penetrations to gather information that wouldn’t otherwise be trapped in surveillance nets that it has set at key Internet gateways.

The new documents are self-congratulatory in tone, drafted to tout the NSA’s counterterrorism capabilities. One is titled “CT MAC Hassan Gul Success.” The files make no mention of other agencies’ roles in a drone program that escalated dramatically in 2009 and 2010 before tapering off in recent years.

Even so, former CIA officials said the files are an accurate reflection of the NSA’s contribution to finding targets in a campaign that has killed more than 3,000 people, including thousands of alleged militants and hundreds of civilians, in Pakistan, according to independent surveys. The officials said the agency has assigned senior analysts to the CIA’s Counterterrorism Center, and deployed others to work alongside CIA counterparts at almost every major U.S. embassy or military base overseas.

“NSA threw the kitchen sink at the FATA,” said a former U.S. intelligence official with experience in Afghanistan and Pakistan, referring to the Federally Administered Tribal Areas, the region in northwest Pakistan where al-Qaeda’s leadership is based.

NSA employees rarely ventured beyond the security gates of the U.S. Embassy in Islamabad, officials said. Surveillance operations that required placing a device or sensor near an al-Qaeda compound were handled by the CIA’s Information Operations Center, which specializes in high-tech devices and “close-in” surveillance work.

“But if you wanted huge coverage of the FATA, NSA had 10 times the manpower, 20 times the budget and 100 times the brainpower,” the former intelligence official said, comparing the surveillance resources of the NSA to the smaller capabilities of the agency’s IOC. The two agencies are the largest in the U.S. intelligence community, with budgets last year of $14.7 billion for the CIA and $10.8 billion for the NSA. “We provided the map,” the former official said, “and they just filled in the pieces.”

In broad terms, the NSA relies on increasingly sophisticated versions of online attacks that are well-known among security experts. Many rely on software implants developed by the agency’s Tailored Access Operations division with code-names such as UNITEDRAKE and VALIDATOR. In other cases, the agency runs “man-in-the-middle” attacks in which it positions itself unnoticed midstream between computers communicating with one another, diverting files for real-time alerts and longer-term analysis in data repositories.

Through these and other tactics, the NSA is able to extract vast quantities of digital information, including audio files, imagery and keystroke logs. The operations amount to silent raids on suspected safe houses and often are carried out by experts sitting behind desks thousands of miles from their targets.

The reach of the NSA’s Tailored Access Operations division extends far beyond Pakistan. Other documents describe efforts to tunnel into systems used by al-Qaeda affiliates in Yemen and Africa, each breach exposing other corridors.

An operation against a suspected facilitator for al-Qaeda’s branch in Yemen led to a trove of files that could be used to “help NSA map out the movement of terrorists and aspiring extremists between Yemen, Syria, Turkey, Egypt, Libya and Iran,” according to the documents. “This may enable NSA to better flag the movement of these individuals” to allied security services that “can put individuals on no-fly lists or monitor them once in country.”

A single penetration yielded 90 encrypted al-Qaeda documents, 16 encryption keys, 30 unencrypted messages as well as “thousands” of chat logs, according to an inventory described in one of the Snowden documents.

The operations are so easy, in some cases, that the NSA is able to start downloading data in less time than it takes the targeted machine to boot up. Last year, a user account on a social media Web site provided an instant portal to an al-Qaeda operative’s hard drive. “Within minutes, we successfully exploited the target,” the document said.

The hunt for Ghul followed a more elaborate path.

Ghul, who is listed in other documents as Mustafa Haji Muhammad Khan, had surfaced on U.S. radar as early as 2003, when an al-Qaeda detainee disclosed that Ghul escorted one of the intended hijackers to a Pakistani safe house a year before the Sept. 11, 2001, attacks.

A trusted facilitator and courier, Ghul was dispatched to Iraq in 2003 to deliver a message to Abu Musab al-Zarqawi, the al-Qaeda firebrand who angered the network’s leaders in Pakistan by launching attacks that often slaughtered innocent Muslims.

When Ghul made another attempt to enter Iraq in 2004, he was detained by Kurdish authorities in an operation directed by the CIA. Almost immediately, Ghul provided a piece of intelligence that would prove more consequential than he may have anticipated: He disclosed that bin Laden relied on a trusted courier known as al-Kuwaiti.

The ripples from that revelation wouldn’t subside for years. The CIA went on to determine the true identity of al-Kuwaiti and followed him to a heavily fortified compound in Abbottabad, Pakistan, where bin Laden was killed in 2011.

Because of the courier tip, Ghul became an unwitting figure in the contentious debate over CIA interrogation measures. He was held at a CIA black site in Eastern Europe, according to declassified Justice Department memos, where he was slapped and subjected to stress positions and sleep deprivation to break his will.

Defenders of the interrogation program have cited Ghul’s courier disclosure as evidence that the agency’s interrogation program was crucial to getting bin Laden. But others, including former CIA operatives directly involved in Ghul’s case, said that he identified the courier while he was being interrogated by Kurdish authorities, who posed questions scripted by CIA analysts in the background.

The debate resurfaced amid the release of the movie “Zero Dark Thirty” last year, in which a detainee’s slip after a brutal interrogation sequence is depicted as a breakthrough in the bin Laden hunt. Ghul’s case also has been explored in detail in a 6,000-page investigation of the CIA interrogation program by the Senate Intelligence Committee that has yet to be released.

Sen. Dianne Feinstein (D-Calif.), the chairman of the panel, sought to settle the Ghul debate in a statement last year that alluded to his role but didn’t mention him by name.

“The CIA detainee who provided the most significant information about the courier provided the information prior to being subjected to coercive interrogation techniques,” Feinstein said in the statement, which was signed by Sen. Carl Levin (D-Mich.).

The George W. Bush administration’s decision to close the secret CIA prisons in 2006 set off a scramble to place prisoners whom the agency did not regard as dangerous or valuable enough to transfer to Guantanamo Bay. Ghul was not among the original 14 high-value CIA detainees sent to the U.S. installation in Cuba. Instead, he was turned over to the CIA’s counterpart in Pakistan, with ostensible assurances that he would remain in custody.

A year later, Ghul was released. There was no public explanation from Pakistani authorities. CIA officials have noted that Ghul had ties to Lashkar-e-Taiba, a militant group supported by Pakistan’s intelligence service. By 2007, he had returned to al-Qaeda’s stronghold in Waziristan.

In 2011, the Treasury Department named Ghul a target of U.S. counterterrorism sanctions. Since his release, the department said, he had helped al-Qaeda reestablish logistics networks, enabling al-Qaeda to move people and money in and out of the country. The NSA document described Ghul as al-Qaeda’s chief of military operations and detailed a broad surveillance effort to find him.

“The most critical piece” came with a discovery that “provided a vector” for compounds used by Ghul, the document said. After months of investigation, and surveillance by CIA drones, the e-mail from his wife erased any remaining doubt.

Even after Ghul was killed in Mir Ali, the NSA’s role in the drone strike wasn’t done. Although the attack was aimed at “an individual believed to be” the correct target, the outcome wasn’t certain until later when, “through SIGINT, it was confirmed that Hassan Ghul was in fact killed.”

By Barton Gellman and Ashkan Soltani

October 30, 2013

The National Security Agency has secretly broken into the main communications links that connect Yahoo and Google data centers around the world, according to documents obtained from former NSA contractor Edward Snowden and interviews with knowledgeable officials.

By tapping those links, the agency has positioned itself to collect at will from hundreds of millions of user accounts, many of them belonging to Americans. The NSA does not keep everything it collects, but it keeps a lot.

According to a top-secret accounting dated Jan. 9, 2013, the NSA’s acquisitions directorate sends millions of records every day from internal Yahoo and Google networks to data warehouses at the agency’s headquarters at Fort Meade, Md. In the preceding 30 days, the report said, field collectors had processed and sent back 181,280,466 new records — including “metadata,” which would indicate who sent or received e-mails and when, as well as content such as text, audio and video.

The NSA’s principal tool to exploit the data links is a project called MUSCULAR, operated jointly with the agency’s British counterpart, the Government Communications Headquarters . From undisclosed interception points, the NSA and the GCHQ are copying entire data flows across fiber-optic cables that carry information among the data centers of the Silicon Valley giants.

The infiltration is especially striking because the NSA, under a separate program known as PRISM, has front-door access to Google and Yahoo user accounts through a court-approved process.

The MUSCULAR project appears to be an unusually aggressive use of NSA tradecraft against flagship American companies. The agency is built for high-tech spying, with a wide range of digital tools, but it has not been known to use them routinely against U.S. companies.

In a statement, the NSA said it is “focused on discovering and developing intelligence about valid foreign intelligence targets only.”

“NSA applies Attorney General-approved processes to protect the privacy of U.S. persons — minimizing the likelihood of their information in our targeting, collection, processing, exploitation, retention, and dissemination,” it said.

In a statement, Google’s chief legal officer, David Drummond, said the company has “long been concerned about the possibility of this kind of snooping” and has not provided the government with access to its systems.

“We are outraged at the lengths to which the government seems to have gone to intercept data from our private fiber networks, and it underscores the need for urgent reform,” he said.

A Yahoo spokeswoman said, “We have strict controls in place to protect the security of our data centers, and we have not given access to our data centers to the NSA or to any other government agency.”

Under PRISM, the NSA gathers huge volumes of online communications records by legally compelling U.S. technology companies, including Yahoo and Google, to turn over any data that match court-approved search terms. That program, which was first disclosed by The Washington Post and the Guardian newspaper in Britain, is authorized under Section 702 of the FISA Amendments Act and overseen by the Foreign Intelligence Surveillance Court (FISC).

Intercepting communications overseas has clear advantages for the NSA, with looser restrictions and less oversight. NSA documents about the effort refer directly to “full take,” “bulk access” and “high volume” operations on Yahoo and Google networks. Such large-scale collection of Internet content would be illegal in the United States, but the operations take place overseas, where the NSA is allowed to presume that anyone using a foreign data link is a foreigner.

Outside U.S. territory, statutory restrictions on surveillance seldom apply and the FISC has no jurisdiction. Senate Intelligence Committee Chairman Dianne Feinstein (D-Calif.) has acknowledged that Congress conducts little oversight of intelligence-gathering under the presidential authority of Executive Order 12333, which defines the basic powers and responsibilities of the intelligence agencies.

John Schindler, a former NSA chief analyst and frequent defender who teaches at the Naval War College, said it is obvious why the agency would prefer to avoid restrictions where it can.

“Look, NSA has platoons of lawyers, and their entire job is figuring out how to stay within the law and maximize collection by exploiting every loophole,” he said. “It’s fair to say the rules are less restrictive under Executive Order 12333 than they are under FISA,” the Foreign Intelligence Surveillance Act.

In a statement, the Office of the Director of National Intelligence denied that it was using executive authority to “get around the limitations” imposed by FISA.

The operation to infiltrate data links exploits a fundamental weakness in systems architecture. To guard against data loss and system slowdowns, Google and Yahoo maintain fortresslike data centers across four continents and connect them with thousands of miles of fiber-optic cable. Data move seamlessly around these globe-spanning “cloud” networks, which represent billions of dollars of investment.

For the data centers to operate effectively, they synchronize large volumes of information about account holders. Yahoo’s internal network, for example, sometimes transmits entire e-mail archives — years of messages and attachments — from one data center to another.

Tapping the Google and Yahoo clouds allows the NSA to intercept communications in real time and to take “a retrospective look at target activity,” according to one internal NSA document.

To obtain free access to data- center traffic, the NSA had to circumvent gold-standard security measures. Google “goes to great lengths to protect the data and intellectual property in these centers,” according to one of the company’s blog posts, with tightly audited access controls, heat-sensitive cameras, round-the-clock guards and biometric verification of identities.

Google and Yahoo also pay for premium data links, designed to be faster, more reliable and more secure. In recent years, both of them are said to have boughtor leased thousands of miles of fiber-optic cables for their own exclusive use. They had reason to think, insiders said, that their private, internal networks were safe from prying eyes.

In an NSA presentation slide on “Google Cloud Exploitation,” however, a sketch shows where the “Public Internet” meets the internal “Google Cloud” where their data reside. In hand-printed letters, the drawing notes that encryption is “added and removed here!” The artist adds a smiley face, a cheeky celebration of victory over Google security.

Two engineers with close ties to Google exploded in profanity when they saw the drawing. “I hope you publish this,” one of them said.

For the MUSCULAR project, the GCHQ directs all intake into a “buffer” that can hold three to five days of traffic before recycling storage space. From the buffer, custom-built NSA tools unpack and decode the special data formats that the two companies use inside their clouds. Then the data are sent through a series of filters to “select” information the NSA wants and “defeat” what it does not.

PowerPoint slides about the Google cloud, for example, show that the NSA tries to filter out all data from the company’s “Web crawler,” which indexes Internet pages.

According to the briefing documents, prepared by participants in the MUSCULAR project, collection from inside Yahoo and Google has produced important intelligence leads against hostile foreign governments that are specified in the documents.

Last month, long before The Post approached Google to discuss the penetration of its cloud, Eric Grosse, vice president for security engineering, said the company is rushing to encrypt the links between its data centers. “It’s an arms race,” he said then. “We see these government agencies as among the most skilled players in this game.”

Yahoo has not announced plans to encrypt its data-center links.

Because digital communications and cloud storage do not usually adhere to national boundaries, MUSCULAR and a previously disclosed NSA operation to collect Internet address books have amassed content and metadata on a previously unknown scale from U.S. citizens and residents. Those operations have gone undebated in public or in Congress because their existence was classified.

The Google and Yahoo operations call attention to an asymmetry in U.S. surveillance law. Although Congress has lifted some restrictions on NSA domestic surveillance on grounds that purely foreign communications sometimes pass over U.S. switches and cables, it has not added restrictions overseas, where American communications or data stores now cross over foreign switches.

“Thirty-five years ago, different countries had their own telecommunications infrastructure, so the division between foreign and domestic collection was clear,” Sen. Ron Wyden (D-Ore.), a member of the intelligence panel, said in an interview. “Today there’s a global communications infrastructure, so there’s a greater risk of collecting on Americans when the NSA collects overseas.”

It is not clear how much data from Americans is collected and how much of that is retained. One weekly report on MUSCULAR says the British operators of the site allow the NSA to contribute 100,000 “selectors,” or search terms. That is more than twice the number in use in the PRISM program, but even 100,000 cannot easily account for the millions of records that are said to be sent to Fort Meade each day.

In 2011, when the FISC learned that the NSA was using similar methods to collect and analyze data streams — on a much smaller scale — from cables on U.S. territory, Judge John D. Bates ruled that the program was illegal under FISA and inconsistent with the requirements of the Fourth Amendment.

Soltani is an independent security researcher and consultant.

Addtional Content:

How the NSA is infiltrating private networks

How the NSA’s MUSCULAR program collects too much data from Yahoo and Google

How we know the NSA had access to internal Google and Yahoo cloud data

By Barton Gellman and Ashkan Soltani

December 4, 2013

The National Security Agency is gathering nearly 5 billion records a day on the whereabouts of cellphones around the world, according to top-secret documents and interviews with U.S. intelligence officials, enabling the agency to track the movements of individuals — and map their relationships — in ways that would have been previously unimaginable.

The records feed a vast database that stores information about the locations of at least hundreds of millions of devices, according to the officials and the documents, which were provided by former NSA contractor Edward Snowden. New projects created to analyze that data have provided the intelligence community with what amounts to a mass surveillance tool.

The NSA does not target Americans’ location data by design, but the agency acquires a substantial amount of information on the whereabouts of domestic cellphones “incidentally,” a legal term that connotes a foreseeable but not deliberate result.

One senior collection manager, speaking on condition of anonymity but with permission from the NSA, said “we are getting vast volumes” of location data from around the world by tapping into the cables that connect mobile networks globally and that serve U.S. cellphones as well as foreign ones. Additionally, data is often collected from the tens of millions of Americans who travel abroad with their cellphones every year.

In scale, scope and potential impact on privacy, the efforts to collect and analyze location data may be unsurpassed among the NSA surveillance programs that have been disclosed since June. Analysts can find cellphones anywhere in the world, retrace their movements and expose hidden relationships among individuals using them.

(Graphic: How the NSA is tracking people right now)

U.S. officials said the programs that collect and analyze location data are lawful and intended strictly to develop intelligence about foreign targets.

Robert Litt, general counsel for the Office of the Director of National Intelligence, which oversees the NSA, said “there is no element of the intelligence community that under any authority is intentionally collecting bulk cellphone location information about cellphones in the United States.”

The NSA has no reason to suspect that the movements of the overwhelming majority of cellphone users would be relevant to national security. Rather, it collects locations in bulk because its most powerful analytic tools — known collectively as CO-TRAVELER — allow it to look for unknown associates of known intelligence targets by tracking people whose movements intersect.

Still, location data, especially when aggregated over time, is widely regarded among privacy advocates as uniquely sensitive. Sophisticated mathematical techniques enable NSA analysts to map cellphone owners’ relationships by correlating their patterns of movement over time with thousands or millions of other phone users who cross their paths. Cellphones broadcast their locations even when they are not being used to place a call or send a text.

CO-TRAVELER and related tools require the methodical collection and storage of location data on what amounts to a planetary scale. The government is tracking people from afar into confidential business meetings or personal visits to medical facilities, hotel rooms, private homes and other traditionally protected spaces.

“One of the key components of location data, and why it’s so sensitive, is that the laws of physics don’t let you keep it private,” said Chris Soghoian, principal technologist at the American Civil Liberties Union. People who value their privacy can encrypt their e-mails and disguise their online identities, but “the only way to hide your location is to disconnect from our modern communication system and live in a cave.”

The NSA cannot know in advance which tiny fraction of 1 percent of the records it may need, so it collects and keeps as many as it can — 27 terabytes, by one account, or more than double the text content of the Library of Congress’s print collection.

The location programs have brought in such volumes of information, according to a May 2012 internal NSA briefing, that they are “outpacing our ability to ingest, process and store” data. In the ensuing year and a half, the NSA has been transitioning to a processing system that provided it with greater capacity.

The possibility that the intelligence community has been collecting location data, particularly of Americans, has long concerned privacy advocates and some lawmakers. Three Democratic senators — Ron Wyden (Ore.), Mark Udall (Colo.) and Barbara Mikulski (Md.) — have introduced an amendment to the 2014 defense spending bill that would require U.S. intelligence agencies to say whether they have ever collected or made plans to collect location data for “a large number of United States persons with no known connection to suspicious activity.”

NSA Director Keith Alexander disclosed in Senate testimony in October that the NSA had run a pilot project in 2010 and 2011 to collect “samples” of U.S. cellphone location data. The data collected were never available for intelligence analysis purposes, and the project was discontinued because it had no “operational value,” he said.

Alexander allowed that a broader collection of such data “may be something that is a future requirement for the country, but it is not right now.”

The number of Americans whose locations are tracked as part of the NSA’s collection of data overseas is impossible to determine from the Snowden documents alone, and senior intelligence officials declined to offer an estimate.

“It’s awkward for us to try to provide any specific numbers,” one intelligence official said in a telephone interview. An NSA spokeswoman who took part in the call cut in to say the agency has no way to calculate such a figure.

An intelligence lawyer, speaking with his agency’s permission, said location data are obtained by methods “tuned to be looking outside the United States,” a formulation he repeated three times. When U.S. cellphone data are collected, he said, the data are not covered by the Fourth Amendment, which protects Americans against unreasonable searches and seizures.

According to top-secret briefing slides, the NSA pulls in location data around the world from 10 major “sigads,” or signals intelligence activity designators.

A sigad known as STORMBREW, for example, relies on two unnamed corporate partners described only as ARTIFICE and WOLFPOINT. According to an NSA site inventory, the companies administer the NSA’s “physical systems,” or interception equipment, and “NSA asks nicely for tasking/updates.”

STORMBREW collects data from 27 telephone links known as OPC/DPC pairs, which refer to originating and destination points and which typically transfer traffic from one provider’s internal network to another’s. That data include cell tower identifiers, which can be used to locate a phone’s location.

The agency’s access to carriers’ networks appears to be vast.

“Many shared databases, such as those used for roaming, are available in their complete form to any carrier who requires access to any part of it,” said Matt Blaze, an associate professor of computer and information science at the University of Pennsylvania. “This ‘flat’ trust model means that a surprisingly large number of entities have access to data about customers that they never actually do business with, and an intelligence agency — hostile or friendly — can get ‘one stop shopping’ to an expansive range of subscriber data just by compromising a few carriers.”

Some documents in the Snowden archive suggest that acquisition of U.S. location data is routine enough to be cited as an example in training materials. In an October 2012 white paper on analytic techniques, for example, the NSA’s counterterrorism analysis unit cites two U.S.-based carriers to illustrate the challenge of correlating the travels of phone users on different mobile networks. Asked about that, a U.S. intelligence official said the example was poorly chosen and did not represent the program’s foreign focus.

The NSA’s capabilities to track location are staggering, based on the Snowden documents, and indicate that the agency is able to render most efforts at communications security effectively futile.

Like encryption and anonymity tools online, which are used by dissidents, journalists and terrorists alike, security-minded behavior — using disposable cellphones and switching them on only long enough to make brief calls — marks a user for special scrutiny. CO-TRAVELER takes note, for example, when a new telephone connects to a cell tower soon after another nearby device is used for the last time.

Side-by-side security efforts — when nearby devices power off and on together over time — “assist in determining whether co-travelers are associated … through behaviorally relevant relationships,” according to the 24-page white paper, which was developed by the NSA in partnership with the National Geospatial Agency, the Australian Signals Directorate and private contractors.

A central feature of each of these tools is that they do not rely on knowing a particular target in advance, or even suspecting one. They operate on the full universe of data in the NSA’s FASCIA repository, which stores trillions of metadata records, of which a large but unknown fraction include locations.

The most basic analytic tools map the date, time, and location of cellphones to look for patterns or significant moments of overlap. Other tools compute speed and trajectory for large numbers of mobile devices, overlaying the electronic data on transportation maps to compute the likely travel time and determine which devices might have intersected.

To solve the problem of undetectable surveillance against CIA officers stationed overseas, one contractor designed an analytic model that would carefully record the case officer’s path and look for other mobile devices in steady proximity.

“Results have not been validated by operational analysts,” the report said.

Julie Tate contributed to this report. Soltani is an independent security researcher and consultant.

Additional Content:

How the NSA is traking people right now

How the NSA uses cellphone tracking to find and ‘develop’ targets (VIDEO)

By Ellen Nakashima and Ann E. Marimow

December 16, 2013

A federal judge ruled Monday that the National Security Agency’s daily collection of virtually all Americans’ phone records is almost certainly unconstitutional.

U.S. District Judge Richard J. Leon found that a lawsuit by Larry Klayman, a conservative legal activist, has “demonstrated a substantial likelihood of success” on the basis of Fourth Amendment privacy protections against unreasonable searches.

Leon granted the request for an injunction that blocks the collection of phone data for Klayman and a co-plaintiff and orders the government to destroy any of their records that have been gathered. But the judge stayed action on his ruling pending a government appeal, recognizing in his 68-page opinion the “significant national security interests at stake in this case and the novelty of the constitutional issues.”

“I cannot imagine a more ‘indiscriminate’ and ‘arbitrary invasion’ than this systematic and high tech collection and retention of personal data on virtually every single citizen for purposes of querying and analyzing it without prior judicial approval,” said Leon, a judge on the U.S. District Court for the District of Columbia. “Surely, such a program infringes on ‘that degree of privacy’ that the founders enshrined in the Fourth Amendment.”

The strongly worded decision stands in contrast to the secret deliberations of 15 judges on the nation’s surveillance court, which hears only the government’s side of cases and since 2006 has held in a series of classified rulings that the program is lawful. It marks the first time a federal judge in open court has opined on the collection of lawfulness in a case not involving a criminal defendant.

A Justice Department spokesman, Andrew Ames, said Monday that the government was reviewing Leon’s decision. “We believe the program is constitutional as previous judges have found,” he said.

The challenge to the NSA’s once-classified collection of phone records is one of a series of cases filed in federal court since the program’s existence was revealed in June by former NSA contractor Edward Snowden.

Snowden praised the ruling in a statement made to journalist Glenn Greenwald, who received NSA documents from Snowden and first reported on the program’s existence.

“I acted on my belief that the NSA’s mass surveillance programs would not withstand a constitutional challenge, and that the American public deserved a chance to see these issues determined by open courts,” said Snowden, who has received temporary asylum in Russia, where he is seeking to avoid U.S. prosecution under the Espionage Act for leaking NSA documents. “Today, a secret program authorized by a secret court was, when exposed to the light of day, found to violate Americans’ rights. It is the first of many.”

The ruling also comes as Congress is debating whether to end the NSA’s “bulk” collection of phone data or endorse it in statute. The White House, U.S. officials say, supports maintaining the program.

“It will be very difficult for the administration to argue that the NSA’s call-tracking program should continue when a federal judge has found it to be unconstitutional,” said Jameel Jaffer, deputy legal director of the American Civil Liberties Union, which has also sued the government over the program’s constitutionality.

But George Washington University law professor Orin Kerr said, “It gives opponents of the NSA program more fuel to add to the fire, but its legal impact is quite limited because the case now just goes to the court of appeals.”

The government has stressed that the program collects only “metadata,” such as numbers dialed and the times and lengths of calls, but no phone content or subscriber names. Officials say that only numbers linked to suspected terrorists are run against the database.

Leon’s opinion countered that the program is so sweeping — the database easily contains billions of records — that it amounts to a “dragnet” that intrudes on the constitutional expectation of privacy. He dismissed the government’s claim that “special needs” requiring quick access to data that could thwart a terrorist plot make a warrant impracticable. “No court has ever recognized a special need sufficient to justify continuous, daily searches of virtually every American citizen without any particularized suspicion,” he said.

The government’s legal justification for the call-tracking program is based on a 1979 case, Smith v. Maryland, involving the surveillance of a criminal suspect over a two-day period. In that case, the Supreme Court said that Americans have no expectation of privacy in the telephone metadata that companies hold as business records, and that therefore a warrant is not required to obtain such information. A succession of judges on the Foreign Intelligence Surveillance Court have adopted the government’s argument based on that ruling.

But Leon said the question the Supreme Court confronted in 1979 is not the same as the one he was faced with. “Indeed, the question in this case can more properly be styled as follows: When do present-day circumstances — the evolutions in the government’s surveillance capabilities, citizens’ phone habits, and the relationship between the NSA and telecom companies — become so thoroughly unlike those considered by the Supreme Court thirty-four years ago, that a precedent like Smith does not apply?” he wrote. “The answer, unfortunately for the government, is now.”

Kerr said Leon is wrong to suggest that Smith no longer applies. That decision, he said, draws a clear distinction between the collection of data on numbers dialed and on call content. The metadata information the government is gathering today, Kerr said, is the same type of information the court said that law enforcement could collect more than 30 years ago. “The opinion is more valid now than it was,” Kerr said, adding that “it’s up to the Supreme Court to overturn its decision, not trial judges.”

Leon, who was nominated by President George W. Bush and appointed to the bench in 2002, said the government has played down the program’s invasiveness.

The “almost-Orwellian technology” that allows the government to collect, store and analyze phone metadata is “unlike anything that could have been conceived in 1979” and, “at best, the stuff of science fiction,” he said.

Klayman, founder of the public interest group Freedom Watch, called Leon’s ruling “courageous.”

“This is a warning to both parties that they’d better start observing the rule of law and protecting the American people or there will be severe consequences,” he said.

Meanwhile, Verizon Communications, named as a defendant, filed a motion Monday asking to be dismissed from the case. The request was based in part on a certification also filed Monday by the Justice Department that the company has immunity against lawsuits since it was compelled by a court order to comply with the government’s request for data. That immunity was authorized under a 2008 law known as the FISA Amendments Act.

In his ruling, Leon rejected the government’s argument that Klayman and a co-plaintiff — the father of an NSA cryptologist killed in Afghanistan in 2011 — lacked standing to bring the suit because they were customers of Verizon subsidiary Verizon Wireless, which has not been publicly revealed as taking part in the program.

“The government,” he said, says it has created a “comprehensive” database — “in which case, the NSA must have collected metadata from Verizon Wireless, the single largest wireless carrier in the United States.” Yet, at the same time, he wrote, the government asserts that the plaintiffs lack standing “based on the theoretical possibility” that the NSA has not collected Verizon’s records. “Candor of this type defies common sense and does not exactly inspire confidence!” he wrote.

To draw an analogy, he wrote, omitting Verizon Wireless, AT&T or Sprint “would be like omitting John, Paul and George” and building a “Ringo-only database.”

Greg Miller, Sari Horwitz, Julie Tate and Peter Wallsten contributed to this report.

Get the latest news and essential information directly in your inbox from The Washington Post.
Sign up for free e-newsletters and alerts.

Connect with The Washington Post on Twitter @washingtonpost and Facebook.

Learn more about the Washington Post Series:

Connect with us for information on new titles and authors from Diversion Books, free excerpts, special promotions, contests, and more:

@DiversionBooks

www.Facebook.com/DiversionBooks

Diversion Books eNewsletter

www.scribd.com/DiversionBooks