The Washington Post

Google, Facebook, Apple, Yahoo deny giving NSA direct access to servers

By Barton Gellman and Laura Poitras

The National Security Agency and the FBI are tapping directly into the central servers of nine leading U.S. Internet companies, extracting audio and video chats, photographs, e-mails, documents, and connection logs that enable analysts to track foreign targets, according to a top-secret document obtained by The Washington Post.

The program, code-named PRISM, has not been made public until now. It may be the first of its kind. The NSA prides itself on stealing secrets and breaking codes, and it is accustomed to corporate partnerships that help it divert data traffic or sidestep barriers. But there has never been a Google or Facebook before, and it is unlikely that there are richer troves of valuable intelligence than the ones in Silicon Valley.

Equally unusual is the way the NSA extracts what it wants, according to the document: “Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.”

London’s Guardian newspaper reported Friday that GCHQ, Britain’s equivalent of the NSA, also has been secretly gathering intelligence from the same internet companies through an operation set up by the NSA.

According to documents obtained by The Guardian, PRISM would appear to allow GCHQ to circumvent the formal legal process required in Britain to seek personal material such as emails, photos and videos from an internet company based outside of the country.

PRISM was launched from the ashes of President George W. Bush’s secret program of warrantless domestic surveillance in 2007, after news media disclosures, lawsuits and the Foreign Intelligence Surveillance Court forced the president to look for new authority.

Congress obliged with the Protect America Act in 2007 and the FISA Amendments Act of 2008, which immunized private companies that cooperated voluntarily with U.S. intelligence collection. PRISM recruited its first partner, Microsoft, and began six years of rapidly growing data collection beneath the surface of a roiling national debate on surveillance and privacy. Late last year, when critics in Congress sought changes in the FISA Amendments Act, the only lawmakers who knew about PRISM were bound by oaths of office to hold their tongues.

The court-approved program is focused on foreign communications traffic, which often flows through U.S. servers even when sent from one overseas location to another. Between 2004 and 2007, Bush administration lawyers persuaded federal FISA judges to issue surveillance orders in a fundamentally new form. Until then the government had to show probable cause that a particular “target” and “facility” were both connected to terrorism or espionage.

In four new orders, which remain classified, the court defined massive data sets as “facilities” and agreed to certify periodically that the government had reasonable procedures in place to minimize collection of “U.S. persons” data without a warrant.

In a statement issue late Thursday, Director of National Intelligence James R. Clapper said “information collected under this program is among the most important and valuable foreign intelligence information we collect, and is used to protect our nation from a wide variety of threats. The unauthorized disclosure of information about this important and entirely legal program is reprehensible and risks important protections for the security of Americans.”

Clapper added that there were numerous inaccuracies in reports about PRISM by The Post and the Guardian newspaper, but he did not specify any.

Jameel Jaffer, deputy legal director of the American Civil Liberties Union, said: “I would just push back on the idea that the court has signed off on it, so why worry? This is a court that meets in secret, allows only the government to appear before it, and publishes almost none of its opinions. It has never been an effective check on government.”

Several companies contacted by The Post said they had no knowledge of the program, did not allow direct government access to their servers and asserted that they responded only to targeted requests for information.

“We do not provide any government organization with direct access to Facebook servers,” said Joe Sullivan, chief security officer for Facebook. “When Facebook is asked for data or information about specific individuals, we carefully scrutinize any such request for compliance with all applicable laws, and provide information only to the extent required by law.”

“We have never heard of PRISM,” said Steve Dowling, a spokesman for Apple. “We do not provide any government agency with direct access to our servers, and any government agency requesting customer data must get a court order.”

It is possible that the conflict between the PRISM slides and the company spokesmen is the result of imprecision on the part of the NSA author. In another classified report obtained by The Post, the arrangement is described as allowing “collection managers [to send] content tasking instructions directly to equipment installed at company-controlled locations,” rather than directly to company servers.

Government officials and the document itself made clear that the NSA regarded the identities of its private partners as PRISM’s most sensitive secret, fearing that the companies would withdraw from the program if exposed. “98 percent of PRISM production is based on Yahoo, Google and Microsoft; we need to make sure we don’t harm these sources,” the briefing’s author wrote in his speaker’s notes.

An internal presentation of 41 briefing slides on PRISM, dated April 2013 and intended for senior analysts in the NSA’s Signals Intelligence Directorate, described the new tool as the most prolific contributor to the President’s Daily Brief, which cited PRISM data in 1,477 items last year. According to the slides and other supporting materials obtained by The Post, “NSA reporting increasingly relies on PRISM” as its leading source of raw material, accounting for nearly 1 in 7 intelligence reports.

That is a remarkable figure in an agency that measures annual intake in the trillions of communications. It is all the more striking because the NSA, whose lawful mission is foreign intelligence, is reaching deep inside the machinery of American companies that host hundreds of millions of American-held accounts on American soil.

The technology companies, whose cooperation is essential to PRISM operations, include most of the dominant global players of Silicon Valley, according to the document. They are listed on a roster that bears their logos in order of entry into the program: “Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” PalTalk, although much smaller, has hosted traffic of substantial intelligence interest during the Arab Spring and in the ongoing Syrian civil war.

Dropbox, the cloud storage and synchronization service, is described as “coming soon.”

Sens. Ron Wyden (D-Ore.) and Mark Udall (D-Colo.), who had classified knowledge of the program as members of the Senate Intelligence Committee, were unable to speak of it when they warned in a Dec. 27, 2012, floor debate that the FISA Amendments Act had what both of them called a “back-door search loophole” for the content of innocent Americans who were swept up in a search for someone else.

“As it is written, there is nothing to prohibit the intelligence community from searching through a pile of communications, which may have been incidentally or accidentally been collected without a warrant, to deliberately search for the phone calls or e-mails of specific Americans,” Udall said.

Wyden repeatedly asked the NSA to estimate the number of Americans whose communications had been incidentally collected, and the agency’s director, Lt. Gen. Keith B. Alexander, insisted there was no way to find out. Eventually Inspector General I. Charles McCullough III wrote Wyden a letter stating that it would violate the privacy of Americans in NSA data banks to try to estimate their number.

Roots in the ’70s

PRISM is an heir, in one sense, to a history of intelligence alliances with as many as 100 trusted U.S. companies since the 1970s. The NSA calls these Special Source Operations, and PRISM falls under that rubric.

The Silicon Valley operation works alongside a parallel program, code-named BLARNEY, that gathers up “metadata” — technical information about communications traffic and network devices — as it streams past choke points along the backbone of the Internet. BLARNEY’s top-secret program summary, set down in the slides alongside a cartoon insignia of a shamrock and a leprechaun hat, describes it as “an ongoing collection program that leverages IC [intelligence community] and commercial partnerships to gain access and exploit foreign intelligence obtained from global networks.”

But the PRISM program appears to more nearly resemble the most controversial of the warrantless surveillance orders issued by President George W. Bush after the al-Qaeda attacks of Sept. 11, 2001. Its history, in which President Obama presided over exponential growth in a program that candidate Obama criticized, shows how fundamentally surveillance law and practice have shifted away from individual suspicion in favor of systematic, mass collection techniques.

 

The Obama administration points to ongoing safeguards in the form of “extensive procedures, specifically approved by the court, to ensure that only non-U.S. persons outside the U.S. are targeted, and that minimize the acquisition, retention and dissemination of incidentally acquired information about U.S. persons.”

And it is true that the PRISM program is not a dragnet, exactly. From inside a company’s data stream the NSA is capable of pulling out anything it likes, but under current rules the agency does not try to collect it all.

Analysts who use the system from a Web portal at Fort Meade, Md., key in “selectors,” or search terms, that are designed to produce at least 51 percent confidence in a target’s “foreignness.” That is not a very stringent test. Training materials obtained by The Post instruct new analysts to make quarterly reports of any accidental collection of U.S. content, but add that “it’s nothing to worry about.”

Even when the system works just as advertised, with no American singled out for targeting, the NSA routinely collects a great deal of American content. That is described as “incidental,” and it is inherent in contact chaining, one of the basic tools of the trade. To collect on a suspected spy or foreign terrorist means, at minimum, that everyone in the suspect’s inbox or outbox is swept in. Intelligence analysts are typically taught to chain through contacts two “hops” out from their target, which increases “incidental collection” exponentially. The same math explains the aphorism, from the John Guare play, that no one is more than “six degrees of separation” from any other person.

A ‘directive’

In exchange for immunity from lawsuits, companies such as Yahoo and AOL are obliged to accept a “directive” from the attorney general and the director of national intelligence to open their servers to the FBI’s Data Intercept Technology Unit, which handles liaison to U.S. companies from the NSA. In 2008, Congress gave the Justice Department authority for a secret order from the Foreign Surveillance Intelligence Court to compel a reluctant company “to comply.”

In practice, there is room for a company to maneuver, delay or resist. When a clandestine intelligence program meets a highly regulated industry, said a lawyer with experience in bridging the gaps, neither side wants to risk a public fight. The engineering problems are so immense, in systems of such complexity and frequent change, that the FBI and NSA would be hard pressed to build in back doors without active help from each company.

Apple demonstrated that resistance is possible when it held out for more than five years, for reasons unknown, after Microsoft became PRISM’s first corporate partner in May 2007. Twitter, which has cultivated a reputation for aggressive defense of its users’ privacy, is still conspicuous by its absence from the list of “private sector partners.”

Google, like the other companies, denied that it permitted direct government access to its servers.

“Google cares deeply about the security of our users’ data,” a company spokesman said. “We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government ‘back door’ into our systems, but Google does not have a ‘back door’ for the government to access private user data.”

Microsoft also provided a statement: “We provide customer data only when we receive a legally binding order or subpoena to do so, and never on a voluntary basis. In addition we only ever comply with orders for requests about specific accounts or identifiers. If the government has a broader voluntary national security program to gather customer data we don’t participate in it.”

Yahoo also issued a denial.

“Yahoo! takes users’ privacy very seriously,” the company said in a statement. “We do not provide the government with direct access to our servers, systems, or network.”

Like market researchers, but with far more privileged access, collection managers in the NSA’s Special Source Operations group, which oversees the PRISM program, are drawn to the wealth of information about their subjects in online accounts. For much the same reason, civil libertarians and some ordinary users may be troubled by the menu available to analysts who hold the required clearances to “task” the PRISM system.

There has been “continued exponential growth in tasking to Facebook and Skype,” according to the PRISM slides. With a few clicks and an affirmation that the subject is believed to be engaged in terrorism, espionage or nuclear proliferation, an analyst obtains full access to Facebook’s “extensive search and surveillance capabilities against the variety of online social networking services.”

According to a separate “User’s Guide for PRISM Skype Collection,” that service can be monitored for audio when one end of the call is a conventional telephone and for any combination of “audio, video, chat, and file transfers” when Skype users connect by computer alone. Google’s offerings include Gmail, voice and video chat, Google Drive files, photo libraries, and live surveillance of search terms.

Firsthand experience with these systems, and horror at their capabilities, is what drove a career intelligence officer to provide PowerPoint slides about PRISM and supporting materials to The Washington Post in order to expose what he believes to be a gross intrusion on privacy. “They quite literally can watch your ideas form as you type,” the officer said.

Poitras is a documentary filmmaker and MacArthur Fellow. Julie Tate, Robert O’Harrow Jr., Cecilia Kang and Ellen Nakashima contributed to this report.

A timeline of surveillance in the United States from 2001 to 2013: from the Patriot Act to the PRISM program.

A REPORTER'S ACCOUNT

To leaker, personal risks were clear

By Barton Gellman

He called me BRASSBANNER, a code name in the double-barreled style of the National Security Agency, where he worked in the signals intelligence directorate.

Verax was the name he chose for himself, “truth teller” in Latin. I asked him early on, without reply, whether he intended to hint at the alternative fates that lay before him.

Two British dissenters had used the pseudonym. Clement Walker, a 17th-century detractor of Parliament, died in the brutal confines of the Tower of London. Two centuries later, social critic Henry Dunckley adopted “Verax” as his byline over weekly columns in the Manchester Examiner. He was showered with testimonials and an honorary degree.

Edward Joseph Snowden, 29, knew full well the risks he had undertaken and the awesome powers that would soon be arrayed to hunt for him. Pseudonyms were the least of his precautions as we corresponded from afar. Snowden was spilling some of the most sensitive secrets of a surveillance apparatus he had grown to detest. By late last month, he believed he was already “on the X” — exposure imminent.

“I understand that I will be made to suffer for my actions, and that the return of this information to the public marks my end,” he wrote in early May, before we had our first direct contact. He warned that even journalists who pursued his story were at risk until they published.

The U.S. intelligence community, he wrote, “will most certainly kill you if they think you are the single point of failure that could stop this disclosure and make them the sole owner of this information.”

I did not believe that literally, but I knew he had reason to fear.

A series of indirect contacts preceded our first direct exchange May 16. Snowden was not yet ready to tell me his name, but he said he was certain to be exposed — by his own hand or somebody else’s. Until then, he asked that I not quote him at length. He said semantic analysis, another of the NSA’s capabilities, would identify him by his patterns of language.

“You can’t protect the source,” he wrote, “but if you help me make the truth known, I will consider it a fair trade.” Later, he added, “There’s no saving me.”

I asked him, at the risk of estrangement, how he could justify exposing intelligence methods that might benefit U.S. adversaries.

“Perhaps I am naive,” he replied, “but I believe that at this point in history, the greatest danger to our freedom and way of life comes from the reasonable fear of omniscient State powers kept in check by nothing more than policy documents.” The steady expansion of surveillance powers, he wrote, is “such a direct threat to democratic governance that I have risked my life and family for it.”

In an e-mail on May 24, he dropped a bombshell. Whistleblowers before him, he said, had been destroyed by the experience. Snowden wanted “to embolden others to step forward,” he wrote, by showing that “they can win.” He therefore planned to apply for asylum in Iceland or some other country “with strong internet and press freedoms,” although “the strength of the reaction will determine how choosy I can be.”

He alluded to other options, aware that he had secrets of considerable financial value, but said, “I have no desire to provide raw source material to a foreign government.”

To effect his plan, Snowden asked for a guarantee that The Washington Post would publish — within 72 hours — the full text of a PowerPoint presentation describing PRISM, a top-secret surveillance program that gathered intelligence from Microsoft, Facebook, Google and other Silicon Valley giants. He also asked that The Post publish online a cryptographic key that he could use to prove to a foreign embassy that he was the document’s source.

I told him we would not make any guarantee about what we published or when. (The Post broke the story two weeks later, on Thursday. The Post sought the views of government officials about the potential harm to national security prior to publication and decided to reproduce only four of the 41 slides.)

Snowden replied succinctly, “I regret that we weren’t able to keep this project unilateral.” Shortly afterward he made contact with Glenn Greenwald of the British newspaper the Guardian.

We continued our correspondence. He was capable of melodrama but wrote with some eloquence about his beliefs.

“The internet is on principle a system that you reveal yourself to in order to fully enjoy, which differentiates it from, say, a music player,” he wrote. “It is a TV that watches you. The majority of people in developed countries spend at least some time interacting with the Internet, and Governments are abusing that necessity in secret to extend their powers beyond what is necessary and appropriate.”

What about legitimate threats to national security?

“We managed to survive greater threats in our history . . . than a few disorganized terrorist groups and rogue states without resorting to these sorts of programs,” he wrote. “It is not that I do not value intelligence, but that I oppose . . . omniscient, automatic, mass surveillance. . . . That seems to me a greater threat to the institutions of free society than missed intelligence reports, and unworthy of the costs.”

Did he impute evil motives to his former colleagues, or the White House?

“Analysts (and government in general) aren’t bad guys, and they don’t want to think of themselves as such,” he replied. But he said they labored under a false premise that “if a surveillance program produces information of value, it legitimizes it. . . . In one step, we’ve managed to justify the operation of the Panopticon” — an 18th-century design by British philosopher Jeremy Bentham for comprehensive surveillance of a prison population.

On Thursday, before The Post published its first story, I made contact on a new channel. He was not expecting me there and responded in alarm.

“Do I know you?” he wrote.

I sent him a note on another channel to verify my digital “fingerprint,” a precaution we had been using for some time. Tired, I sent the wrong one. “That is not at all the right fingerprint,” he wrote, preparing to sign off. “You’re getting MITM’d.” He was talking about a “man in the middle” attack, a standard NSA technique to bypass encryption. I hastily corrected my error.

“The police already visited my house [in Hawaii] this morning” with questions on his whereabouts, he wrote, explaining his jitters. “It obviously has a profound and intimidating impact on my family.”

Despite our previous dispute about publishing the PRISM document in full, Snowden said he did not intend to release a pile of unedited documents upon the world. “I don’t desire to enable the Bradley Manning argument that these were released recklessly and unreviewed,” he said.

On Sunday afternoon, as his name was released to the world, Snowden chatted with me live from a Hong Kong hotel room, not far from a CIA base in the U.S. Consulate.

“There’s no precedent in my life for this kind of thing,” he wrote. “I’ve been a spy for almost all of my adult life — I don’t like being in the spotlight.”

I asked him once more which of the two Veraxes he expected to become: the happy ending or life behind bars?

“That’s up to the global public,” he typed back. “If asylum is offered, we’ll have the first example. If not, we’ll have the second. I am prepared for both.”

Four-pronged U.S. approach relies heavily on data behind Internet, phone communications

By Barton Gellman

On March 12, 2004, acting attorney general James B. Comey and the Justice Department’s top leadership reached the brink of resignation over electronic surveillance orders that they believed to be illegal.

President George W. Bush backed down, halting secret foreign-intelligence-gathering operations that had crossed into domestic terrain. That morning marked the beginning of the end of STELLARWIND, the cover name for a set of four surveillance programs that brought Americans and American territory within the domain of the National Security Agency for the first time in decades. It was also a prelude to new legal structures that allowed Bush and then President Obama to reproduce each of those programs and expand their reach.

What exactly STELLARWIND did has never been disclosed in an unclassified form. Which parts of it did Comey approve? Which did he shut down? What became of the programs when the crisis passed and Comey, now Obama’s expected nominee for FBI director, returned to private life?

Authoritative new answers to those questions, drawing upon a classified NSA history of STELLARWIND and interviews with high-ranking intelligence officials, offer the clearest map yet of the Bush-era programs and the NSA’s contemporary U.S. operations.

STELLARWIND was succeeded by four major lines of intelligence collection in the territorial United States, together capable of spanning the full range of modern telecommunications, according to the interviews and documents.

Foreigners, not Americans, are the NSA’s “targets,” as the law defines that term. But the programs are structured broadly enough that they touch nearly every American household in some way. Obama administration officials and career intelligence officers say Americans should take comfort that privacy protections are built into the design and oversight, but they are not prepared to discuss the details.

The White House, the NSA and the Office of the Director of National Intelligence declined to comment on the record for this article. A senior intelligence official agreed to answer questions if not identified.

“We have rich oversight across three branches of government. I’ve got an [inspector general] here, a fairly robust legal staff here . . . and there’s the Justice Department’s national security division,” the official said. “For those things done under court jurisdiction, the courts are intrusive in my business, appropriately so, and there are two congressional committees. It’s a belts-and-suspenders-and-Velcro approach, and inside there’s rich auditing.”

But privacy advocates, such as Sen. Ron Wyden (D-Ore.), said the intelligence committee on which he serves needs “straight answers” to do vigorous oversight.

He added: “The typical person says, ‘If I am law-abiding and the government is out there collecting lots of information about me — who I call, when I call, where I call from’ . . . I think the typical person is going to say, ‘That sure sounds like it could have some effect on my privacy.’ ”

Two of the four collection programs, one each for telephony and the Internet, process trillions of “metadata” records for storage and analysis in systems called MAINWAY and MARINA, respectively. Metadata includes highly revealing information about the times, places, devices and participants in electronic communication, but not its contents. The bulk collection of telephone call records from Verizon Business Services, disclosed this month by the British newspaper the Guardian, is one source of raw intelligence for MAINWAY.

The other two types of collection, which operate on a much smaller scale, are aimed at content. One of them intercepts telephone calls and routes the spoken words to a system called ­NUCLEON.

For Internet content, the most important source collection is the PRISM project reported on June 6 by The Washington Post and the Guardian. It draws from data held by Google, Yahoo, Microsoft and other Silicon Valley giants, collectively the richest depositories of personal information in history.

Former NSA contractor Edward Snowden, 29, who unmasked himself as the source behind the PRISM and Verizon revelations, said he hoped for a systematic debate about the “danger to our freedom and way of life” posed by a surveillance apparatus “kept in check by nothing more than policy.”

For well over a week, he has had his wish. Startling disclosures have poured out of the nation’s largest and arguably tightest-lipped spy agency at an unprecedented pace. Snowden’s disclosures have opened a national conversation about the limits of secret surveillance in a free society and an outcry overseas against U.S. espionage.

The debate has focused on two of the four U.S.-based collection programs: PRISM, for Internet content, and the comprehensive collection of telephone call records, foreign and domestic, that the Guardian revealed by posting a classified order from the Foreign Intelligence Surveillance Court to Verizon Business Services.

The Post has learned that similar orders have been renewed every three months for other large U.S. phone companies, including Bell South and AT&T, since May 24, 2006. On that day, the surveillance court made a fundamental shift in its approach to Section 215 of the Patriot Act, which permits the FBI to compel production of “business records” that are relevant to a particular terrorism investigation and to share those in some circumstances with the NSA. Henceforth, the court ruled, it would define the relevant business records as the entirety of a telephone company’s call database.

The Bush administration, by then, had been taking “bulk metadata” from the phone companies under voluntary agreements for more than four years. The volume of information overwhelmed the MAINWAY database, according to a classified report from the NSA inspector general in 2009. The agency spent $146 million in supplemental counterterrorism funds to buy new hardware and contract support — and to make unspecified payments to the phone companies for “collaborative partnerships.”

When the New York Times revealed the warrantless surveillance of voice calls, in December 2005, the telephone companies got nervous. One of them, unnamed in the report, approached the NSA with a request. Rather than volunteer the data, at a price, the “provider preferred to be compelled to do so by a court order,” the report said. Other companies followed suit. The surveillance court order that recast the meaning of business records “essentially gave NSA the same authority to collect bulk telephony metadata from business records that it had” under Bush’s asserted authority alone.

Telephone metadata was not the issue that sparked a rebellion at the Justice Department, first by Jack Goldsmith of the Office of Legal Counsel and then by Comey, who was acting attorney general because John D. Ashcroft was in intensive care with acute gallstone pancreatitis. It was Internet metadata.

At Bush’s direction, in orders prepared by David Addington, the counsel to Vice President Richard B. Cheney, the NSA had been siphoning e-mail metadata and technical records of Skype calls from data links owned by AT&T, Sprint and MCI, which later merged with Verizon.

For reasons unspecified in the report, Goldsmith and Comey became convinced that Bush had no lawful authority to do that.

MARINA and the collection tools that feed it are probably the least known of the NSA’s domestic operations, even among experts who follow the subject closely. Yet they probably capture information about more American citizens than any other, because the volume of e-mail, chats and other Internet communications far exceeds the volume of standard telephone calls.

 

The NSA calls Internet metadata “digital network information.” Sophisticated analysis of those records can reveal unknown associates of known terrorism suspects. Depending on the methods applied, it can also expose medical conditions, political or religious affiliations, confidential business negotiations and extramarital affairs.

What permits the former and prevents the latter is a complex set of policies that the public is not permitted to see. “You could do analyses that give you more information, but the law and procedures don’t allow that,” a senior U.S. intelligence lawyer said.

In the urgent aftermath of Sept. 11, 2001, with more attacks thought to be imminent, analysts wanted to use “contact chaining” techniques to build what the NSA describes as network graphs of people who represented potential threats.

The legal challenge for the NSA was that its practice of collecting high volumes of data from digital links did not seem to meet even the relatively low requirements of Bush’s authorization, which allowed collection of Internet metadata “for communications with at least one communicant outside the United States or for which no communicant was known to be a citizen of the United States,” the NSA inspector general’s report said.

Lawyers for the agency came up with an interpretation that said the NSA did not “acquire” the communications, a term with formal meaning in surveillance law, until analysts ran searches against it. The NSA could “obtain” metadata in bulk, they argued, without meeting the required standards for acquisition.

Goldsmith and Comey did not buy that argument, and a high-ranking U.S. intelligence official said the NSA does not rely on it today.

As soon as surveillance data “touches us, we’ve got it, whatever verbs you choose to use,” the official said in an interview. “We’re not saying there’s a magic formula that lets us have it without having it.”

When Comey finally ordered a stop to the program, Bush signed an order renewing it anyway. Comey, Goldsmith, FBI Director Robert S. Mueller III and most of the senior Bush appointees in the Justice Department began drafting letters of resignation.

Then-NSA Director Michael V. Hayden was not among them. According to the inspector general’s classified report, Cheney’s lawyer, Addington, placed a phone call and “General Hayden had to decide whether NSA would execute the Authorization without the Attorney General’s signature.” He decided to go along.

The following morning, when Mueller told Bush that he and Comey intended to resign, the president reversed himself.

Three months later, on July 15, the secret surveillance court allowed the NSA to resume bulk collection under the court’s own authority. The opinion, which remains highly classified, was based on a provision of electronic surveillance law, known as “pen register, trap and trace,” that was written to allow law enforcement officers to obtain the phone numbers of incoming and outgoing calls from a single telephone line.

When the NSA aims for foreign targets whose communications cross U.S. infrastructure, it expects to sweep in some American content “incidentally” or “inadvertently,” which are terms of art in regulations governing the NSA. Contact chaining, because it extends to the contacts of contacts of targets, inevitably collects even more American data.

Current NSA director Keith B. Alexander and Director of National Intelligence James R. Clapper Jr. have resolutely refused to offer an estimate of the number of Americans whose calls or e-mails have thus made their way into content databases such as ­NUCLEON.

The agency and its advocates maintain that its protection of that data is subject to rigorous controls and oversight by Congress and courts. For the public, it comes down to a question of unverifiable trust.

“The constraints that I operate under are much more remarkable than the powers that I enjoy,” said the senior intelligence official who declined to be named.

When asked why the NSA could not release an unclassified copy of its “minimization procedures,” which are supposed to strip accidentally collected records of their identifying details, the official suggested a reporter submit a freedom-of-information request.

As for bulk collection of Internet metadata, the question that triggered the crisis of 2004, another official said the NSA is no longer doing it. When pressed on that question, he said he was speaking only of collections under authority of the surveillance court.

“I’m not going to say we’re not collecting any Internet metadata,” he added. “We’re not using this program and these kinds of accesses to collect Internet metadata in bulk.”

Julie Tate and Ellen Nakashima contributed to this report.

 

By Peter Wallstein, Carol D. Leonnig and Alice Crites

Wedged into a secure, windowless basement room deep below the Capitol Visitors Center, U.S. District Court Judge John Bates appeared before dozens of senators earlier this month for a highly unusual, top-secret briefing.

The lawmakers pressed Bates, according to people familiar with the session, to discuss the inner workings of the United States’ clandestine terrorism surveillance tribunal, which Bates oversaw from 2006 until earlier this year.

Bates had rarely spoken of his sensitive work. He reluctantly agreed to appear at the behest of Senate Intelligence Committee Chairman Dianne Feinstein (D-Calif.), who arranged the session after new disclosures that the court had granted the government broad access to millions of Americans’ telephone and Internet communications.

The two-hour meeting on June 13 featuring Bates and two top spy agency officials — prompted by reports days earlier by The Washington Post and Britain’s Guardian newspaper about the vast reach of the programs — reflects a new and uncomfortable reality for the Foreign Intelligence Surveillance Court and its previously obscure members. Within the past month, lawmakers have begun to ask who the court’s judges are, what they do, why they have almost never declined a government surveillance request and why their work is so secretive.

The public is getting a peek into the little-known workings of a powerful and mostly invisible government entity. And it is seeing a court whose secret rulings have in effect created a body of law separate from the one on the books — one that gives U.S. spy agencies the authority to collect bulk information about Americans’ medical care, firearms purchases, credit card usage and other interactions with business and commerce, according to Sen. Ron Wyden (D-Ore.).

“The government can get virtually anything,” said Wyden, who as a member of the Senate Intelligence Committee is allowed to read many of the court’s classified rulings. “Health, guns, credit cards — my reading is not what has been done, it’s what can be done.”

Members of Congress from both parties are pursuing legislation to force the court’s orders into the open and have stepped up demands that the Obama administration release at least summaries of the court’s opinions.

Critics, including some with knowledge of the court’s internal operations, say the court has undergone a disturbing shift. It was created in 1978 to handle routine surveillance warrants, but these critics say it is now issuing complex, classified, Supreme Court-style rulings that are quietly expanding the government’s reach into the private lives of unwitting Americans.

Surveillance court judges are selected from the pool of sitting federal judges by the chief justice of the United States, as is required by the law that established the panel. There is no additional confirmation process. Members serve staggered terms of up to seven years.

Typical federal courts are presided over by judges nominated by presidents and confirmed by the Senate. Cases are argued by two opposing sides; judges issue orders and opinions that can be read, analyzed and appealed; and appellate opinions set precedents that shape American jurisprudence.

The surveillance court is a different world of secret case law, non-adversarial proceedings, and rulings written by individual judges who rarely meet as a panel.

Judges generally confer only with government lawyers, and out of public view. Yet the judges have the power to interpret the Constitution and set long-lasting and far-reaching precedent on matters involving Americans’ rights to privacy and due process under the Fourth Amendment. And this fast-growing body of law is almost entirely out of view of legal scholars and the public. Most Americans do not have access to the judiciary’s full interpretation of the Constitution on matters of surveillance, search and seizure when it comes to snooping for terrorist plots — and are limited in their ability to challenge it.

All 11 of the current members were tapped by Chief Justice John G. Roberts Jr. Ten were originally appointed to the federal bench by Republican presidents. Six are former prosecutors.

“The judges that are assigned to this court are judges that are not likely to rock the boat,” said Nancy Gertner, a former federal judge from Massachusetts who teaches at Harvard Law School. Gertner, a former defense and civil rights lawyer named to the bench by Democrat Bill Clinton, added: “All of the structural pressures that keep a judge independent are missing there. It’s ­one-sided, secret, and the judges are chosen in a selection process by one man.”

Steven Aftergood, director of the government secrecy program at the Federation of American Scientists, called the court “an astonishing departure from what we thought we knew about the judiciary.”

Defending the court

Several current and former members of the court, as well as government officials, reject the criticism. They say internal checks are built into the system to ensure Americans’ rights are not violated.

The court’s current chief, D.C. District Court Judge Reggie B. Walton, was so perturbed about recent critiques of the court that he issued a rare public statement in the wake of newspaper reports about the court’s approval of the phone and Internet surveillance programs.

“The perception that the court is a rubber stamp is absolutely false,” Walton said. “There is a rigorous review process of applications submitted by the executive branch, spearheaded initially by five judicial branch lawyers who are national security experts and then by the judges, to ensure that the court’s authorizations comport with what the applicable statutes authorize.”

Administration officials echoed those sentiments last week during a public hearing before the House Intelligence Committee, telling lawmakers that the process of seeking approval for a new warrant takes extensive time and effort. The judges “push back a lot,” said Deputy Attorney General James Cole. “These are very thick applications that have a lot in them. And when they see anything that raises an issue, they will push back and say, ‘We need more information.’ ”

Roberts and an aide vet judges as candidates for the secret court. The contenders, who have undergone Senate confirmation for their original judicial posts, are screened again using an unusually exhaustive FBI background check that examines their lives “going back to birth,” according to a person with knowledge of the process. Candidates are told to withdraw if anything in their lives could prove embarrassing — the chief justice reads each FBI report. He has rejected candidates for traits such as excessive alcohol use, the person said.

The court was expanded from seven judges after the attacks of Sept. 11, 2001. At least three of the judges must live in the Washington area to ensure that a judge is always personally reachable by government officials in case of emergencies. Court members also continue to manage their regular dockets as district judges.

One of the most recent appointees, Judge Michael W. Mosman of Oregon, drew attention in 2008 when, in his position as a district court judge, he temporarily blocked a new state law allowing gay people to obtain domestic-partnership status.

Days after U.S. District Judge Rosemary M. Collyer’s March appointment to the secret court, her decision in a high-profile case involving government secrecy was overturned. She had ruled that the CIA could keep secret its list of drone targets, but a higher court overruled her.

Another member is Susan Webber Wright, the Arkansas judge who presided over the Paula Jones sexual-harassment suit against Clinton and famously held the president in contempt.

Walton is a former prosecutor who sentenced former Richard B. Cheney adviser I. Lewis “Scooter” Libby to more than two years in prison for his role in the Valerie Plame leak case. President George W. Bush later commuted Libby’s sentence.

Court officials reject suggestions that the judges reflect any partisan or ideological bent. They note that two former presiding judges — Joyce Hens Green and Colleen Kollar-Kotelly — were appointed to the federal bench by Democratic presidents. Neither is currently on the surveillance court.

Judges say they take the roles seriously.

“There’s no question that every judge who has ever served on this court has thought it was the most significant thing they’ve ever done as a judge,” U.S. District Judge Royce C. Lamberth said in a rare public interview on the subject posted on a federal court Web site in 2002. “When I did the hearings on the embassy bombings in Africa, we started the hearings in my living room at 3:00 in the morning. And some of the taps I did that night turned out to be very significant and were used in the New York trials of the people indicted for the bombings.”

Tensions have bubbled to the surface in recent days, with some of the court’s judges privately expressing frustration that it has become the center of attention and an object of criticism. They note that Congress helped pass the laws allowing the government’s broad spying powers and that the administration instructs the court to keep its inner workings secret.

Walton, who took over as chief earlier this year, issued an order last month demanding that the Obama administration respond to a request from a civil liberties group, the Electronic Frontier Foundation, for the release of a classified ruling in which the court found that the government had engaged in unconstitutional surveillance of Americans. The court has even taken the rare step over the past two weeks of creating a public docket Web page featuring the Electronic Frontier Foundation case as well as a separate, new motion brought by the American Civil Liberties Union seeking records of the phone surveillance program.

Bates’s June 13 appearance before lawmakers came after Feinstein, a staunch defender of the program, called Roberts to request that he dispatch Bates to the briefing. The session was open to all senators; 47 attended, according to someone familiar with the meeting.

Bates, a former prosecutor and Bush-appointed judge in the D.C. district court, rebuffed several questions about the court’s orders, telling senators they should address their questions to executive branch officials, according to people briefed on the session. He stressed that the government’s collection and surveillance programs were classified as top-secret by the Obama administration, not by the judiciary.

Still, the government almost always gets much of what it wants from the court.

In 2012, the court received 1,789 requests for electronic surveillance, according to the annual report it files with the Senate. One was withdrawn. The rest were approved, sometimes after back-and-forth interactions in which judges required the government to tweak or scale back its plans. Significant opinions in recent years have been sent to congressional intelligence committee members but remain classified.

‘Expansive’ rulings

Now, outside critics, lawmakers and some with internal knowledge of the court are starting to push for an overhaul.

Wyden said the surveillance court has issued “pretty stunning rulings, rulings that I think are about as expansive as anything you can imagine.”

Wyden pointed to court orders authorizing collection of bulk phone data, which The Post reported had dated to 2006, as indicators of the court’s broad view of government powers. At issue is a provision of the Patriot Act, passed by Congress after the Sept. 11 attacks, which permitted the FBI to compel the production of “business records” deemed relevant to terrorism and espionage investigations and to share those with intelligence officials.

Those orders followed a turbulent time for the secret court. Some judges were outraged that they had not been aware of the Bush administration’s warrantless wiretapping operation, which was first reported by the New York Times in late 2005. One member of the panel, U.S. District Judge James Robertson, resigned in protest, confiding to colleagues that he was concerned the program may have been illegal and could have tainted the court’s work.

One person close to the court, speaking on the condition of anonymity to discuss the secretive body, said the newly revealed orders indicate a shift in which the court blesses the bulk collection of Americans’ communications data to make investigations easier rather than weighing the merits of violating the privacy of one person on a case-by-case basis. Before this change, the person said, “it was one warrant at a time.”

The court’s under-the-radar approach proved a particular challenge this spring to the Electronic Frontier Foundation when it sought to file its motion seeking release of the prior finding of the unlawful government surveillance. It turned out that the mere act of finding the court proved a steep hurdle.

Repeated calls to the court clerk from the foundation went unreturned, said David Sobel, an attorney for the group. The group wound up submitting the motion through a staffer at the Justice Department, whose officials were actively opposing the group’s efforts.

“We never had any direct contact with the court,” Sobel said, “and the other party in the proceeding was the gatekeeper.”

Chief Justice Roberts himself signaled some discomfort with the system during his 2005 confirmation hearings.

“I’ll be very candid,” he told senators. “When I first learned about the FISA court, I was surprised. It’s not what we usually think of when we think of a court. We think of a place where we can go, we can watch, the lawyers argue, and it’s subject to the glare of publicity. And the judges explain their decision to the public and they can examine them. That’s what we think of as a court.”

NO 'COORDINATION' WITH EXECUTIVE

Special court held NSA to account, jurist says

By Carol D. Leonnig, Ellen Nakashima and Barton Gellman

Recent leaks of classified documents have pointed to the role of a special court in enabling the government’s secret surveillance programs, but members of the court are chafing at the suggestion that they were collaborating with the executive branch.

A classified 2009 draft report by the National Security Agency’s inspector general relayed some details about the interaction between the court’s judges and the NSA, which sought approval for the Bush administration’s top-secret domestic surveillance programs. The report was described in The Washington Post on June 16 and released in full Thursday by The Post and the British newspaper the Guardian.

U.S. District Judge Colleen Kollar-Kotelly, the former chief judge of the Foreign Intelligence Surveillance Court, took the highly unusual step Friday of voicing open frustration at the account in the report and court’s inability to explain its decisions.

“In my view, that draft report contains major omissions, and some inaccuracies, regarding the actions I took as Presiding Judge of the FISC and my interactions with Executive Branch officials,” Kollar-Kotelly said in a statement to The Post. It was her first public comment describing her work on the intelligence court.

The inspector general’s draft report is among the many documents leaked by former NSA contractor Edward Snowden, touching off a roiling national debate about the proper balance between the government’s reach into Americans’ lives and the effort to protect the nation in the Internet age.

The document portrays the surveillance court as “amenable” to the government’s legal theory to “re-create” authority for the Internet metadata program that had initially been authorized by President George W. Bush without court or congressional approval. The program was shut down in March 2004 when acting Attorney General James B. Comey and senior leaders at the Justice Department threatened to resign over what they felt was an illegal program.

Kollar-Kotelly disputed the NSA report’s suggestion of a fairly high level of coordination between the court and the NSA and Justice in 2004 to re-create certain authorities under the Foreign Intelligence Surveillance Act, the 1978 law that created the court in response to abuses of domestic surveillance in the 1960s and 1970s.

“That is incorrect,” she said. “I participated in a process of adjudication, not ‘coordination’ with the executive branch. The discussions I had with executive branch officials were in most respects typical of how I and other district court judges entertain applications for criminal wiretaps under Title III, where issues are discussed ex parte.”

The perception that the court works too closely with the government arises in large part from the tribunal’s “ex parte” nature, which means that unlike in a traditional court, there is no legal sparring between adversaries with the judge as arbiter. Instead, a Justice Department official makes the case for the government agency seeking permission to carry out surveillance inside the United States. No one speaks for the target of the surveillance or the company that is ordered to allow its networks to be tapped or to turn over its customers’ data.

Some critics say the court is a rubber stamp for government investigators because it almost never has turned down a warrant application. However, that high batting average doesn’t take into account changes the court requires in some requests and other applications that the government withdraws.

For about 30 years, the court was located on the sixth floor of the Justice Department’s headquarters, down the hall from the officials who would argue in front of it. (The court moved to the District’s federal courthouse in 2009.) “There is a collaborative process that would be unnatural in the public, criminal court setting,” said a former Justice official familiar with the court.

Kollar-Kotelly, who was the court’s chief judge from 2002 to 2006, said she could not comment further on the matter because “the underlying subjects” in the report generally remain classified by the executive branch.

Other judges on the court have confided to colleagues their frustration at the court’s portrayal, according to people familiar with their discussion.

The inspector general’s report, combined with persistent refusals by the government to declassify the opinions, have left the public in the dark about the court’s legal justifications for approving the broad surveillance programs.

“The court is a neutral party, not a collaborator or arm of the government,” said one government official close to the court. “But the information out there now leaves people wondering how and why the court endorsed these programs.”

The court historically has authorized in secret hearings classified warrants to wiretap the calls and monitor the movements of suspected criminals. After the terrorist attacks of Sept. 11, 2001, far-reaching programs to gather Internet and telephone content and metadata were launched under presidential authority, without congressional action or approval from the surveillance court.

The Internet metadata portion of that program had to be revamped after Comey and other Justice officials threatened to resign. Metadata are information indicating facts such as an e-mail’s sender and recipient and its time and date, but not its content.

In May 2004, the NSA briefed Kollar-Kotelly on the technical aspects of that program’s collection, according to the report. She also met with the NSA director, Lt. Gen. Michael V. Hayden, on two successive Saturdays during the summer of 2004 to discuss the issue, the report said.

“It was very professional,” Hayden said in an interview. “We of course had to explain to her what it was we had been doing, what it was we wanted to do, how we would do it, what kind of safeguards we felt able to put in. We left it to her judgment whether there was proportionality in terms of was this worth doing, in the balance between security and liberty.”

He said in response to her concerns, the agency made some technical adjustments so that “the odds were greater that you’d pick up fewer protected communications of U.S. persons.”

Said Hayden: “She wasn’t in league with us. We were down there presenting what we thought was appropriate.”

On July 14, 2004, the surveillance court for the first time approved the gathering of information by the NSA, which created the equivalent of a digital vault to hold Internet metadata. Kollar-Kotelly’s order authorized the metadata program under a FISA provision known as the “pen register/trap and trace,” or PRTT.

The ruling was a secret not just to the public and most of Congress, but to all of Kollar-Kotelly’s surveillance court colleagues. Under orders from the president, none of the court’s other 10 members could be told about the Internet metadata program, which was one prong of a larger and highly classified data-gathering effort known as the President’s Surveillance Program, or PSP.

But the importance of her order — which approved the collection based on a 1986 law typically used for phone records — was hard to overstate.

“The order essentially gave NSA the same authority to collect bulk Internet metadata that it had under the PSP,” the inspector general’s report said, with some minor caveats including reducing the number of people who could access the records.

On May 24, 2006, Kollar-Kotelly signed another order, this one authorizing the bulk collection of phone metadata from U.S. phone companies, under a FISA provision known as Section 215, or the ”business records provision,” of the USA Patriot Act.

As with the PRTT order, the Justice Department and NSA “collaboratively designed the application, prepared declarations and responded to questions from court advisers,” the inspector general’s report said. “Their previous experience in drafting the PRTT order made this process more efficient.”

The court also agreed in 2007 to permit the government to collect the content of e-mails and phone calls to and from the United States when “there is probable cause to believe” that one of the parties is a member of al-Qaeda or an associated terrorist group. That program, known today as PRISM and described in documents obtained by The Washington Post, eventually was authorized by Congress.

Kollar-Kotelly could be a stern taskmaster when she thought the NSA was overstepping its bounds. In 2004, she temporarily shut down the government’s surveillance program when she learned of a key NSA failure, The Post reported in 2006. The agency was not properly walling off information gained in warrantless surveillance and may have been using the information to obtain court warrants, which was forbidden. In 2005, the problem resurfaced and she issued a strong warning to the government that it had to fix the problem or would face trouble obtaining court warrants.

Kollar-Kotelly “understood the problems that the government, particularly the Defense Department and the intelligence community, were facing in trying to keep this country safe,” said Robert L. Deitz, former NSA general counsel under Hayden.

But, he said, the court was no rubber stamp. “The judges ask searching questions,” he said. “If they don’t get the right answer, they don’t stamp things ‘reject.’ They say, ‘I’m not signing this.’ Then we go back and say, ‘Okay, we’ve got to do this the following way.’ ”

Still secret are the 2004 decision accompanying the PRTT court order and the legal opinion accompanying the 2006 business records order.

A former senior Justice Department official, who spoke on the condition of anonymity because of the subject’s sensitivity, said he believes the government should consider releasing declassified summaries of relevant opinions.

“I think it would help” quell the “furor” raised by the recent disclosures, he said. “In this current environment, you may have to lean forward a little more in declassifying stuff than you otherwise would. You might be able to prepare reasonable summaries that would be helpful to the American people.”

Lawmakers and civil-liberties advocates have been pushing the Obama administration for several years to declassify these opinions and other opinions from Justice’s Office of Legal Counsel that explain the legal justification for these programs.

The Office of the Director of National Intelligence has led an effort to review these opinions to see what, if anything, can be declassified. But Robert S. Litt, ODNI general counsel, has argued that declassification can be difficult when so much of the legal reasoning is intertwined with facts that need to remain secret lest they tip off enemies about surveillance methods.

Still, the former official explained, segregating relevant facts from classified material is routinely done in criminal proceedings under the Classified Information Procedures Act. In those cases, the government can extract the information that is relevant to the defense, the judge approves it, and it is provided to the defense.

“This is not unheard-of in the unclassified world, and some kind of summary document can be generated,” the former official said. “Maybe that’s a middle ground that can be done.”

Sari Horwitz contributed to this report.

U.S. officials have parsed, hedged and misstated facts

By Greg Miller

Amid the cascading disclosures about National Security Agency surveillance programs, the top lawyer in the U.S. intelligence community opened his remarks at a rare public appearance last week with a lament about how much of the information being spilled was wrong.

“A lie can get halfway around the world before the truth gets its boots on,” said Robert Litt, citing a line often attributed to Mark Twain. “Unfortunately, there’s been a lot of misinformation that’s come out about these programs.”

The remark by Litt, general counsel for the Office of the Director of National Intelligence, was aimed at news organizations. But details that have emerged from the exposure of hundreds of pages of previously classified NSA documents indicate that public assertions about these programs by senior U.S. officials have also often been misleading, erroneous or simply false.

The same day Litt spoke, the NSA quietly removed from its Web site a fact sheet about its collection activities because it contained inaccuracies discovered by lawmakers.

A week earlier, President Obama, in a television interview, asserted that oversight of the surveillance programs was “transparent” because of the involvement of a special court, even though that court’s sessions and decisions are sealed from the public. “It is transparent,” Obama said of the oversight process. “That’s why we set up the FISA court.”

A remark by Litt’s boss, Director of National Intelligence James R. Clapper Jr., has perhaps drawn the most attention. Asked during a congressional hearing in March whether the NSA collected data on millions of Americans, Clapper replied, “No, sir.”

U.S. officials have cited a variety of factors to explain the discrepancies, including the challenge of speaking publicly and definitively about programs that remain classified and involve procedures and technical systems that are highly complex.

Jane Harman, a former ranking Democrat on the House Intelligence Committee, said that speaking about secret programs can be a “minefield” for public officials.

“Are people deliberately misleading other people? I suppose it can happen,” Harman said in an interview. Facts can be obscured through “selective declassification that means you put out some pieces but not others,” she said. “But I assume most people are acting in good faith.”

Acknowledging the “heated controversy” over his remark, Clapper sent a letter to the Senate Intelligence Committee on June 21 saying that he had misunderstood the question he had been asked.

“I have thought long and hard to re-create what went through my mind at the time,” Clapper said in the previously undisclosed letter. “My response was clearly erroneous — for which I apologize.”

Beyond inadvertent missteps, however, an examination of public statements over a period of years suggests that officials have often relied on legalistic parsing and carefully hedged characterizations in discussing the NSA’s collection of communications.

Obama’s assurances have hinged, for example, on a term — targeting — that has a specific meaning for U.S. spy agencies that would elude most ordinary citizens.

“What I can say unequivocally is that if you are a U.S. person, the NSA cannot listen to your telephone calls and the NSA cannot target your e-mails,” Obama said in his June 17 interview on PBS’s “Charlie Rose Show.”

But even if it is not allowed to target U.S. citizens, the NSA has significant latitude to collect and keep the contents of e-mails and other communications of U.S. citizens that are swept up as part of the agency’s court-approved monitoring of a target overseas.

The law allows the NSA to examine such messages and share them with other agencies if it determines that the information contained is evidence of a crime, conveys a serious threat or is necessary to understand foreign intelligence.

The threshold for scrutinizing other data not regarded as content but still potentially revealing is lower than it is for the contents of communications. A 2009 report by the NSA inspector general and obtained by The Washington Post indicates that the agency for years examined metadata on e-mails flowing into and out of the United States, including “the sender and recipient e-mail addresses.”

President George W. Bush at times engaged in similarly careful phrasing to defend surveillance programs in the years after the Sept. 11, 2001, attacks. In 2004, while calling for renewal of the Patriot Act, Bush sought to assuage critics by saying “the government can’t move on wiretaps or roving wiretaps without getting a court order.”

At the time, it had not yet been publicly disclosed that Bush had secretly authorized NSA surveillance of communications between U.S. residents and contacts overseas while bypassing the Foreign Intelligence Surveillance Court.

When the wiretapping operation was exposed in the news media two years later, Bush defended it as a program “that listens to a few numbers, called from outside of the United States, and of known al-Qaeda or affiliate people.” Subsequent revelations have made clear that the scope was far greater than his words would suggest.

News accounts of the NSA programs have also contained inaccuracies, in some cases because of the source materials. Classified NSA slides that were published by The Post indicated that the NSA was able to tap directly into the servers of Google, Microsoft, Apple and other technology companies. The companies denied that they allowed direct access to their equipment, although they did not dispute that they cooperated with the NSA.

Current and former U.S. officials have defended the programs, and some have called for greater transparency as a way of allaying concerns.

“I’m convinced, the more the American people know exactly what it is we are doing in this balance between privacy and security — the more they know, the more comfortable they will feel,” Michael V. Hayden, former director of the NSA and CIA, told “Face the Nation” on Sunday. “Frankly, I think we ought to be doing a bit more to explain what it is we’re doing, why, and the very tight safeguards under which we’re operating.”

For now, the crumbling secrecy surrounding the programs has underscored the extent to which obscuring their dimensions had served government interests beyond the importance of the intelligence they produced.

Secret court rulings that allowed the NSA to gather phone records enabled the spy service to assemble a massive database on Americans’ phone records without public debate or the risk of political blowback.

The binding secrecy built into the PRISM program of tracking international e-mail allowed the NSA to compel powerful technology companies to comply with requests for information about their users while keeping them essentially powerless to protest.

The careful depiction of NSA programs also served diplomatic ends. Until recently, the United States had positioned itself as such an innocent victim of cyber intrusions by Russia and China that the State Department issued a secret demarche, or official diplomatic communication, in January scolding Beijing. That posture became more problematic after leaks by the former NSA contractor and acknowledged source of the NSA leaks, Edward Snowden, who fled to Hong Kong and is thought to be stuck at Sheremetyevo International Airport in Moscow.

Clapper’s testimony before the Senate Intelligence Committee in March has drawn comparisons to other cases in which U.S. intelligence officials faced, under oath, questions that to answer truthfully would require exposing a classified program.

In 1973, then-CIA Director Richard Helms denied agency involvement in CIA operations in Chile, a falsehood that led to him pleading no contest four years later to misdemeanor charges of misleading Congress.

There is no indication that lawmakers have contemplated pursuing such a course against Clapper, in part because he subsequently corrected his claim, although there is disagreement over how quickly he did so.

Sen. Ron Wyden (D-Ore.), who had asked Clapper the question about information collection on Americans, said in a recent statement that the director had failed to clarify the remark promptly despite being asked to do so. Clapper disputed that in his note to the committee, saying his “staff acknowledged the error to Senator Wyden’s staff soon after the hearing.”

In early June, after the NSA leaks had brought renewed attention to Clapper’s “No, sir,” Clapper cited the difficulty of answering a question about a classified program and said in an interview on NBC News that he had responded in the “least most untruthful manner.”

He made a new attempt to explain the exchange in his June 21 correspondence, which included a hand-written note to Wyden saying that an attached letter was addressed to the committee chairman but that he “wanted [Wyden] to see this first.”

Clapper said he thought Wyden was referring to NSA surveillance of e-mail traffic involving overseas targets, not the separate program in which the agency is authorized to collect records of Americans’ phone calls that include the numbers and duration of calls but not individuals’ names or the contents of their calls.

Referring to his appearances before Congress over several decades, Clapper concluded by saying that “mistakes will happen, and when I make one, I correct it.”

Julie Tate contributed to this report.

 

CONGRESS GAVE BROAD POWERS IN 2008

Many violations involve spying on Americans

By Barton Gellman

The National Security Agency has broken privacy rules or overstepped its legal authority thousands of times each year since Congress granted the agency broad new powers in 2008, according to an internal audit and other top-secret documents.

Most of the infractions involve unauthorized surveillance of Americans or foreign intelligence targets in the United States, both of which are restricted by statute and executive order. They range from significant violations of law to typographical errors that resulted in unintended interception of U.S. e-mails and telephone calls.

The documents, provided earlier this summer to The Washington Post by former NSA contractor Edward Snowden, include a level of detail and analysis that is not routinely shared with Congress or the special court that oversees surveillance. In one of the documents, agency personnel are instructed to remove details and substitute more generic language in reports to the Justice Department and the Office of the Director of National Intelligence.

In one instance, the NSA decided that it need not report the unintended surveillance of Americans. A notable example in 2008 was the interception of a “large number” of calls placed from Washington when a programming error confused the U.S. area code 202 for 20, the international dialing code for Egypt, according to a “quality assurance” review that was not distributed to the NSA’s oversight staff.

In another case, the Foreign Intelligence Surveillance Court, which has authority over some NSA operations, did not learn about a new collection method until it had been in operation for many months. The court ruled it unconstitutional.

The Obama administration has provided almost no public information about the NSA’s compliance record. In June, after promising to explain the NSA’s record in “as transparent a way as we possibly can,” Deputy Attorney General James Cole described extensive safeguards and oversight that keep the agency in check. “Every now and then, there may be a mistake,” Cole said in congressional testimony.

The NSA audit obtained by The Post, dated May 2012, counted 2,776 incidents in the preceding 12 months of unauthorized collection, storage, access to or distribution of legally protected communications. Most were unintended. Many involved failures of due diligence or violations of standard operating procedure. The most serious incidents included a violation of a court order and unauthorized use of data about more than 3,000 Americans and green-card holders.

In a statement in response to questions for this article, the NSA said it attempts to identify problems “at the earliest possible moment, implement mitigation measures wherever possible, and drive the numbers down.” The government was made aware of The Post’s intention to publish the documents that accompany this article online.

“We’re a human-run agency operating in a complex environment with a number of different regulatory regimes, so at times we find ourselves on the wrong side of the line,” a senior NSA official said in an interview, speaking with White House permission on the condition of anonymity.

“You can look at it as a percentage of our total activity that occurs each day,” he said. “You look at a number in absolute terms that looks big, and when you look at it in relative terms, it looks a little different.”

There is no reliable way to calculate from the number of recorded compliance issues how many Americans have had their communications improperly collected, stored or distributed by the NSA.

The causes and severity of NSA infractions vary widely. One in 10 incidents is attributed to a typographical error in which an analyst enters an incorrect query and retrieves data about U.S phone calls or e-mails.

But the more serious lapses include unauthorized access to intercepted communications, the distribution of protected content and the use of automated systems without built-in safeguards to prevent unlawful surveillance.

The May 2012 audit, intended for the agency’s top leaders, counts only incidents at the NSA’s Fort Meade headquarters and other ­facilities in the Washington area. Three government officials, speak­ing on the condition of anonymity to discuss classified matters, said the number would be substantially higher if it included other NSA operating units and regional collection centers.

Senate Intelligence Committee Chairman Dianne Feinstein (D-Calif.), who did not receive a copy of the 2012 audit until The Post asked her staff about it, said in a statement late Thursday that the committee “can and should do more to independently verify that NSA’s operations are appropriate, and its reports of compliance incidents are accurate.”

Despite the quadrupling of the NSA’s oversight staff after a series of significant violations in 2009, the rate of infractions increased throughout 2011 and early 2012. An NSA spokesman declined to disclose whether the trend has continued since last year.

One major problem is largely unpreventable, the audit says, because current operations rely on technology that cannot quickly determine whether a foreign mobile phone has entered the United States.

In what appears to be one of the most serious violations, the NSA diverted large volumes of international data passing through fiber-optic cables in the United States into a repository where the material could be stored temporarily for processing and selection.

The operation to obtain what the agency called “multiple communications transactions” collected and commingled U.S. and foreign e-mails, according to an article in SSO News, a top-secret internal newsletter of the NSA’s Special Source Operations unit. NSA lawyers told the court that the agency could not practicably filter out the communications of Americans.

In October 2011, months after the program got underway, the Foreign Intelligence Surveillance Court ruled that the collection effort was unconstitutional. The court said that the methods used were “deficient on statutory and constitutional grounds,” according to a top-secret summary of the opinion, and it ordered the NSA to comply with standard privacy protections or stop the program.

James R. Clapper Jr., the director of national intelligence, has acknowledged that the court found the NSA in breach of the Fourth Amendment, which prohibits unreasonable searches and seizures, but the Obama administration has fought a Freedom of Information lawsuit that seeks the opinion.

Generally, the NSA reveals nothing in public about its errors and infractions. The unclassified versions of the administration’s semiannual reports to Congress feature blacked-out pages under the headline “Statistical Data Relating to Compliance Incidents.”

Members of Congress may read the unredacted documents, but only in a special secure room, and they are not allowed to take notes. Fewer than 10 percent of lawmakers employ a staff member who has the security clearance to read the reports and provide advice about their meaning and significance.

The limited portions of the reports that can be read by the public acknowledge “a small number of compliance incidents.”

Under NSA auditing guidelines, the incident count does not usually disclose the number of Americans affected.

“What you really want to know, I would think, is how many innocent U.S. person communications are, one, collected at all, and two, subject to scrutiny,” said Julian Sanchez, a research scholar and close student of the NSA at the Cato Institute.

The documents provided by Snowden offer only glimpses of those questions. Some reports make clear that an unauthorized search produced no records. But a single “incident” in February 2012 involved the unlawful retention of 3,032 files that the surveillance court had ordered the NSA to destroy, according to the May 2012 audit. Each file contained an undisclosed number of telephone call records.

One of the documents sheds new light on a statement by NSA Director Keith B. Alexander last year that “we don’t hold data on U.S. citizens.”

Some Obama administration officials, speaking on the condition of anonymity, have defended Alexander with assertions that the agency’s internal definition of “data” does not cover “metadata” such as the trillions of American call records that the NSA is now known to have collected and stored since 2006. Those records include the telephone numbers of the parties and the times and durations of conversations, among other details, but not their content or the names of callers.

The NSA’s authoritative def­inition of data includes those call records. “Signals Intelligence Management Directive 421,” which is quoted in secret oversight and auditing guidelines, states that “raw SIGINT data . . . includes, but is not limited to, unevaluated and/or unminimized transcripts, gists, facsimiles, telex, voice, and some forms of computer-generated data, such as call event records and other Digital Network Intelligence (DNI) metadata as well as DNI message text.”

In the case of the collection effort that confused calls placed from Washington with those placed from Egypt, it is unclear what the NSA meant by a “large number” of intercepted calls. A spokesman declined to discuss the matter.

The NSA has different reporting requirements for each branch of government and each of its legal authorities. The “202” collection was deemed irrelevant to any of them. “The issue pertained to Metadata ONLY so there were no defects to report,” according to the author of the secret memo from March 2013.

The large number of database query incidents, which involve previously collected communications, confirms long-standing suspicions that the NSA’s vast data banks — with code names such as MARINA, PINWALE and XKEYSCORE — house a considerable volume of information about Americans. Ordinarily the identities of people in the United States are masked, but intelligence “customers” may request unmasking, either one case at a time or in standing orders.

In dozens of cases, NSA personnel made careless use of the agency’s extraordinary powers, according to individual auditing reports. One team of analysts in Hawaii, for example, asked a system called DISHFIRE to find any communications that mentioned both the Swedish manufacturer Ericsson and “radio” or “radar” — a query that could just as easily have collected on people in the United States as on their Pakistani military target.

The NSA uses the term “incidental” when it sweeps up the records of an American while targeting a foreigner or a U.S. person who is believed to be involved in terrorism. Official guidelines for NSA personnel say that kind of incident, pervasive under current practices, “does not constitute a . . . violation” and “does not have to be reported” to the NSA inspector general for inclusion in quarterly reports to Congress. Once added to its databases, absent other restrictions, the communications of Americans may be searched freely.

In one required tutorial, NSA collectors and analysts are taught to fill out oversight forms without giving “extraneous information” to “our FAA overseers.” FAA is a reference to the FISA Amendments Act of 2008, which granted broad new authorities to the NSA in exchange for regular audits from the Justice Department and the Office of the Director of National Intelligence and periodic reports to Congress and the surveillance court.

Using real-world examples, the “Target Analyst Rationale Instructions” explain how NSA employees should strip out details and substitute generic descriptions of the evidence and analysis behind their targeting choices.

“I realize you can read those words a certain way,” said the high-ranking NSA official who spoke with White House authority, but the instructions were not intended to withhold information from auditors. “Think of a book of individual recipes,” he said. Each target “has a short, concise description,” but that is “not a substitute for the full recipe that follows, which our overseers also have access to.”

Julie Tate and Carol D. Leonnig contributed to this report.

Top-secret summary details U.S. spy network’s successes, failures and objectives

By Barton Gellman and Greg Miller

U.S. spy agencies have built an intelligence-gathering colossus since the attacks of Sept. 11, 2001, but remain unable to provide critical information to the president on a range of national security threats, according to the government’s top-secret budget.

The $52.6 billion “black budget” for fiscal 2013, obtained by The Washington Post from former ­intelligence contractor Edward Snowden, maps a bureaucratic and operational landscape that has never been subject to public scrutiny. Although the government has annually released its overall level of intelligence spending since 2007, it has not divulged how it uses the money or how it performs against the goals set by the president and Congress.

The 178-page budget summary for the National Intelligence Program details the successes, failures and objectives of the 16 spy agencies that make up the U.S. intelligence community, which has 107,035 employees.

The summary describes cutting-edge technologies, agent recruiting and ongoing operations. The Post is withholding some information after consultation with U.S. officials who expressed concerns about the risk to intelligence sources and methods. Sensitive details are so pervasive in the documents that The Post is publishing only summary tables and charts online.

“The United States has made a considerable investment in the Intelligence Community since the terror attacks of 9/11, a time which includes wars in Iraq and Afghanistan, the Arab Spring, the proliferation of weapons of mass destruction technology, and asymmetric threats in such areas as cyber-warfare,” Director of National Intelligence James R. Clapper Jr. wrote in response to inquiries from The Post.

“Our budgets are classified as they could provide insight for foreign intelligence services to discern our top national priorities, capabilities and sources and methods that allow us to obtain information to counter threats,” he said.

Among the notable revelations in the budget summary:

●Spending by the CIA has surged past that of every other spy agency, with $14.7 billion in requested funding for 2013. The figure vastly exceeds outside estimates and is nearly 50 percent above that of the National Security Agency, which conducts eavesdropping operations and has long been considered the behemoth of the community.

●The CIA and the NSA have begun aggressive new efforts to hack into foreign computer networks to steal information or sabotage enemy systems, embracing what the budget refers to as “offensive cyber operations.”

●Long before Snowden’s leaks, the U.S. intelligence community worried about “anomalous behavior” by employees and contractors with access to classified material. The NSA planned to ward off a “potential insider compromise of sensitive information” by re-investigating at least 4,000 people this year who hold high-level security clearances.

●U.S. intelligence officials take an active interest in friends as well as foes. Pakistan is described in detail as an “intractable target,” and counterintelligence operations “are strategically focused against [the] priority targets of China, Russia, Iran, Cuba and Israel.” The latter is a U.S. ally but has a history of espionage attempts against the United States.

●In words, deeds and dollars, intelligence agencies remain fixed on terrorism as the gravest threat to national security, which is listed first among five “mission ob­jectives.” Counterterrorism programs employ one in four members of the intelligence workforce and account for one-third of the intelligence program’s spending.

●The governments of Iran, China and Russia are difficult to penetrate, but North Korea’s may be the most opaque. There are five “critical” gaps in U.S. intelligence about Pyongyang’s nuclear and missile programs, and analysts know virtually nothing about the intentions of North Korean leader Kim Jong Un.

Formally known as the Congressional Budget Justification for the National Intelligence Program, the “top-secret” blueprint represents spending levels proposed to the House and Senate intelligence committees in February 2012. Congress may have made changes before the fiscal year began on Oct 1. Clapper is expected to release the actual total spending figure after the fiscal year ends on Sept. 30.

The document describes a constellation of spy agencies that track millions of surveillance targets and carry out operations that include hundreds of lethal strikes. They are organized around five priorities: combating terrorism, stopping the spread of nuclear and other unconventional weapons, warning U.S. leaders about critical events overseas, defending against foreign espionage, and conducting cyber-operations.

In an introduction, Clapper said the threats facing the United States “virtually defy rank-ordering.” He warned of “hard choices” as the intelligence community — sometimes referred to as the “IC” — seeks to rein in spending after a decade of often double-digit budget increases.

The current budget proposal envisions that spending will remain roughly level through 2017 and amounts to a case against substantial cuts.

“Never before has the IC been called upon to master such complexity and so many issues in such a resource-constrained environment,” Clapper wrote.

An espionage empire

The summary provides a detailed look at how the U.S. intelligence community has been reconfigured by the massive infusion of resources that followed the 2001 attacks. The United States has spent more than $500 billion on intelligence during that period, an outlay that U.S. officials say has succeeded in its main objective: preventing another catastrophic terrorist attack in the United States.

The result is an espionage empire with resources and a reach beyond those of any adversary, sustained even now by spending that rivals or exceeds the levels at the height of the Cold War.

The current total budget request was 2.4 percent below that of fiscal 2012. In constant dollars, it was about twice the estimated size of the 2001 budget and 25 percent above that of 2006, five years into what was then known as the “global war on terror.”

Historical data on U.S. intelligence spending is largely nonexistent. Through extrapolation, experts have estimated that Cold War spending probably peaked in the late 1980s at an amount that would be the equivalent of $71 billion today.

Spending in the most recent cycle surpassed that amount, based on the $52.6 billion detailed in documents obtained by The Post plus a separate $23 billion devoted to intelligence programs that more directly support the U.S. military.

Lee H. Hamilton, an Indiana Democrat who chaired the House Intelligence Committee and co-chaired the commission that investigated the Sept. 11 attacks, said that access to budget details will enable an informed public debate on intelligence spending for the first time, much as Snowden’s disclosures of NSA surveillance programs brought attention to operations that had assembled data on nearly every U.S. citizen.

“Much of the work that the intelligence community does has a profound impact on the life of ordinary Americans, and they ought not to be excluded from the process,” Hamilton said.

“Nobody is arguing that we should be so transparent as to create dangers for the country,” he said. But, he added, “there is a mind-set in the national security community: ‘Leave it to us, we can handle it, the American people have to trust us.’ They carry it to quite an extraordinary length so that they have resisted over a period of decades transparency. . . . The burden of persuasion as to keeping something secret should be on the intelligence community, the burden should not be on the American public.”

Experts said that access to such details about U.S. spy programs is without precedent.

“It was a titanic struggle just to get the top-line budget number disclosed, and that has only been done consistently since 2007,” said Steven Aftergood, an expert at the Federation of American Scientists, a Washington-based organization that provides analyses of national security issues. “But a real grasp of the structure and operations of the intelligence bureaucracy has been totally beyond public reach. This kind of material, even on a historical basis, has simply not been available.”

The only meaningful frame of reference came in 1994, when a congressional subcommittee inadvertently published a partial breakdown of the National Intelligence Program. At the time, the CIA accounted for just $4.8 billion of a budget that totaled $43.4 billion in 2012 dollars. The NSA and the National Reconnaissance Office, which operates satellites and other sensors, commanded far larger shares of U.S. intelligence budgets until years after the Cold War ended.

During the past decade, they have taken a back seat to the CIA.

The NSA was in line to receive $10.5 billion in 2013, and the NRO was to get $10.3 billion — both far below the CIA, whose share had surged to 28 percent of the total budget.

Overall, the U.S. government spends 10 times as much on the Defense Department as it does on spy agencies.

“Today’s world is as fluid and unstable as it has been in the past half century,” Clapper said in his statement to The Post. “Even with stepped up spending on the IC over the past decade, the United States currently spends less than one percent of GDP on the Intelligence Community.”

Dominant position

The CIA’s dominant position is likely to stun outside experts. It represents a remarkable recovery for an agency that seemed poised to lose power and prestige after acknowledging intelligence failures leading up to the 2001 attacks and the 2003 U.S.-led invasion of Iraq.

The surge in resources for the agency funded secret prisons, a controversial interrogation program, the deployment of lethal drones and a huge expansion of its counterterrorism center. The agency was transformed from a spy service struggling to emerge from the Cold War into a paramilitary force.

The CIA has devoted billions of dollars to recruiting and training a new generation of case officers, with the workforce growing from about 17,000 a decade ago to 21,575 this year.

The agency’s budget allocates $2.3 billion for human intelligence operations and $2.5 billion to cover the cost of supporting the security, logistics and other needs of those missions around the world. A relatively small amount of that total, $68.6 million, was earmarked for creating and maintaining “cover,” the false identities employed by operatives overseas.

There is no specific entry for the CIA’s fleet of armed drones in the budget summary, but a broad line item hints at the dimensions of the agency’s expanded paramilitary role, providing more than $2.6 billion for “covert action programs” that would include drone operations in Pakistan and Yemen, payments to militias in Afghanistan and Africa, and attempts to sabotage Iran’s nuclear program.

The black budget illuminates for the first time the intelligence burden of the wars in Afghanistan and Iraq. For 2013, U.S. spy agencies were projected to spend $4.9 billion on “overseas contingency operations.” The CIA accounted for about half of that figure, a sum factored into its overall $14.7 billion budget.

Those war expenditures are projected to shrink as the United States withdraws forces from Afghanistan. The budget also indicates that the intelligence community has cut the number of contractors it hires over the past five years by about 30 percent.

Critical gaps

Despite the vast outlays, the budget blueprint catalogues persistent and in some cases critical blind spots.

Throughout the document, U.S. spy agencies attempt to rate their efforts in tables akin to report cards, generally citing progress but often acknowledging that only a fraction of their questions could be answered — even on the community’s foremost priority, counterterrorism.

In 2011, the budget assessment says intelligence agencies made at least “moderate progress” on 38 of their 50 top counterterrorism gaps, the term used to describe blind spots. Several concern Lebanon’s Hezbollah movement, an enemy of Israel that has not attacked U.S. interests directly since the 1990s.

Other blank spots include questions about the security of Pakistan’s nuclear components when they are being transported, the capabilities of China’s next-generation fighter aircraft, and how Russia’s government leaders are likely to respond to “potentially destabilizing events in Moscow, such as large protests and terrorist attacks.”

A chart outlining efforts to address key questions on biological and chemical weapons is particularly bleak. U.S. agencies set annual goals for at least five categories of intelligence collection related to these weapons. In 2011, the agencies made headway on just two gaps; a year earlier, the mark was zero.

The documents describe expanded efforts to “collect on Russian chemical warfare countermeasures” and assess the security of biological and chemical laboratories in Pakistan.

A table of “critical” gaps listed five for North Korea, more than for any other country that has pursued or is pursuing a nuclear bomb.

The intelligence community seems particularly daunted by the emergence of “homegrown” terrorists who plan attacks in the United States without direct support or instruction from abroad, a threat realized this year, after the budget was submitted, in twin bombings at the Boston Marathon.

The National Counterterrorism Center has convened dozens of analysts from other agencies in attempts to identify “indicators” that could help law enforcement officials understand the path from religious extremism to violence. The FBI was in line for funding to increase the number of agents who surreptitiously track activity on jihadist Web sites.

But a year before the bombings in Boston, the search for meaningful insight into the stages of radicalization was described as one of the “more challenging intelligence gaps.”

High-tech surveillance

The documents make clear that U.S. spy agencies’ long-standing reliance on technology remains intact. If anything, their dependence on high-tech surveillance systems to fill gaps in human intelligence has intensified.

A section on North Korea indicates that the United States has all but surrounded the nuclear-armed country with surveillance platforms. Distant ground sensors monitor seismic activity and scan the country for signs that might point to construction of new nuclear sites. U.S. agencies seek to capture photos, air ­samples and infrared imagery “around the clock.”

In Iran, new surveillance techniques and technologies have enabled analysts to identify suspected nuclear sites that had not been detected in satellite images, according to the document.

In Syria, NSA listening posts were able to monitor unencrypted communications among senior military officials at the outset of the civil war there, a vulnerability that President Bashar al-Assad’s forces apparently later recognized. One of the NRO’s functions is to extract data from sensors placed on the ground near suspected illicit weapons sites in Syria and other countries.

Across this catalogue of technical prowess, one category is ­depicted as particularly indis­pensable: signals intelligence, or SIGINT.

The NSA’s ability to monitor e-mails, phone calls and Internet traffic has come under new scrutiny in recent months as a result of disclosures by Snowden, who worked as a contract computer specialist for the agency before stockpiling secret documents and then fleeing, first to Hong Kong and then Moscow.

The NSA was projected to spend $48.6 million on research projects to assist in “coping with information overload,” an occupational hazard as the volumes of intake have increased sharply from fiber-optic cables and Silicon Valley Internet providers.

The agency’s ability to monitor the communications of al-Qaeda operatives is described in the documents as “often the best and only means to compromise seemingly intractable targets.”

Signals intercepts also have been used to direct the flight paths of drones, gather clues to the composition of North Korea’s leadership and evaluate the response plans of Russia’s government in the event of a terrorist attack in Moscow.

The resources devoted to signals intercepts are extraordinary.

Nearly 35,000 employees are listed under a category called the Consolidated Cryptologic Program, which includes the NSA as well as the surveillance and code-breaking components of the Air Force, Army, Navy and Marines.

The NSA is planning high-risk covert missions, a lesser-known part of its work, to plant what it calls “tailored radio frequency solutions” — close-in sensors to intercept communications that do not pass through global networks.

Even the CIA devotes $1.7 billion, or nearly 12 percent of its budget, to technical collection efforts, including a joint program with the NSA called “CLANSIG,” a covert program to intercept radio and telephone communications from hostile territory.

The agency also is pursuing tracking systems “that minimize or eliminate the need for physical access and enable deep concealment operations against hard targets.”

The CIA has deployed new biometric sensors to confirm the identities and locations of al-Qaeda operatives. The system has been used in the CIA’s drone campaign.

Spending on satellite systems and almost every other category of collection is projected to shrink or remain stagnant in coming years, as Washington grapples with budget cuts across the government. But the 2013 intelligence budget called for increased investment in SIGINT.

Counterintelligence

The budget includes a lengthy section on funding for counterintelligence programs designed to protect against the danger posed by foreign intelligence services as well as betrayals from within the U.S. spy ranks.

The document describes programs to “mitigate insider threats by trusted insiders who seek to exploit their authorized access to sensitive information to harm U.S. interests.”

The agencies had budgeted for a major counterintelligence initiative in fiscal 2012, but most of those resources were diverted to an all-hands emergency response to successive floods of classified data released by the anti-secrecy group WikiLeaks.

For this year, the budget promised a renewed “focus . . . on safeguarding classified networks” and a strict “review of high-risk, high-gain applicants and contractors” — the young, nontraditional computer coders with the skills the NSA needed.

Among them was Snowden, then a 29-year-old contract computer specialist whom the NSA trained to circumvent computer network security. He was copying thousands of highly classified documents at an NSA facility in Hawaii, and preparing to leak them, as the agency embarked on the new security sweep.

“NSA will initiate a minimum of 4,000 periodic reinvestigations of potential insider compromise of sensitive information,” according to the budget, scanning its systems for “anomalies and alerts.”

Julie Tate contributed to this report.

Covert action. Surveillance. Counterintelligence. The U.S. “black budget” spans over a dozen agencies that make up the National Intelligence Program.

U.S. intelligence services carried out 231 offensive operations in 2011

By Barton Gellman and Ellen Nakashima

U.S. intelligence services carried out 231 offensive cyber-operations in 2011, the leading edge of a clandestine campaign that embraces the Internet as a theater of spying, sabotage and war, according to top-secret documents obtained by The Washington Post.

That disclosure, in a classified intelligence budget provided by NSA leaker Edward Snowden, provides new evidence that the Obama administration’s growing ranks of cyberwarriors infiltrate and disrupt foreign computer networks.

Additionally, under an extensive effort code-named GENIE, U.S. computer specialists break into foreign networks so that they can be put under surreptitious U.S. control. Budget documents say the $652 million project has placed “covert implants,” sophisticated malware transmitted from far away, in computers, routers and firewalls on tens of thousands of machines every year, with plans to expand those numbers into the millions.

The documents provided by Snowden and interviews with former U.S. officials describe a campaign of computer intrusions that is far broader and more aggressive than previously understood. The Obama administration treats all such cyber-operations as clandestine and declines to acknowledge them.

The scope and scale of offensive operations represent an evolution in policy, which in the past sought to preserve an international norm against acts of aggression in cyberspace, in part because U.S. economic and military power depend so heavily on computers.

“The policy debate has moved so that offensive options are more prominent now,” said former deputy defense secretary William J. Lynn III, who has not seen the budget document and was speaking generally. “I think there’s more of a case made now that offensive cyberoptions can be an important element in deterring certain adversaries.”

Of the 231 offensive operations conducted in 2011, the budget said, nearly three-quarters were against top-priority targets, which former officials say includes adversaries such as Iran, Russia, China and North Korea and activities such as nuclear proliferation. The document provided few other details about the operations.

Stuxnet, a computer worm reportedly developed by the United States and Israel that destroyed Iranian nuclear centrifuges in attacks in 2009 and 2010, is often cited as the most dramatic use of a cyberweapon. Experts said no other known cyberattacks carried out by the United States match the physical damage inflicted in that case.

U.S. agencies define offensive cyber-operations as activities intended “to manipulate, disrupt, deny, degrade, or destroy information resident in computers or computer networks, or the computers and networks themselves,” according to a presidential directive issued in October 2012.

Most offensive operations have immediate effects only on data or the proper functioning of an adversary’s machine: slowing its network connection, filling its screen with static or scrambling the results of basic calculations. Any of those could have powerful effects if they caused an adversary to botch the timing of an attack, lose control of a computer or miscalculate locations.

U.S. intelligence services are making routine use around the world of government-built malware that differs little in function from the “advanced persistent threats” that U.S. officials attribute to China. The principal difference, U.S. officials told The Post, is that China steals U.S. corporate secrets for financial gain.

“The Department of Defense does engage” in computer network exploitation, according to an e-mailed statement from an NSA spokesman, whose agency is part of the Defense Department. “The department does ***not*** engage in economic espionage in any domain, including cyber.”

‘Millions of implants’

The administration’s cyber-operations sometimes involve what one budget document calls “field operations” abroad, commonly with the help of CIA operatives or clandestine military forces, “to physically place hardware implants or software modifications.”

Much more often, an implant is coded entirely in software by an NSA group called Tailored Access Operations (TAO). As its name suggests, TAO builds attack tools that are custom-fitted to their targets.

The NSA unit’s software engineers would rather tap into networks than individual computers because there are usually many devices on each network. Tailored Access Operations has software templates to break into common brands and models of “routers, switches and firewalls from multiple product vendor lines,” according to one document describing its work.

The implants that TAO creates are intended to persist through software and equipment upgrades, to copy stored data, “harvest” communications and tunnel into other connected networks. This year TAO is working on implants that “can identify select voice conversations of interest within a target network and exfiltrate select cuts,” or excerpts, according to one budget document. In some cases, a single compromised device opens the door to hundreds or thousands of others.

Sometimes an implant’s purpose is to create a back door for future access. “You pry open the window somewhere and leave it so when you come back the owner doesn’t know it’s unlocked, but you can get back in when you want to,” said one intelligence official, who was speaking generally about the topic and was not privy to the budget. The official spoke on the condition of anonymity to discuss sensitive technology.

Under U.S. cyberdoctrine, these operations are known as “exploitation,” not “attack,” but they are essential precursors both to attack and defense.

By the end of this year, GENIE is projected to control at least 85,000 implants in strategically chosen machines around the world. That is quadruple the number — 21,252 — available in 2008, according to the U.S. intelligence budget.

The NSA appears to be planning a rapid expansion of those numbers, which were limited until recently by the need for human operators to take remote control of compromised machines. Even with a staff of 1,870 people, GENIE made full use of only 8,448 of the 68,975 machines with active implants in 2011.

For GENIE’s next phase, according to an authoritative reference document, the NSA has brought online an automated system, code-named TURBINE, that is capable of managing “potentially millions of implants” for intelligence gathering “and active attack.”

‘The ROC’

When it comes time to fight the cyberwar against the best of the NSA’s global competitors, the TAO calls in its elite operators, who work at the agency’s Fort Meade headquarters and in regional operations centers in Georgia, Texas, Colorado and Hawaii. The NSA’s organizational chart has the main office as S321. Nearly everyone calls it “the ROC,” pronounced “rock”: the Remote Operations Center.

“To the NSA as a whole, the ROC is where the hackers live,” said a former operator from another section who has worked closely with the exploitation teams. “It’s basically the one-stop shop for any kind of active operation that’s not defensive.”

Once the hackers find a hole in an adversary’s defense, “[t]argeted systems are compromised electronically, typically providing access to system functions as well as data. System logs and processes are modified to cloak the intrusion, facilitate future access, and accomplish other operational goals,” according to a 570-page budget blueprint for what the government calls its Consolidated Cryptologic Program, which includes the NSA.

Teams from the FBI, the CIA and U.S. Cyber Command work alongside the ROC, with overlapping missions and legal authorities. So do the operators from the NSA’s National Threat Operations Center, whose mission is focused primarily on cyber­defense. That was Snowden’s job as a Booz Allen Hamilton contractor, and it required him to learn the NSA’s best hacking techniques.

According to one key document, the ROC teams give Cyber Command “specific target related technical and operational material (identification/recognition), tools and techniques that allow the employment of U.S. national and tactical specific computer network attack mechanisms.”

The intelligence community’s cybermissions include defense of military and other classified computer networks against foreign attack, a task that absorbs roughly one-third of a total cyber operations budget of $1.02 billion in fiscal 2013, according to the Cryptologic Program budget. The ROC’s breaking-and-entering mission, supported by the GENIE infrastructure, spends nearly twice as much: $651.7 million.

Most GENIE operations aim for “exploitation” of foreign systems, a term defined in the intelligence budget summary as “surreptitious virtual or physical access to create and sustain a presence inside targeted systems or facilities.” The document adds: “System logs and processes are modified to cloak the intrusion, facilitate future access, and accomplish other operational goals.”

 

The NSA designs most of its own implants, but it devoted $25.1 million this year to “additional covert purchases of software vulnerabilities” from private malware vendors, a growing gray-market industry based largely in Europe.

 

‘Most challenging targets’

 

The budget documents cast U.S. attacks as integral to cyber­defense — describing them in some cases as “active defense.”

“If you’re neutralizing someone’s nuclear command and control, that’s a huge attack,” said one former defense official. The greater the physical effect, officials said, the less likely it is that an intrusion can remain hidden.

“The United States is moving toward the use of tools short of traditional weapons that are unattributable — that cannot be easily tied to the attacker — to convince an adversary to change their behavior at a strategic level,” said another former senior U.S. official, who also spoke on the condition of anonymity to discuss sensitive operations.

China and Russia are regarded as the most formidable cyber­threats, and it is not always easy to tell who works for whom. China’s offensive operations are centered in the Technical Reconnaissance Bureau of the People’s Liberation Army, but U.S. intelligence has come to believe that those state-employed hackers by day return to work at night for personal profit, stealing valuable U.S. defense industry secrets and selling them.

Iran is a distant third in capability but is thought to be more strongly motivated to retaliate for Stuxnet with an operation that would not only steal information but erase it and attempt to damage U.S. hardware.

The “most challenging targets” to penetrate are the same in cyber-operations as for all other forms of data collection described in the intelligence budget: Iran, North Korea, China and Russia. GENIE and ROC operators place special focus on locating suspected terrorists “in Afghanistan, Pakistan, Yemen, Iraq, Somalia, and other extremist safe havens,” according to one list of priorities.

The growth of Tailored Access Operations at the NSA has been accompanied by a major expansion of the CIA’s Information Operations Center, or IOC.

The CIA unit employs hundreds of people at facilities in Northern Virginia and has become one of the CIA’s largest divisions. Its primary focus has shifted in recent years from counterterrorism to cybersecurity, according to the budget document.

The military’s cyber-operations, including U.S. Cyber Command, have drawn much of the public’s attention, but the IOC undertakes some of the most notable offensive operations, including the recruitment of several new intelligence sources, the document said.

Military cyber-operations personnel grouse that the actions they can take are constrained by the legal authorities that govern them. The presidential policy directive on cyber-operations issued in October made clear that military cyber-operations that result in the disruption or destruction or even manipulation of computers must be approved by the president. But the directive, the existence of which was first reported last fall by The Post and leaked in June by Snowden, largely does not apply to the intelligence community.

Given the “vast volumes of data” pulled in by the NSA, storage has become a pressing question. The NSA is nearing completion of a massive new data center in Utah. A second one will be built at Fort Meade “to keep pace with cyber processing demands,” the budget document said.

According to the document, a high-performance computing center in Utah will manage “storage, analysis, and intelligence production.” This will allow intelligence agencies “to evaluate similarities among intrusions that could indicate the presence of a coordinated cyber attack, whether from an organized criminal enterprise or a nation-state.”

By Barton Gellman, Craig Timberg and Steven Rich

On Nov. 1, 2007, the National Security Agency hosted a talk by Roger Dingledine, principal designer of one of the world’s leading Internet privacy tools. It was a wary encounter, akin to mutual intelligence gathering, between a spy agency and a man who built tools to ward off electronic surveillance.

According to a top-secret NSA summary of the meeting, Dingledine told the assembled NSA staff that his service, called Tor, offered anonymity to people who needed it badly — to keep business secrets, protect their identities from oppressive political regimes or conduct research without revealing themselves. In the minds of NSA officials, Tor was offering protection to terrorists and other intelligence targets.

As he spoke to the NSA, Dingledine said in an interview Friday, he suspected the agency was attempting to break into Tor, which is used by millions of people around the world to shield their identities. Documents provided to The Washington Post by former agency contractor Edward Snowden show that he was right.

Beginning at least a year before Dingledine’s visit, the NSA has mounted increasingly successful attacks to unmask the identities and locations of users of Tor. In some cases, the agency has succeeded in blocking access to the anonymous network, diverting Tor users to insecure channels. In others, it has been able to “stain” anonymous traffic as it enters the Tor network, enabling the NSA to identify users as it exits.

Tor works by encrypting traffic repeatedly as it flows across a global network of servers, mostly run by volunteers. The traffic, which can include e-mails, information from a Web site and almost anything else on the Internet, is supposed to arrive at its destination with no identifying information about its origin or the path it took.

The Snowden documents, including a detailed PowerPoint presentation, suggest that the NSA cannot see directly inside Tor’s anonymous network but that it has repeatedly uncloaked users by circumventing Tor’s protections. The documents also illustrate the power of the NSA to at least partially penetrate what have long been considered the most secure corners of the Internet.

The U.S. Naval Research Laboratory first developed Tor more than a decade ago as a tool to allow anonymous communications and Web browsing. It was embraced by privacy advocates, including the Electronic Frontier Foundation, and continues to receive substantial federal funding. Tor is now maintained by Dingledine’s nonprofit group, the Tor Project.

The State Department trains political activists worldwide on how to use Tor to protect communications from the intelligence services of repressive governments. But the anonymity service also has become popular with criminals — especially dealers of illicit drugs, military-grade weapons and child pornography — and terrorists seeking to evade tracking by Western intelligence services.

One of the documents provided by Snowden said an NSA technique code-named EGOTISTICALGIRAFFE had succeeded in unmasking 24 Tor users in a single weekend. The same operation allowed the NSA to discover the identity of a key propagandist for al-Qaeda in the Arabian Peninsula, as the group’s offshoot in Yemen is known, after he posted information and instructions on the group’s Web site.

NSA anti-anonymity techniques are now also being used by law enforcement agencies. In August, civilian security researchers detected an FBI operation against an alleged child pornography ring that used a Tor-based Web server called Freedom Hosting. The FBI mounted a cyber­attack to unmask the location and owner of that anonymous server, using precisely the technique described as EGOTISTICALGIRAFFE.

The Washington Post is not releasing certain details from the documents, including the name of the al-Qaeda operative. Documents about the NSA’s attempts to penetrate Tor were also shared with the British newspaper the Guardian, which published a report on the effort Friday.

In a statement, Director of National Intelligence James R. Clapper Jr., who oversees the NSA and other intelligence agencies, said that the intelligence community “seeks to understand” tools that facilitate anonymous communication. He added that it does so because of the “undeniable fact that these are the tools our adversaries use to communicate and coordinate attacks against the United States and our allies.”

The intelligence community “is only interested in communication related to valid foreign intelligence and counterintelligence purposes,” Clapper said.

There is no evidence that the NSA is capable of unmasking Tor traffic routinely on a global scale. But for almost seven years, it has been trying.

Since 2006, according to a 49-page research paper titled simply “Tor,” the agency has worked on several methods that, if successful, would allow the NSA to uncloak anonymous traffic on a “wide scale” — effectively by watching communications as they enter and exit the Tor system, rather than trying to follow them inside. One type of attack, for example, would identify users by minute differences in the clock times on their computers.

Dingledine expressed no surprise that the NSA has tried to defeat efforts at anonymity. In the interview, he said the weaknesses in Tor described in the PowerPoint presentation likely could be exploited only against a relatively small number of individual users. That, he said, is reassuring.

“If those documents actually represent what they can do, they are not as big an adversary as I thought,” he said.

The Tor Browser Bundle, available for free at www.torproject.org, was downloaded 40 million times last year. Until a recent security upgrade to the Firefox browser, which is incorporated in the bundle, the NSA could trick the browser into leaking the real Internet address of a targeted user. One slide described these tactics as “pretty much guaranteed to succeed.”

Mozilla, the nonprofit organization that develops Firefox, declined to comment.

One document provided by Snowden included an internal exchange among NSA hackers in which one of them said the agency’s Remote Operations Center was capable of targeting anyone who visited an al-Qaeda Web site using Tor.

“The ROC currently [operates] against certain extremist web forums at the moment,” the employee wrote. “I am under the impression that they can serve up an exploit” — hacker jargon for malicious code — “to pretty much anyone that visits the particular web forum, though.”

“Like any tool, [Tor] can be used for something good, and it can be used for something bad,” said Garth Bruen, a Boston-based investigator who studies Internet crimes. “It’s all about how people are using it, and criminals have been using it to great advantage. . . . It’s a nightmare.”

An FBI agent told an Irish court last month that Freedom Hosting, unmasked with NSA-devised techniques, was among the largest purveyors of child pornography in the world, according to news reports. Silk Road, an online marketplace some called “the eBay of illicit substances,” also relied on Tor — and was targeted by the FBI. Federal officials arrested the alleged founder and shut down the site Wednesday.

Privacy advocates, however, say Tor is valuable and should be protected even if it is sometimes used by criminals. “Tor is networking technology,” said Christopher Soghoian, an American Civil Liberties Union technology expert. “It is no different from a postage stamp or a highway. Good people use highways, and bad people use highways.”

The NSA documents portray a years-long program to defeat what the agency called “The Tor Problem,” with the agency repeatedly updating its tactics as Tor’s developers made changes to the network.

The NSA also altered tactics as Mozilla introduced new versions of Firefox. In anticipation of a new release of Firefox, one agency official wrote in January that a new exploit was under development: “I’m confident we can have it ready when they release something new, or very soon after :).”

In late 2006, when the NSA prepared a working paper on methods to defeat Tor, the anonymous network had an estimated 200,000 users and 1,000 servers. Among the secret NSA documents were lists of hundreds of servers the agency believed to be “nodes” on that network.

Along with EGOTISTICALGIRAFFE, the agency’s cover names for Tor attacks have included MJOLNIR, MOTHMONSTER and EGOTISTICALGOAT. A similar program at Britain’s Government Communications Headquarters, the NSA’s close counterpart, was called STUNT WORM.

One NSA PowerPoint presentation provided by Snowden is titled “Peeling Back the Layers of TOR with EGOTISTICAL­GIRAFFE.”

The agency began identifying browsers that were using Tor by noting how the encryption program reset what’s called the ­BuildID — a 14-digit code representing the exact date and time when that version of Firefox was released. On versions using Tor, the BuildID is reset to “0.” That feature made it hard to distinguish one Tor user from another, but it also allowed the NSA to pick out Tor-enabled browsers from among all others in use at any given moment.

“It’s easy!” a slide describing the technique said.

Mozilla issued a patch to Firefox that would protect newer versions of the browser against such an attack, though the NSA documents make clear that research into new exploits remains active.

One PowerPoint slide sums up a multistep method for learning the identity and location of Tor users and implanting NSA code in the browser. It ends with a final bullet point saying, “Win!”

Ashkan Soltani and Julie Tate contributed to this report.

Although interception occurs overseas, it sweeps in many Americans’ contact lists

By Barton Gellman and Ashkan Solanti

The National Security Agency is harvesting hundreds of millions of contact lists from personal e-mail and instant messaging accounts around the world, many of them belonging to Americans, according to senior intelligence officials and top-secret documents provided by former NSA contractor Edward Snowden.

The collection program, which has not been disclosed before, intercepts e-mail address books and “buddy lists” from instant messaging services as they move across global data links. Online services often transmit those contacts when a user logs on, composes a message, or synchronizes a computer or mobile device with information stored on remote servers.

Rather than targeting individual users, the NSA is gathering contact lists in large numbers that amount to a sizable fraction of the world’s e-mail and instant messaging accounts. Analysis of that data enables the agency to search for hidden connections and to map relationships within a much smaller universe of foreign intelligence targets.

During a single day last year, the NSA’s Special Source Operations branch collected 444,743 e-mail address books from Yahoo, 105,068 from Hotmail, 82,857 from Facebook, 33,697 from Gmail and 22,881 from unspecified other providers, according to an internal NSA PowerPoint presentation. Those figures, described as a typical daily intake in the document, correspond to a rate of more than 250 million a year.

Each day, the presentation said, the NSA collects contacts from an estimated 500,000 buddy lists on live-chat services as well as from the inbox displays of Web-based e-mail accounts.

The collection depends on secret arrangements with foreign telecommunications companies or allied intelligence services in control of facilities that direct traffic along the Internet’s main data routes.

Although the collection takes place overseas, two senior U.S. intelligence officials acknowledged that it sweeps in the contacts of many Americans. They declined to offer an estimate but did not dispute that the number is likely to be in the millions or tens of millions.

A spokesman for the Office of the Director of National Intelligence, which oversees the NSA, said the agency “is focused on discovering and developing intelligence about valid foreign intelligence targets like terrorists, human traffickers and drug smugglers. We are not interested in personal information about ordinary Americans.”

The spokesman, Shawn Turner, added that rules approved by the attorney general require the NSA to “minimize the acquisition, use and dissemination” of information that identifies a U.S. citizen or permanent resident.

The NSA’s collection of nearly all U.S. call records, under a separate program, has generated significant controversy since it was revealed in June. The NSA’s director, Gen. Keith B. Alexander, has defended “bulk” collection as an essential counterterrorism and foreign intelligence tool, saying, “You need the haystack to find the needle.”

Contact lists stored online provide the NSA with far richer sources of data than call records alone. Address books commonly include not only names and e-mail addresses, but also telephone numbers, street addresses, and business and family information. Inbox listings of e-mail accounts stored in the “cloud” sometimes contain content, such as the first few lines of a message.

Taken together, the data would enable the NSA, if permitted, to draw detailed maps of a person’s life, as told by personal, professional, political and religious connections. The picture can also be misleading, creating false “associations” with ex-spouses or people with whom an account holder has had no contact in many years.

The NSA has not been authorized by Congress or the special intelligence court that oversees foreign surveillance to collect contact lists in bulk, and senior intelligence officials said it would be illegal to do so from facilities in the United States. The agency avoids the restrictions in the Foreign Intelligence Surveillance Act by intercepting contact lists from access points “all over the world,” one official said, speaking on the condition of anonymity to discuss the classified program. “None of those are on U.S. territory.”

Because of the method employed, the agency is not legally required or technically able to restrict its intake to contact lists belonging to specified foreign intelligence targets, he said.

When information passes through “the overseas collection apparatus,” the official added, “the assumption is you’re not a U.S. person.”

In practice, data from Americans is collected in large volumes — in part because they live and work overseas, but also because data crosses international boundaries even when its American owners stay at home. Large technology companies, including Google and Facebook, maintain data centers around the world to balance loads on their servers and work around outages.

A senior U.S. intelligence official said the privacy of Americans is protected, despite mass collection, because “we have checks and balances built into our tools.”

NSA analysts, he said, may not search within the contacts database or distribute information from it unless they can “make the case that something in there is a valid foreign intelligence target in and of itself.”

 

In this program, the NSA is obliged to make that case only to itself or others in the executive branch. With few exceptions, intelligence operations overseas fall solely within the president’s legal purview. The Foreign Intelligence Surveillance Act, enacted in 1978, imposes restrictions only on electronic surveillance that targets Americans or takes place on U.S. territory.

 

By contrast, the NSA draws on authority in the Patriot Act for its bulk collection of domestic phone records, and it gathers online records from U.S. Internet companies, in a program known as PRISM, under powers granted by Congress in the FISA Amendments Act. Those operations are overseen by the Foreign Intelligence Surveillance Court.

Sen. Dianne Feinstein, the California Democrat who chairs the Senate Intelligence Committee, said in August that the committee has less information about, and conducts less oversight of, intelligence gathering that relies solely on presidential authority. She said she planned to ask for more briefings on those programs.

“In general, the committee is far less aware of operations conducted under 12333,” said a senior committee staff member, referring to Executive Order 12333, which defines the basic powers and responsibilities of the intelligence agencies. “I believe the NSA would answer questions if we asked them, and if we knew to ask them, but it would not routinely report these things, and, in general, they would not fall within the focus of the committee.”

Because the agency captures contact lists “on the fly” as they cross major Internet switches, rather than “at rest” on computer servers, the NSA has no need to notify the U.S. companies that host the information or to ask for help from them.

“We have neither knowledge of nor participation in this mass collection of web-mail addresses or chat lists by the government,” said Google spokeswoman Niki Fenwick.

At Microsoft, spokeswoman Nicole Miller said the company “does not provide any government with direct or unfettered access to our customers’ data,” adding that “we would have significant concerns if these allegations about government actions are true.”

Facebook spokeswoman Jodi Seth said that “we did not know and did not assist” in the NSA’s interception of contact lists.

It is unclear why the NSA collects more than twice as many address books from Yahoo than the other big services combined. One possibility is that Yahoo, unlike other service providers, has left connections to its users unencrypted by default.

Suzanne Philion, a Yahoo spokeswoman, said Monday in response to an inquiry from The Washington Post that, beginning in January, Yahoo would begin encrypting all its e-mail connections.

Google was the first to secure all its e-mail connections, turning on “SSL encryption” globally in 2010. People with inside knowledge said the move was intended in part to thwart large-scale collection of its users’ information by the NSA and other intelligence agencies.

The volume of NSA contacts collection is so high that it has occasionally threatened to overwhelm storage repositories, forcing the agency to halt its intake with “emergency detasking” orders. Three NSA documents describe short-term efforts to build an “across-the-board technology throttle for truly heinous data” and longer-term efforts to filter out information that the NSA does not need.

Spam has proven to be a significant problem for the NSA — clogging databases with information that holds no foreign intelligence value. The majority of all e-mails, one NSA document says, “are SPAM from ‘fake’ addresses and never ‘delivered’ to targets.”

In fall 2011, according to an NSA presentation, the Yahoo account of an Iranian target was “hacked by an unknown actor,” who used it to send spam. The Iranian had “a number of Yahoo groups in his/her contact list, some with many hundreds or thousands of members.”

The cascading effects of repeated spam messages, compounded by the automatic addition of the Iranian’s contacts to other people’s address books, led to a massive spike in the volume of traffic collected by the Britain intelligence service on the NSA’s behalf.

After nine days of data-bombing, the Iranian’s contact book and contact books for several people within it were “emergency detasked.”

In a briefing from the NSA’s Large Access Exploitation working group, that example was used to illustrate the need to narrow the criteria for data interception. It called for a “shifting collection philosophy”: “Memorialize what you need” vs. “Order one of everything off the menu and eat what you want.”

Julie Tate contributed to this report. Soltani is an independent security researcher and consultant.

Bin Laden associate Hassan Ghul was killed as part of targeted U.S. program

By Greg Miller, Julie Tate and Barton Gellman

It was an innocuous e-mail, one of millions sent every day by spouses with updates on the situation at home. But this one was of particular interest to the National Security Agency and contained clues that put the sender’s husband in the crosshairs of a CIA drone.

Days later, Hassan Ghul — an associate of Osama bin Laden who provided a critical piece of intelligence that helped the CIA find the al-Qaeda leader — was killed by a drone strike in Pakistan’s tribal belt.

The U.S. government has never publicly acknowledged killing Ghul. But documents provided to The Washington Post by former NSA contractor Edward Snowden confirm his demise in October 2012 and reveal the agency’s extensive involvement in the targeted killing program that has served as a centerpiece of President Obama’s counterterrorism strategy.

An al-Qaeda operative who had a knack for surfacing at dramatic moments in the post-Sept. 11 story line, Ghul was an emissary to Iraq for the terrorist group at the height of that war. He was captured in 2004 and helped expose bin Laden’s courier network before spending two years at a secret CIA prison. Then, in 2006, the United States delivered him to his native Pakistan, where he was released and returned to the al-Qaeda fold.

But beyond filling in gaps about Ghul, the documents provide the most detailed account of the intricate collaboration between the CIA and the NSA in the drone campaign.

The Post is withholding many details about those missions, at the request of U.S. intelligence officials who cited potential damage to ongoing operations and national security.

The NSA is “focused on discovering and developing intelligence about valid foreign intelligence targets,” an NSA spokeswoman said in a statement provided to The Post on Wednesday, adding that the agency’s operations “protect the nation and its interests from threats such as terrorism and the proliferation of weapons of mass destruction.”

In the search for targets, the NSA has draped a surveillance blanket over dozens of square miles of northwest Pakistan. In Ghul’s case, the agency deployed an arsenal of cyber-espionage tools, secretly seizing control of laptops, siphoning audio files and other messages, and tracking radio transmissions to determine where Ghul might “bed down.”

The e-mail from Ghul’s wife “about her current living conditions” contained enough detail to confirm the coordinates of that household, according to a document summarizing the mission. “This information enabled a capture/kill operation against an individual believed to be Hassan Ghul on October 1,” it said.

The file is part of a collection of records in the Snowden trove that make clear that the drone campaign — often depicted as the CIA’s exclusive domain — relies heavily on the NSA’s ability to vacuum up enormous quantities of e-mail, phone calls and other fragments of signals intelligence, or SIGINT.

To handle the expanding workload, the NSA created a secret unit known as the Counter-Terrorism Mission Aligned Cell, or CT MAC, to concentrate the agency’s vast resources on hard-to-find terrorism targets. The unit spent a year tracking Ghul and his courier network, tunneling into an array of systems and devices, before he was killed. Without those penetrations, the document concluded, “this opportunity would not have been possible.”

At a time when the NSA is facing intense criticism for gathering data on Americans, the drone files may bolster the agency’s case that its resources are focused on fighting terrorism and supporting U.S. operations overseas.

“Ours is a noble cause,” NSA Director Keith B. Alexander said during a public event last month. “Our job is to defend this nation and to protect our civil liberties and privacy.”

The documents do not explain how the Ghul e-mail was obtained or whether it was intercepted using legal authorities that have emerged as a source of controversy in recent months and enable the NSA to compel technology giants including Microsoft and Google to turn over information about their users. Nor is there a reference to another NSA program facing scrutiny after Snowden’s leaks, its metadata collection of numbers dialed by nearly every person in the United States.

To the contrary, the records indicate that the agency depends heavily on highly targeted network penetrations to gather information that wouldn’t otherwise be trapped in surveillance nets that it has set at key Internet gateways.

The new documents are self-congratulatory in tone, drafted to tout the NSA’s counterterrorism capabilities. One is titled “CT MAC Hassan Gul Success.” The files make no mention of other agencies’ roles in a drone program that escalated dramatically in 2009 and 2010 before tapering off in recent years.

Even so, former CIA officials said the files are an accurate reflection of the NSA’s contribution to finding targets in a campaign that has killed more than 3,000 people, including thousands of alleged militants and hundreds of civilians, in Pakistan, according to independent surveys. The officials said the agency has assigned senior analysts to the CIA’s Counterterrorism Center, and deployed others to work alongside CIA counterparts at almost every major U.S. embassy or military base overseas.

“NSA threw the kitchen sink at the FATA,” said a former U.S. intelligence official with experience in Afghanistan and Pakistan, referring to the Federally Administered Tribal Areas, the region in northwest Pakistan where al-Qaeda’s leadership is based.

NSA employees rarely ventured beyond the security gates of the U.S. Embassy in Islamabad, officials said. Surveillance operations that required placing a device or sensor near an al-Qaeda compound were handled by the CIA’s Information Operations Center, which specializes in high-tech devices and “close-in” surveillance work.

“But if you wanted huge coverage of the FATA, NSA had 10 times the manpower, 20 times the budget and 100 times the brainpower,” the former intelligence official said, comparing the surveillance resources of the NSA to the smaller capabilities of the agency's IOC. The two agencies are the largest in the U.S. intelligence community, with budgets last year of $14.7 billion for the CIA and $10.8 billion for the NSA. “We provided the map,” the former official said, “and they just filled in the pieces.”

In broad terms, the NSA relies on increasingly sophisticated versions of online attacks that are well-known among security experts. Many rely on software implants developed by the agency’s Tailored Access Operations division with code-names such as UNITEDRAKE and VALIDATOR. In other cases, the agency runs “man-in-the-middle” attacks in which it positions itself unnoticed midstream between computers communicating with one another, diverting files for real-time alerts and longer-term analysis in data repositories.

Through these and other tactics, the NSA is able to extract vast quantities of digital information, including audio files, imagery and keystroke logs. The operations amount to silent raids on suspected safe houses and often are carried out by experts sitting behind desks thousands of miles from their targets.

The reach of the NSA’s Tailored Access Operations division extends far beyond Pakistan. Other documents describe efforts to tunnel into systems used by al-Qaeda affiliates in Yemen and Africa, each breach exposing other corridors.

An operation against a suspected facilitator for al-Qaeda’s branch in Yemen led to a trove of files that could be used to “help NSA map out the movement of terrorists and aspiring extremists between Yemen, Syria, Turkey, Egypt, Libya and Iran,” according to the documents. “This may enable NSA to better flag the movement of these individuals” to allied security services that “can put individuals on no-fly lists or monitor them once in country.”

A single penetration yielded 90 encrypted al-Qaeda documents, 16 encryption keys, 30 unencrypted messages as well as “thousands” of chat logs, according to an inventory described in one of the Snowden documents.

The operations are so easy, in some cases, that the NSA is able to start downloading data in less time than it takes the targeted machine to boot up. Last year, a user account on a social media Web site provided an instant portal to an al-Qaeda operative’s hard drive. “Within minutes, we successfully exploited the target,” the document said.

The hunt for Ghul followed a more elaborate path.

Ghul, who is listed in other documents as Mustafa Haji Muhammad Khan, had surfaced on U.S. radar as early as 2003, when an al-Qaeda detainee disclosed that Ghul escorted one of the intended hijackers to a Pakistani safe house a year before the Sept. 11, 2001, attacks.

A trusted facilitator and courier, Ghul was dispatched to Iraq in 2003 to deliver a message to Abu Musab al-Zarqawi, the al-Qaeda firebrand who angered the network’s leaders in Pakistan by launching attacks that often slaughtered innocent Muslims.

When Ghul made another attempt to enter Iraq in 2004, he was detained by Kurdish authorities in an operation directed by the CIA. Almost immediately, Ghul provided a piece of intelligence that would prove more consequential than he may have anticipated: He disclosed that bin Laden relied on a trusted courier known as al-Kuwaiti.

The ripples from that revelation wouldn’t subside for years. The CIA went on to determine the true identity of al-Kuwaiti and followed him to a heavily fortified compound in Abbottabad, Pakistan, where bin Laden was killed in 2011.

Because of the courier tip, Ghul became an unwitting figure in the contentious debate over CIA interrogation measures. He was held at a CIA black site in Eastern Europe, according to declassified Justice Department memos, where he was slapped and subjected to stress positions and sleep deprivation to break his will.

Defenders of the interrogation program have cited Ghul’s courier disclosure as evidence that the agency’s interrogation program was crucial to getting bin Laden. But others, including former CIA operatives directly involved in Ghul’s case, said that he identified the courier while he was being interrogated by Kurdish authorities, who posed questions scripted by CIA analysts in the background.

The debate resurfaced amid the release of the movie “Zero Dark Thirty” last year, in which a detainee’s slip after a brutal interrogation sequence is depicted as a breakthrough in the bin Laden hunt. Ghul’s case also has been explored in detail in a 6,000-page investigation of the CIA interrogation program by the Senate Intelligence Committee that has yet to be released.

Sen. Dianne Feinstein (D-Calif.), the chairman of the panel, sought to settle the Ghul debate in a statement last year that alluded to his role but didn’t mention him by name.

“The CIA detainee who provided the most significant information about the courier provided the information prior to being subjected to coercive interrogation techniques,” Feinstein said in the statement, which was signed by Sen. Carl Levin (D-Mich.).

The George W. Bush administration’s decision to close the secret CIA prisons in 2006 set off a scramble to place prisoners whom the agency did not regard as dangerous or valuable enough to transfer to Guantanamo Bay. Ghul was not among the original 14 high-value CIA detainees sent to the U.S. installation in Cuba. Instead, he was turned over to the CIA’s counterpart in Pakistan, with ostensible assurances that he would remain in custody.

A year later, Ghul was released. There was no public explanation from Pakistani authorities. CIA officials have noted that Ghul had ties to Lashkar-e-Taiba, a militant group supported by Pakistan’s intelligence service. By 2007, he had returned to al-Qaeda’s stronghold in Waziristan.

In 2011, the Treasury Department named Ghul a target of U.S. counterterrorism sanctions. Since his release, the department said, he had helped al-Qaeda reestablish logistics networks, enabling al-Qaeda to move people and money in and out of the country. The NSA document described Ghul as al-Qaeda’s chief of military operations and detailed a broad surveillance effort to find him.

“The most critical piece” came with a discovery that “provided a vector” for compounds used by Ghul, the document said. After months of investigation, and surveillance by CIA drones, the e-mail from his wife erased any remaining doubt.

Even after Ghul was killed in Mir Ali, the NSA’s role in the drone strike wasn’t done. Although the attack was aimed at “an individual believed to be” the correct target, the outcome wasn’t certain until later when, “through SIGINT, it was confirmed that Hassan Ghul was in fact killed.”

Correction: An earlier version of this story included a photograph that purported to show al-Qaeda suspect Hassan Ghul and that was reproduced from a classified document. The photograph is not of Ghul, who was not pictured elsewhere in the document.

 

DATA CENTERS AT ISSUE

Agency has access to millions of accounts

By Barton Gellman and Ashkan Solanti

The National Security Agency has secretly broken into the main communications links that connect Yahoo and Google data centers around the world, according to documents obtained from former NSA contractor Edward Snowden and interviews with knowledgeable officials.

By tapping those links, the agency has positioned itself to collect at will from hundreds of millions of user accounts, many of them belonging to Americans. The NSA does not keep everything it collects, but it keeps a lot.

According to a top-secret accounting dated Jan. 9, 2013, the NSA’s acquisitions directorate sends millions of records every day from internal Yahoo and Google networks to data warehouses at the agency’s headquarters at Fort Meade, Md. In the preceding 30 days, the report said, field collectors had processed and sent back 181,280,466 new records — including “metadata,” which would indicate who sent or received e-mails and when, as well as content such as text, audio and video.

The NSA’s principal tool to exploit the data links is a project called MUSCULAR, operated jointly with the agency’s British counterpart, the Government Communications Headquarters . From undisclosed interception points, the NSA and the GCHQ are copying entire data flows across fiber-optic cables that carry information among the data centers of the Silicon Valley giants.

The infiltration is especially striking because the NSA, under a separate program known as PRISM, has front-door access to Google and Yahoo user accounts through a court-approved process.

The MUSCULAR project appears to be an unusually aggressive use of NSA tradecraft against flagship American companies. The agency is built for high-tech spying, with a wide range of digital tools, but it has not been known to use them routinely against U.S. companies.

In a statement, the NSA said it is “focused on discovering and developing intelligence about valid foreign intelligence targets only.”

“NSA applies Attorney General-approved processes to protect the privacy of U.S. persons — minimizing the likelihood of their information in our targeting, collection, processing, exploitation, retention, and dissemination,” it said.

In a statement, Google’s chief legal officer, David Drummond, said the company has “long been concerned about the possibility of this kind of snooping” and has not provided the government with access to its systems.

“We are outraged at the lengths to which the government seems to have gone to intercept data from our private fiber networks, and it underscores the need for urgent reform,” he said.

A Yahoo spokeswoman said, “We have strict controls in place to protect the security of our data centers, and we have not given access to our data centers to the NSA or to any other government agency.”

Under PRISM, the NSA gathers huge volumes of online communications records by legally compelling U.S. technology companies, including Yahoo and Google, to turn over any data that match court-approved search terms. That program, which was first disclosed by The Washington Post and the Guardian newspaper in Britain, is authorized under Section 702 of the FISA Amendments Act and overseen by the Foreign ­Intelligence Surveillance Court (FISC).

Intercepting communications overseas has clear advantages for the NSA, with looser restrictions and less oversight. NSA documents about the effort refer directly to “full take,” “bulk access” and “high volume” operations on Yahoo and Google networks. Such large-scale collection of Internet content would be illegal in the United States, but the operations take place overseas, where the NSA is allowed to presume that anyone using a foreign data link is a foreigner.

Outside U.S. territory, statutory restrictions on surveillance seldom apply and the FISC has no jurisdiction. Senate Intelligence Committee Chairman Dianne Feinstein (D-Calif.) has acknowledged that Congress conducts little oversight of intelligence-gathering under the presidential authority of Executive Order 12333 , which defines the basic powers and responsibilities of the intelligence agencies.

John Schindler, a former NSA chief analyst and frequent defender who teaches at the Naval War College, said it is obvious why the agency would prefer to avoid restrictions where it can.

“Look, NSA has platoons of lawyers, and their entire job is figuring out how to stay within the law and maximize collection by exploiting every loophole,” he said. “It’s fair to say the rules are less restrictive under Executive Order 12333 than they are under FISA,” the Foreign Intelligence Surveillance Act.

In a statement, the Office of the Director of National Intelligence denied that it was using executive authority to “get around the limitations” imposed by FISA.

The operation to infiltrate data links exploits a fundamental weakness in systems architecture. To guard against data loss and system slowdowns, Google and Yahoo maintain fortresslike data centers across four continents and connect them with thousands of miles of fiber-optic cable. Data move seamlessly around these globe-spanning “cloud” networks, which represent billions of dollars of investment.

For the data centers to operate effectively, they synchronize large volumes of information about account holders. Yahoo’s internal network, for example, sometimes transmits entire e-mail archives — years of messages and attachments — from one data center to another.

Tapping the Google and Yahoo clouds allows the NSA to intercept communications in real time and to take “a retrospective look at target activity,” according to one internal NSA document.

To obtain free access to data- center traffic, the NSA had to circumvent gold-standard security measures. Google “goes to great lengths to protect the data and intellectual property in these centers,” according to one of the company’s blog posts, with tightly audited access controls, heat-sensitive cameras, round-the-clock guards and biometric verification of identities.

Google and Yahoo also pay for premium data links, designed to be faster, more reliable and more secure. In recent years, both of them are said to have bought

or leased thousands of miles of fiber-optic cables for their own exclusive use. They had reason to think, insiders said, that their private, internal networks were safe from prying eyes.

In an NSA presentation slide on “Google Cloud Exploitation,” however, a sketch shows where the “Public Internet” meets the internal “Google Cloud” where their data reside. In hand-printed letters, the drawing notes that encryption is “added and removed here!” The artist adds a smiley face, a cheeky celebration of victory over Google security.

Two engineers with close ties to Google exploded in profanity when they saw the drawing. “I hope you publish this,” one of them said.

For the MUSCULAR project, the GCHQ directs all intake into a “buffer” that can hold three to five days of traffic before recycling storage space. From the buffer, custom-built NSA tools unpack and decode the special data formats that the two companies use inside their clouds. Then the data are sent through a series of filters to “select” information the NSA wants and “defeat” what it does not.

PowerPoint slides about the Google cloud, for example, show that the NSA tries to filter out all data from the company’s “Web crawler,” which indexes Internet pages.

According to the briefing documents, prepared by participants in the MUSCULAR project, collection from inside Yahoo and Google has produced important intelligence leads against hostile foreign governments that are specified in the documents.

Last month, long before The Post approached Google to discuss the penetration of its cloud, Eric Grosse, vice president for security engineering, said the company is rushing to encrypt the links between its data centers. “It’s an arms race,” he said then. “We see these government agencies as among the most skilled players in this game.”

Yahoo has not announced plans to encrypt its data-center links.

Because digital communications and cloud storage do not usually adhere to national boundaries, MUSCULAR and a previously disclosed NSA operation to collect Internet address books have amassed content and metadata on a previously unknown scale from U.S. citizens and residents. Those operations have gone undebated in public or in Congress because their existence was classified.

The Google and Yahoo operations call attention to an asymmetry in U.S. surveillance law. Although Congress has lifted some restrictions on NSA domestic surveillance on grounds that purely foreign communications sometimes pass over U.S. switches and cables, it has not added restrictions overseas, where American communications or data stores now cross over foreign switches.

“Thirty-five years ago, different countries had their own telecommunications infrastructure, so the division between foreign and domestic collection was clear,” Sen. Ron Wyden (D-Ore.), a member of the intelligence panel, said in an interview. “Today there’s a global communications infrastructure, so there’s a greater risk of collecting on Americans when the NSA collects overseas.”

It is not clear how much data from Americans is collected and how much of that is retained. One weekly report on MUSCULAR says the British operators of the site allow the NSA to contribute 100,000 “selectors,” or search terms. That is more than twice the number in use in the PRISM program, but even 100,000 cannot easily account for the millions of records that are said to be sent to Fort Meade each day.

In 2011, when the FISC learned that the NSA was using similar methods to collect and analyze data streams — on a much smaller scale — from cables on U.S. territory, Judge John D. Bates ruled that the program was illegal under FISA and inconsistent with the requirements of the Fourth Amendment.

Soltani is an independent security researcher and consultant.

By Barton Gellman and Ashkan Solanti

The National Security Agency is gathering nearly 5 billion records a day on the whereabouts of cellphones around the world, according to top-secret documents and interviews with U.S. intelligence officials, enabling the agency to track the movements of individuals — and map their relationships — in ways that would have been previously unimaginable.

The records feed a vast database that stores information about the locations of at least hundreds of millions of devices, according to the officials and the documents, which were provided by former NSA contractor Edward Snowden. New projects created to analyze that data have provided the intelligence community with what amounts to a mass surveillance tool.

The NSA does not target Americans’ location data by design, but the agency acquires a substantial amount of information on the whereabouts of domestic cellphones “incidentally,” a legal term that connotes a foreseeable but not deliberate result.

One senior collection manager, speaking on the condition of anonymity but with permission from the NSA, said “we are getting vast volumes” of location data from around the world by tapping into the cables that connect mobile networks globally and that serve U.S. cellphones as well as foreign ones. Additionally, data are often collected from the tens of millions of Americans who travel abroad with their cellphones every year.

In scale, scope and potential impact on privacy, the efforts to collect and analyze location data may be unsurpassed among the NSA surveillance programs that have been disclosed since June. Analysts can find cellphones anywhere in the world, retrace their movements and expose hidden relationships among the people using them.

U.S. officials said the programs that collect and analyze location data are lawful and intended strictly to develop intelligence about foreign targets.

Robert Litt, general counsel for the Office of the Director of National Intelligence, which oversees the NSA, said “there is no element of the intelligence community that under any authority is intentionally collecting bulk cellphone location information about cellphones in the United States.”

The NSA has no reason to suspect that the movements of the overwhelming majority of cellphone users would be relevant to national security. Rather, it collects locations in bulk because its most powerful analytic tools — known collectively as CO-TRAVELER — allow it to look for unknown associates of known intelligence targets by tracking people whose movements intersect.

Still, location data, especially when aggregated over time, are widely regarded among privacy advocates as uniquely sensitive. Sophisticated mathematical tech­niques enable NSA analysts to map cellphone owners’ relationships by correlating their patterns of movement over time with thousands or millions of other phone users who cross their paths. Cellphones broadcast their locations even when they are not being used to place a call or send a text message.

CO-TRAVELER and related tools require the methodical collection and storage of location data on what amounts to a planetary scale. The government is tracking people from afar into confidential business meetings or personal visits to medical facilities, hotel rooms, private homes and other traditionally protected spaces.

“One of the key components of location data, and why it’s so sensitive, is that the laws of physics don’t let you keep it private,” said Chris Soghoian, principal technologist at the American Civil Liberties Union. People who value their privacy can encrypt their e-mails and disguise their online identities, but “the only way to hide your location is to disconnect from our modern communication system and live in a cave.”

The NSA cannot know in advance which tiny fraction of 1 percent of the records it may need, so it collects and keeps as many as it can — 27 terabytes, by one account, or more than double the text content of the Library of Congress’s print collection.

The location programs have brought in such volumes of information, according to a May 2012 internal NSA briefing, that they are “outpacing our ability to ingest, process and store” data. In the ensuing year and a half, the NSA has been transitioning to a processing system that provided it with greater capacity.

The possibility that the intelligence community has been collecting location data, particularly of Americans, has long concerned privacy advocates and some lawmakers. Three Democratic senators — Ron Wyden (Ore.), Mark Udall (Colo.) and Barbara A. Mikulski (Md.) — have introduced an amendment to the 2014 defense spending bill that would require U.S. intelligence agencies to say whether they have ever collected or made plans to collect location data for “a large number of United States persons with no known connection to suspicious activity.”

NSA Director Keith B. Alexander disclosed in Senate testimony in October that the NSA had run a pilot project in 2010 and 2011 to collect “samples” of U.S. cellphone location data. The data collected were never available for intelligence analysis purposes, and the project was discontinued because it had no “operational value,” he said.

Alexander allowed that a broader collection of such data “may be something that is a future requirement for the country, but it is not right now.”

The number of Americans whose locations are tracked as part of the NSA’s collection of data overseas is impossible to determine from the Snowden documents alone, and senior intelligence officials declined to offer an estimate.

“It’s awkward for us to try to provide any specific numbers,” one intelligence official said in a telephone interview. An NSA spokeswoman who took part in the call cut in to say the agency has no way to calculate such a figure.

An intelligence lawyer, speaking with his agency’s permission, said location data are obtained by methods “tuned to be looking outside the United States,” a formulation he repeated three times. When U.S. cellphone data are collected, he said, the data are not covered by the Fourth Amendment, which protects Americans against unreasonable searches and seizures.

According to top-secret briefing slides, the NSA pulls in location data around the world from 10 major “sigads,” or signals intelligence activity designators.

A sigad known as STORMBREW, for example, relies on two unnamed corporate partners described only as ARTIFICE and WOLFPOINT. According to an NSA site inventory, the companies administer the NSA’s “physical systems,” or interception equipment, and “NSA asks nicely for tasking/updates.”

STORMBREW collects data from 27 telephone links known as OPC/DPC pairs, which refer to originating and destination points and which typically transfer traffic from one provider’s internal network to another’s. That data include cell tower identifiers, which can be used to locate a phone’s location.

The agency’s access to carriers’ networks appears to be vast.

“Many shared databases, such as those used for roaming, are available in their complete form to any carrier who requires access to any part of it,” said Matt Blaze, an associate professor of computer and information science at the University of Pennsylvania. “This ‘flat’ trust model means that a surprisingly large number of entities have access to data about customers that they never actually do business with, and an intelligence agency — hostile or friendly — can get ‘one-stop shopping’ to an expansive range of subscriber data just by compromising a few carriers.”

Some documents in the Snowden archive suggest that acquisition of U.S. location data is routine enough to be cited as an example in training materials. In an October 2012 white paper on analytic techniques, for example, the NSA’s counterterrorism analysis unit describes the challenges of tracking customers who use two different mobile networks, saying it would be hard to correlate a user on the T-Mobile network with one on Verizon. Asked about that, a U.S. intelligence official said the example was poorly chosen and did not represent the program’s foreign focus. There is no evidence that either company cooperates with the NSA, and both declined to comment.

The NSA’s capabilities to track location are staggering, based on the Snowden documents, and indicate that the agency is able to render most efforts at communications security effectively futile.

Like encryption and anonymity tools online, which are used by dissidents, journalists and terrorists alike, security-minded behavior — using disposable cellphones and switching them on only long enough to make brief calls — marks a user for special scrutiny. CO-TRAVELER takes note, for example, when a new telephone connects to a cell tower soon after another nearby device is used for the last time.

Side-by-side security efforts — when nearby devices power off and on together over time — “assist in determining whether co-travelers are associated . . . through behaviorally relevant relationships,” according to the 24-page white paper, which was developed by the NSA in partnership with the National Geospatial-Intelligence Agency, the Australian Signals Directorate and private contractors.

A central feature of each of these tools is that they do not rely on knowing a particular target in advance, or even suspecting one. They operate on the full universe of data in the NSA’s FASCIA repository, which stores trillions of metadata records, of which a large but unknown fraction include locations.

The most basic analytic tools map the date, time, and location of cellphones to look for patterns or significant moments of overlap. Other tools compute speed and trajectory for large numbers of mobile devices, overlaying the electronic data on transportation maps to compute the likely travel time and determine which devices might have intersected.

To solve the problem of un­detectable surveillance against CIA officers stationed overseas, one contractor designed an analytic model that would carefully record the case officer’s path and look for other mobile devices in steady proximity.

“Results have not been validated by operational analysts,” the report said.

Julie Tate contributed to this report. Soltani is an independent security researcher and consultant.

Americans worried about their lives being tracked often use snooping tools themselves

By Marc Fischer and Craig Timberg

Lynne Beliveau, right, and her daughter, Jessica Beliveau, 17, outside their home in Ashburn,Va. Beliveau says she would never use a tracking program just casually to check her daughter’s whereabouts. (Matt McClain/The Washington Post)

Julie Beliveau’s 16-year-old daughter, a new driver, was heading from her home in Ashburn toward a job interview the other night when she found herself in Leesburg — the wrong direction entirely. Upset and fearing that she’d blow the interview, she called her mother, who instantly launched her tracking program.

“I just opened my phone, and I could see where she was,” Beliveau said. Mother guided daughter to the interview, where she got the job. Score one for surveillance.

Yet Beliveau says she would never use the program just casually to check her daughter’s whereabouts. “That’s going over the line,” she said.

Amid this year’s revelations about the federal government’s vast apparatus for tracking the movements and communications of people worldwide, Americans are uneasy with the extent of surveillance yet often use snooping tools in their own lives, a Washington Post poll has found.

The sweet spot between liberty and security has been hard to pinpoint ever since the Sept. 11, 2001, attacks on New York and Washington. Remarkable advances in information technology have enabled counterterrorism tactics far more sweeping and intrusive — and powerful — than the United States had ever deployed. At the same time, the relationship between consumers and businesses was elementally altered as mobile phones, GPS, Google and Facebook gave corporations a new capacity to track their customers’ behavior.

This year, in the months since former National Security Agency contractor Edward Snowden leaked secret documents detailing U.S. surveillance programs, it has become clear that there are not yet widely accepted norms about who may watch whom and when and where tracking is justified. The Post’s poll found that Americans’ attitudes about surveillance are anything but consistent, whether the sample is the entire nation or a single, conflicted person.

Nearly seven in 10 Americans are concerned about how much personal information government agencies and private companies collect, the poll found. But among parents 40 or older — the group most likely to have teenagers — 70 percent said they monitor the Web sites their children visit. Many also review their kids’ texts, e-mails and social-media use. A small number of Americans also report tracking the movements of their spouses or using video feeds to monitor elderly parents.

Northern Virginia tech entrepreneur Zachary Thompson, 30, is typical in disliking government surveillance but embracing similar tools in his own life. Thompson, who owns an Internet service provider, YellowFiber Networks, has had to respond to federal court orders to hand over user data. “It sucks, to be blunt, because there’s nothing you can do,” he said.

But when it comes to his daughter — not yet 2 years old — he already plans to track her in a way that parents a generation ago could hardly have imagined. He and others said a parent’s relationship with a child is fundamentally different from a government’s with its citizens.

“I fully anticipate when she comes of age putting up a fire wall and monitoring everything she does,” he said. “That’s a parent’s responsibility.”

In Ashburn, the sprawling Northern Virginia suburb where development was supercharged by Loudoun County’s 1990s tech boom, massive data centers — the guts of the Internet — have replaced farms along rolling, formerly rural roads. People in Ashburn have lived and worked with the Internet since it first became widely used, yet even here, in townhouse communities and estate home developments built hard by the data centers, residents wrestle with how to distinguish between useful surveillance and unacceptable intrusions — by the NSA, by private companies and by family members.

Attitudes toward surveillance often vary depending on who is doing it and how clear the purpose is. Beliveau, for example, does not mind the tracking the government does to sniff out potential terrorists, because she believes innocent people have nothing to fear.

She is not particularly bothered by the intimate knowledge Wegmans collects about her grocery shopping, because she gains from the exchange — the store offers her discounts. But she finds the storehouse of information that Facebook and Google collect about users to be creepy and intrusive, because she does not see any benefit from giving them her data.

Nothing to hide

Go ahead, listen to my calls, track my movements, said Terry Brickerd, mother of college and high school students and vice president of the Broad Run High parents association: “I don’t care. I don’t have anything to hide. Because they’re not listening to us individually — they look for patterns. So what’s the big deal? I mean, without that, how many terror attacks would we have?”

Brickerd does not mind when companies track her purchases and online searches, either. “I’m glad American Express tracks me, because twice they’ve called me when something unusual happened, and that protected me,” she said.

The Post poll found Americans almost equally bothered by government surveillance as they are by corporate snooping, with 69 percent concerned about tracking by Internet search and social-media companies and 66 percent worried about what the government does. Overall, more-educated and affluent Americans were less likely to be concerned about surveillance. Political conservatives tended to be more concerned about government surveillance.

The survey did not find significant differences in attitudes toward government surveillance across age groups. Forty-five percent of Americans younger than 30, more than any other age group, said they were “very concerned” about how sites such as Facebook and Twitter use their information.

“What privacy? On the Internet, there is almost no privacy,” said Austin McCuiston, 19, a food runner at Ford’s Fish Shack in Ashburn, near the epicenter of Loudoun’s 4.5 million square feet of data centers. He is very cautious about what he posts on Facebook or other Internet services. “People can take that stuff and really dig into your life.”

Bob Moses, 50, an information technology worker for the AFL-CIO, said he would like more anonymity when he shops online, but he understands that Google and others offer their services without charge and need a way to make money.

“For that, I give up some of my rights,” Moses said. “It’s a trade-off I accept, at least right now.” He’s similarly sanguine about the government’s tracking. “It’s not the details that the NSA is harvesting, but it’s the relationships,” he said. “If you’ve got something to hide, then you ought to be worried about it.”

Moses appreciates technology’s capacity to keep his family safe. When his children were younger, he demanded to be their friends on Facebook so he could monitor their activity. He recently helped a colleague buy a “granny cam” for the home of her elderly mother, who struggles with dementia. And he used an AT&T service called “FamilyMap” to track the movements of his children, ages 17 and 22 — stopping with his older child only when she moved out of the house.

“It’s peace of mind,” Moses said. “It’s the 21st-century family.”

Most Americans seem to have made their peace with video surveillance cameras, which are now widely used by governments and businesses, especially in densely populated areas. In The Post’s poll, more than four out of five Americans were comfortable with the number of cameras in use or even would favor having more installed. Only 14 percent would like to see fewer cameras.

But about half of Americans wanted limits on how long police may keep location data on citizens. Such data are collected by advanced video surveillance systems, license-plate readers and other technologies.

“Whatever you do on your phone, you shouldn’t mind anybody seeing it,” said Nia Farmer, 18, a student at Howard University whose family owns a place in Ashburn Village. She is fine with NSA efforts to locate terrorists, even if it means collecting information from her phone. “That’s all there to protect us,” she said.

Farmer is even fine with her mother’s insistence on tracking her whereabouts until she turns 25. After all, her mother has been watching her movements remotely since she got her first phone at age 10. “Legally, I’m an adult,” Farmer said, “but I keep it on for her because it’s all about staying safe. Anyway, if I turn off the app, she gets right on the phone, so I might as well just keep it on.”

In the few years since smartphones, social media and the plummeting cost of video technology made it cheap and easy for people to track each other, Americans have grown so comfortable with these technologies that large majorities say they take little or no precautions to protect their digital privacy. Nearly six in 10 Internet users do not use tools that can block Web sites from tracking their behavior, seven in 10 say they have not deleted online posts that might be embarrassing, and more than eight in 10 say they never encrypt their communications or use tools that allow people to browse anonymously.

Those who act to defend their data are more likely to be male, conservative and well educated.

John Burke, 70, is retired from a career in federal law enforcement and has been disappointed, even angered, by news reports about the NSA’s approach to collecting data from U.S. citizens. Burke eschews Facebook and other social media, avoids giving out his Social Security number and tries to steer clear of businesses that sell customer information to other marketers, but he wonders if the effort is worthwhile. “I doubt my precautions are very effective,” he said. “We really have no way of knowing what anyone does with our information, and especially what the government does.”

Many Americans expressed similar concerns; the poll found that only 9 percent of Internet users are “very confident” that their efforts will protect their privacy. That lack of clarity drives some people to search for new ways to protect their information, while others shrug and conclude that they are powerless.

“What worries you is sometimes you don’t know what the scope of it or the scale of it is,” said Tarek El-Ghazawi, 55, a computer engineering professor and director of the High Performance Computing Lab at George Washington University’s campus in Ashburn. “But you know you are giving up part of your life.”

El-Ghazawi, a U.S. citizen who emigrated from Egypt three decades ago and has seen many other societies through his travel for work, said that, by comparison, he trusts that the press, Congress and the courts will help curb the NSA’s excesses.

“These are things that are missing in many countries,” he said. “I’m not saying that the government is perfect. I’m just saying that the government may be self-correcting over time.”

But he does not see similar protections against overreach by private companies. “They are trying to make a quick profit,” he said, “and there’s not a standard in terms of ethics.”

Every time she goes to a CVS, Peggy Brown gets evidence that the drugstore tracks her purchases. The personalized discount offer she is handed with her receipt “shows that they know what medications I’ve gotten,” she said. “And that bothers me more than what the government’s doing, because they’re doing things to protect us.”

But Brown, 59 and a recent retiree from a job in a medical office, figures there is little the average person can do to protect personal data. “Everything just gets more and more exposed,” she said.

Some of those who decide not to monitor family members argue that using such technologies undermines trust, effectively declaring that ordinary human connections are insufficient.

Jessica Beliveau, a junior at Broad Run High School and Julie Beliveau’s niece, said she appreciates her mother Lynne’s decision not to track her whereabouts. “I feel there should be some things that parents don’t necessarily know,” Jessica said. “And anyway, in the past, before all this technology, most kids turned out just fine.”

For families, a fine line?

But many parents say they would be shirking their responsibility if they did not take advantage of available technology to monitor their children’s online behavior.

One mother in Ashburn who spoke on the condition of anonymity to avoid embarrassing her child, said she and her husband read through their middle-school-age daughter’s Facebook chat transcripts and discovered that boys had been pressuring her to send revealing photos of herself. The parents intervened and prevented a sexting incident; now, a few years later, they have decided not to monitor their daughter’s Facebook account, to teach her the benefits of trust.

The struggle over just how much to watch varies from family to family, and no clear guidelines have emerged, according to school counselors. Some teens are now being monitored not only by parents but also by their schools, a few thousand of which have contracted with a California company, Geo Listening, to sift through students’ social-media postings to look for potentially dangerous situations.

As attitudes shift, businesses keep coming up with new ways for consumers to keep tabs on relatives, friends and colleagues.

Although the Post poll found that only 6 percent of those surveyed use “granny cams” to monitor elderly relatives, market research indicates that the popularity of such products is likely to grow sharply. Life360, an app that allows families or friends to share their locations on their phones, has been downloaded more than 60 million times. “More children than adults say they feel safer” when their location is being monitored, said Amanda Zweerink, a Life360 vice president.

Julia Roberts, a 48-year-old writer who uses Life360 to check on her children, ages 12 and 14, and her husband, started using the program even before her kids ventured out on their own, to get them used to the idea that mother would always be watching.

“I pay for their phones, so it’s part of the bargain,” she said. Her daughter, who is 12, protests against the monitoring, but Roberts responds that even parents in past generations kept tabs on kids. “I always had to give my mother an itinerary of where I was going to be,” she said.

Roberts said that if she is going to use technology to keep her family safe, it is reasonable for the government to do the same to protect the nation. “I don’t care if somebody in the government listens to my phone calls,” she said. “I don’t mind being checked on. I don’t mind being tracked. And our children will care even less, because they’re growing up with all this, always connected. It’s just who we are.”

Peyton Craighill and Scott Clement contributed to this report. The poll was conducted Nov. 14 to 17 among a random national sample of 1,006 adults, including land-line and cellphone respondents. Overall results have a margin of sampling error of 3.5 percentage points.

By Barton Gellman

MOSCOW — The familiar voice on the hotel room phone did not waste words.

“What time does your clock say, exactly?” he asked.

He checked the reply against his watch and described a place to meet.

“I’ll see you there,” he said.

Edward Joseph Snowden emerged at the appointed hour, alone, blending into a light crowd of locals and tourists. He cocked his arm for a handshake, then turned his shoulder to indicate a path. Before long he had guided his visitor to a secure space out of public view.

During more than 14 hours of interviews, the first he has conducted in person since arriving here in June, Snowden did not part the curtains or step outside. Russia granted him temporary asylum on Aug. 1, but Snowden remains a target of surpassing interest to the intelligence services whose secrets he spilled on an epic scale.

Late this spring, Snowden supplied three journalists, including this one, with caches of top-secret documents from the National Security Agency, where he worked as a contractor. Dozens of revelations followed, and then hundreds, as news organizations around the world picked up the story. Congress pressed for explanations, new evidence revived old lawsuits and the Obama administration was obliged to declassify thousands of pages it had fought for years to conceal.

Taken together, the revelations have brought to light a global surveillance system that cast off many of its historical restraints after the attacks of Sept. 11, 2001. Secret legal authorities empowered the NSA to sweep in the telephone, Internet and location records of whole populations. One of the leaked presentation slides described the agency’s “collection philosophy” as “Order one of everything off the menu.”

Six months after the first revelations appeared in The Washington Post and Britain’s Guardian newspaper, Snowden agreed to reflect at length on the roots and repercussions of his choice. He was relaxed and animated over two days of nearly unbroken conversation, fueled by burgers, pasta, ice cream and Russian pastry.

Snowden offered vignettes from his intelligence career and from his recent life as “an indoor cat” in Russia. But he consistently steered the conversation back to surveillance, democracy and the meaning of the documents he exposed.

“For me, in terms of personal satisfaction, the mission’s already accomplished,” he said. “I already won. As soon as the journalists were able to work, everything that I had been trying to do was validated. Because, remember, I didn’t want to change society. I wanted to give society a chance to determine if it should change itself.”

“All I wanted was for the public to be able to have a say in how they are governed,” he said. “That is a milestone we left a long time ago. Right now, all we are looking at are stretch goals.”

‘Going in blind’

Snowden is an orderly thinker, with an engineer’s approach to problem-solving. He had come to believe that a dangerous machine of mass surveillance was growing unchecked. Closed-door oversight by Congress and the Foreign Intelligence Surveillance Court was a “graveyard of judgment,” he said, manipulated by the agency it was supposed to keep in check. Classification rules erected walls to prevent public debate.

Toppling those walls would be a spectacular act of transgression against the norms that prevailed inside them. Someone would have to bypass security, extract the secrets, make undetected contact with journalists and provide them with enough proof to tell the stories.

 

The NSA’s business is “information dominance,” the use of other people’s secrets to shape events. At 29, Snowden upended the agency on its own turf.

“You recognize that you’re going in blind, that there’s no model,” Snowden said, acknowledging that he had no way to know whether the public would share his views.

“But when you weigh that against the alternative, which is not to act,” he said, “you realize that some analysis is better than no analysis. Because even if your analysis proves to be wrong, the marketplace of ideas will bear that out. If you look at it from an engineering perspective, an iterative perspective, it’s clear that you have to try something rather than do nothing.”

By his own terms, Snowden succeeded beyond plausible ambition. The NSA, accustomed to watching without being watched, faces scrutiny it has not endured since the 1970s, or perhaps ever.

The cascading effects have made themselves felt in Congress, the courts, popular culture, Silicon Valley and world capitals. The basic structure of the Internet itself is now in question, as Brazil and members of the European Union consider measures to keep their data away from U.S. territory and U.S. technology giants including Google, Microsoft and Yahoo take extraordinary steps to block the collection of data by their government.

For months, Obama administration officials attacked Snowden’s motives and said the work of the NSA was distorted by selective leaks and misinterpretations.

On Dec. 16, in a lawsuit that could not have gone forward without the disclosures made possible by Snowden, U.S. District Judge Richard J. Leon described the NSA’s capabilities as “almost Orwellian” and said its bulk collection of U.S. domestic telephone records was probably unconstitutional.

The next day, in the Roosevelt Room, an unusual delegation of executives from old telephone companies and young Internet firms told President Obama that the NSA’s intrusion into their networks was a threat to the U.S. information economy. The following day, an advisory panel appointed by Obama recommended substantial new restrictions on the NSA, including an end to the domestic call-records program.

“This week is a turning point,” said the Government Accountability Project’s Jesselyn Radack, who is one of Snowden’s legal advisers. “It has been just a cascade.”

‘They elected me’

On June 22, the Justice Department unsealed a criminal complaint charging Snowden with espionage and felony theft of government property. It was a dry enumeration of statutes, without a trace of the anger pulsing through Snowden’s former precincts.

In the intelligence and national security establishments, Snowden is widely viewed as a reckless saboteur, and journalists abetting him little less so.

At the Aspen Security Forum in July, a four-star military officer known for his even keel seethed through one meeting alongside a reporter he knew to be in contact with Snowden. Before walking away, he turned and pointed a finger.

“We didn’t have another 9/11,” he said angrily, because intelligence enabled warfighters to find the enemy first. “Until you’ve got to pull the trigger, until you’ve had to bury your people, you don’t have a clue.”

It is commonly said of Snowden that he broke an oath of secrecy, a turn of phrase that captures a sense of betrayal. NSA Director Keith B. Alexander and Director of National Intelligence James R. Clapper Jr., among many others, have used that formula.

In his interview with The Post, Snowden noted matter-of-factly that Standard Form 312, the ­classified-information nondisclosure agreement, is a civil contract. He signed it, but he pledged his fealty elsewhere.

“The oath of allegiance is not an oath of secrecy,” he said. “That is an oath to the Constitution. That is the oath that I kept that Keith Alexander and James Clapper did not.”

People who accuse him of disloyalty, he said, mistake his purpose.

“I am not trying to bring down the NSA, I am working to improve the NSA,” he said. “I am still working for the NSA right now. They are the only ones who don’t realize it.”

What entitled Snowden, now 30, to take on that responsibility?

“That whole question — who elected you? — inverts the model,” he said. “They elected me. The overseers.”

He named the chairmen of the Senate and House intelligence committees.

“Dianne Feinstein elected me when she asked softball questions” in committee hearings, he said. “Mike Rogers elected me when he kept these programs hidden. . . . The FISA court elected me when they decided to legislate from the bench on things that were far beyond the mandate of what that court was ever intended to do. The system failed comprehensively, and each level of oversight, each level of responsibility that should have addressed this, abdicated their responsibility.”

“It wasn’t that they put it on me as an individual — that I’m uniquely qualified, an angel descending from the heavens — as that they put it on someone, somewhere,” he said. “You have the capability, and you realize every other [person] sitting around the table has the same capability but they don't do it. So somebody has to be the first.”

‘Front-page test’

Snowden grants that NSA employees by and large believe in their mission and trust the agency to handle the secrets it takes from ordinary people — deliberately, in the case of bulk records collection, and “incidentally,” when the content of American phone calls and e-mails are swept into NSA systems along with foreign targets.

But Snowden also said acceptance of the agency’s operations was not universal. He began to test that proposition more than a year ago, he said, in periodic conversations with co-workers and superiors that foreshadowed his emerging plan.

Beginning in October 2012, he said, he brought his misgivings to two superiors in the NSA’s Technology Directorate and two more in the NSA Threat Operations Center’s regional base in Hawaii. For each of them, and 15 other co-workers, Snowden said he opened a data query tool called BOUNDLESSINFORMANT, which used color-coded “heat maps” to depict the volume of data ingested by NSA taps.

His colleagues were often “astonished to learn we are collecting more in the United States on Americans than we are on Russians in Russia,” he said. Many of them were troubled, he said, and several said they did not want to know any more.

“I asked these people, ‘What do you think the public would do if this was on the front page?’ ” he said. He noted that critics have accused him of bypassing internal channels of dissent. “How is that not reporting it? How is that not raising it?” he said.

By last December, Snowden was contacting reporters, although he had not yet passed along any classified information. He continued to give his colleagues the “front-page test,” he said, until April.

Asked about those conversations, NSA spokeswoman Vanee Vines sent a prepared statement to The Post: “After extensive investigation, including interviews with his former NSA supervisors and co-workers, we have not found any evidence to support Mr. Snowden’s contention that he brought these matters to anyone’s attention.”

Snowden recounted another set of conversations that he said took place three years earlier, when he was sent by the NSA’s Technology Directorate to support operations at a listening post in Japan. As a system administrator, he had full access to security and auditing controls. He said he saw serious flaws with information security.

“I actually recommended they move to two-man control for administrative access back in 2009,” he said, first to his supervisor in Japan and then to the directorate’s chief of operations in the Pacific. “Sure, a whistleblower could use these things, but so could a spy.”

That precaution, which requires a second set of credentials to perform risky operations such as copying files onto a removable drive, has been among the principal security responses to the Snowden affair.

Vines, the NSA spokeswoman, said there was no record of those conversations, either.

U.S. ‘would cease to exist’

Just before releasing the documents this spring, Snowden made a final review of the risks. He had overcome what he described at the time as a “selfish fear” of the consequences for himself.

“I said to you the only fear [left] is apathy — that people won’t care, that they won’t want change,” he recalled this month.

The documents leaked by Snowden compelled attention because they revealed to Americans a history they did not know they had.

Internal briefing documents reveled in the “Golden Age of Electronic Surveillance.” Brawny cover names such as MUSCULAR, TUMULT and TURMOIL boasted of the agency’s prowess.

With assistance from private communications firms, the NSA had learned to capture enormous flows of data at the speed of light from fiber-optic cables that carried Internet and telephone traffic over continents and under seas. According to one document in Snowden’s cache, the agency’s Special Source Operations group, which as early as 2006 was said to be ingesting “one Library of Congress every 14.4 seconds,” had an official seal that might have been parody: an eagle with all the world’s cables in its grasp.

Each year, NSA systems collected hundreds of millions of e-mail address books, hundreds of billions of cellphone location records and trillions of domestic call logs.

Most of that data, by definition and intent, belonged to ordinary people suspected of nothing. But vast new storage capacity and processing tools enabled the NSA to use the information to map human relationships on a planetary scale. Only this way, its leadership believed, could the NSA reach beyond its universe of known intelligence targets.

In the view of the NSA, signals intelligence, or electronic eavesdropping, was a matter of life and death, “without which America would cease to exist as we know it,” according to an internal presentation in the first week of October 2001 as the agency ramped up its response to the al-Qaeda attacks on the World Trade Center and the Pentagon.

With stakes such as those, there was no capability the NSA believed it should leave on the table. The agency followed orders from President George W. Bush to begin domestic collection without authority from Congress and the courts. When the NSA won those authorities later, some of them under secret interpretations of laws passed by Congress between 2007 and 2012, the Obama administration went further still.

Using PRISM, the cover name for collection of user data from Google, Yahoo, Microsoft, Apple and five other U.S.-based companies, the NSA could obtain all communications to or from any specified target. The companies had no choice but to comply with the government's request for data.

But the NSA could not use PRISM, which was overseen once a year by the surveillance court, for the collection of virtually all data handled by those companies. To widen its access, it teamed up with its British counterpart, Government Communications Headquarters, or GCHQ, to break into the private fiber-optic links that connected Google and Yahoo data centers around the world.

 

That operation, which used the cover name MUSCULAR, tapped into U.S. company data from outside U.S. territory. The NSA, therefore, believed it did not need permission from Congress or judicial oversight. Data from hundreds of millions of U.S. accounts flowed over those Google and Yahoo links, but classified rules allowed the NSA to presume that data ingested overseas belonged to foreigners.

 

‘Persistent threat’

Disclosure of the MUSCULAR project enraged and galvanized U.S. technology executives. They believed the NSA had lawful access to their front doors — and had broken down the back doors anyway.

Microsoft general counsel Brad Smith took to his company’s blog and called the NSA an “advanced persistent threat” — the worst of all fighting words in U.S. cybersecurity circles, generally reserved for Chinese state-sponsored hackers and sophisticated criminal enterprises.

“For the industry as a whole, it caused everyone to ask whether we knew as much as we thought,” Smith recalled in an interview. “It underscored the fact that while people were confident that the U.S. government was complying with U.S. laws for activity within U.S. territory, perhaps there were things going on outside the United States . . . that made this bigger and more complicated and more disconcerting than we knew.”

They wondered, he said, whether the NSA was “collecting proprietary information from the companies themselves.”

Led by Google and then Yahoo, one company after another announced expensive plans to encrypt its data traffic over tens of thousands of miles of cable. It was a direct — in some cases, explicit — blow to NSA collection of user data in bulk. If the NSA wanted the information, it would have to request it or circumvent the encryption one target at a time.

As these projects are completed, the Internet will become a less friendly place for the NSA to work. The agency can still collect data from virtually anyone, but collecting from everyone will be harder.

The industry’s response, Smith acknowledged, was driven by a business threat. U.S. companies could not afford to be seen as candy stores for U.S. intelligence. But the principle of the thing, Smith said, “is fundamentally about ensuring that customer data is turned over to governments pursuant to valid legal orders and in accordance with constitutional principles.”

‘Warheads on foreheads’

Snowden has focused on much the same point from the beginning: Individual targeting would cure most of what he believes is wrong with the NSA.

Six months ago, a reporter asked him by encrypted e-mail why Americans would want the NSA to give up bulk data collection if that would limit a useful intelligence tool.

“I believe the cost of frank public debate about the powers of our government is less than the danger posed by allowing these powers to continue growing in secret,” he replied, calling them “a direct threat to democratic governance.”

In the Moscow interview, Snowden said, “What the government wants is something they never had before,” adding: “They want total awareness. The question is, is that something we should be allowing?”

Snowden likened the NSA’s powers to those used by British authorities in Colonial America, when “general warrants” allowed for anyone to be searched. The FISA court, Snowden said, “is authorizing general warrants for the entire country’s metadata.”

“The last time that happened, we fought a war over it,” he said.

Technology, of course, has enabled a great deal of consumer surveillance by private companies, as well. The difference with the NSA’s possession of the data, Snowden said, is that government has the power to take away life or freedom.

At the NSA, he said, “there are people in the office who joke about, ‘We put warheads on foreheads.’ Twitter doesn’t put warheads on foreheads.”

Privacy, as Snowden sees it, is a universal right, applicable to American and foreign surveillance alike.

“I don’t care whether you’re the pope or Osama bin Laden,” he said. “As long as there’s an individualized, articulable, probable cause for targeting these people as legitimate foreign intelligence, that’s fine. I don’t think it’s imposing a ridiculous burden by asking for probable cause. Because, you have to understand, when you have access to the tools the NSA does, probable cause falls out of trees.”

‘Everybody knows’

On June 29, Gilles de Kerchove, the European Union’s counter­terrorism coordinator, awoke to a report in Der Spiegel that U.S. intelligence had broken into E.U. offices, including his, to implant surveillance devices.

The 56-year-old Belgian, whose work is often classified, did not consider himself naive. But he took the news personally, and more so when he heard unofficial explanations from Washington.

“ ‘Everybody knows. Everybody does’ — Keith Alexander said that,” de Kerchove said in an interview. “I don’t like the idea that the NSA will put bugs in my office. No. I don’t like it. No. Between allies? No. I’m surprised that people find that noble.”

Comparable reactions, expressed less politely in private, accompanied revelations that the NSA had tapped the cellphones of German Chancellor Angela Merkel and Brazilian President Dilma Rousseff. The blowback roiled relations with both allies, among others. Rousseff canceled a state dinner with Obama in September.

When it comes to spying on allies, by Snowden’s lights, the news is not always about the target.

“It’s the deception of the government that’s revealed,” Snowden said, noting that the Obama administration offered false public assurances after the initial reports about NSA surveillance in Germany “The U.S. government said: ‘We follow German laws in Germany. We never target German citizens.’ And then the story comes out and it’s: ‘What are you talking about? You’re spying on the chancellor.’ You just lied to the entire country, in front of Congress.”

In private, U.S. intelligence officials still maintain that spying among friends is routine for all concerned, but they are giving greater weight to the risk of getting caught.

“There are many things we do in intelligence that, if revealed, would have the potential for all kinds of blowback,” Clapper told a House panel in October.

‘They will make mistakes’

U.S. officials say it is obvious that Snowden’s disclosures will do grave harm to intelligence gathering, exposing methods that adversaries will learn to avoid.

“We’re seeing al-Qaeda and related groups start to look for ways to adjust how they communicate,” said Matthew Olsen, director of the National Counterterrorism Center and a former general counsel at the NSA.

Other officials, who declined to speak on the record about particulars, said they had watched some of their surveillance targets, in effect, changing channels. That evidence can be read another way, they acknowledged, given that the NSA managed to monitor the shift.

Clapper has said repeatedly in public that the leaks did great damage, but in private he has taken a more nuanced stance. A review of early damage assessments in previous espionage cases, he said in one closed-door briefing this fall, found that dire forecasts of harm were seldom borne out.

“People must communicate,” he said, according to one participant who described the confidential meeting on the condition of anonymity. “They will make mistakes, and we will exploit them.”

According to senior intelligence officials, two uncertainties feed their greatest concerns. One is whether Russia or China managed to take the Snowden archive from his computer, a worst-case assumption for which three officials acknowledged there is no evidence.

In a previous assignment, Snowden taught U.S. intelligence personnel how to operate securely in a “high-threat digital environment,” using a training scenario in which China was the designated threat. He declined to discuss the whereabouts of the files, but he said that he is confident he did not expose them to Chinese intelligence in Hong Kong. And he said he did not bring them to Russia.

“There’s nothing on it,” he said, turning his laptop screen toward his visitor. “My hard drive is completely blank.”

The other big question is how many documents Snowden took. The NSA’s incoming deputy director, Rick Ledgett, said on CBS’s “60 Minutes” recently that the number may approach 1.7 million, a huge and unexplained spike over previous estimates. Ledgett said he would favor trying to negotiate an amnesty with Snowden in exchange for “assurances that the remainder of the data could be secured.”

Obama’s national security adviser, Susan E. Rice, later dismissed the possibility.

“The government knows where to find us if they want to have a productive conversation about resolutions that don’t involve Edward Snowden behind bars,” said the American Civil Liberties Union’s Ben Wizner, the central figure on Snowden’s legal team.

Some news accounts have quoted U.S. government officials as saying Snowden has arranged for the automated release of sensitive documents if he is arrested or harmed. There are strong reasons to doubt that, beginning with Snowden’s insistence, to this reporter and others, that he does not want the documents published in bulk.

If Snowden were fool enough to rig a “dead man’s switch,” confidants said, he would be inviting anyone who wants the documents to kill him.

Asked about such a mechanism in the Moscow interview, Snowden made a face and declined to reply. Later, he sent an encrypted message. “That sounds more like a suicide switch,” he wrote. “It wouldn’t make sense.”

‘It’s not about me’

By temperament and circumstance, Snowden is a reticent man, reluctant to discuss details about his personal life.

Over two days his guard never dropped, but he allowed a few fragments to emerge. He is an “ascetic,” he said. He lives off ramen noodles and chips. He has visitors, and many of them bring books. The books pile up, unread. The Internet is an endless library and a window on the progress of his cause.

“It has always been really difficult to get me to leave the house,” he said. “I just don’t have a lot of needs. . . . Occasionally there’s things to go do, things to go see, people to meet, tasks to accomplish. But it’s really got to be goal-oriented, you know. Otherwise, as long as I can sit down and think and write and talk to somebody, that’s more meaningful to me than going out and looking at landmarks.”

In hope of keeping focus on the NSA, Snowden has ignored attacks on himself.

“Let them say what they want,” he said. “It’s not about me.”

Former NSA and CIA director Michael V. Hayden predicted that Snowden will waste away in Moscow as an alcoholic, like other “defectors.” To this, Snowden shrugged. He does not drink at all. Never has.

But Snowden knows his presence here is easy ammunition for critics. He did not choose refuge in Moscow as a final destination. He said that once the U.S. government voided his passport as he tried to change planes en route to Latin America, he had no other choice.

It would be odd if Russian authorities did not keep an eye on him, but no retinue accompanied Snowden and his visitor saw no one else nearby. Snowden neither tried to communicate furtively nor asked that his visitor do so. He has had continuous Internet access and has talked to his attorneys and to journalists daily, from his first day in the transit lounge at Sheremetyevo airport.

“There is no evidence at all for the claim that I have loyalties to Russia or China or any country other than the United States,” he said. “I have no relationship with the Russian government. I have not entered into any agreements with them.”

“If I defected at all,” Snowden said, “I defected from the government to the public.”

Julie Tate contributed to this report.

To the judges:

In the massive cache of top-secret documents he leaked, Edward Snowden, the former contractor for the National Security Agency, included a simple drawing. The sketch showed two clouds — one where the “public Internet” resides and another where Google maintains its oceans of data. Various arrows pointed in various directions. The drawing would have meant nothing to a layperson. When The Washington Post showed it to two engineers with close ties to Google, they exploded in profanity. “I hope you publish this,” one of them said.

The Post did.

The story that resulted from that reporting revealed for the first time that the NSA was secretly breaking into the main communications links that connect Google and Yahoo data centers around the world. That clandestine program allowed the NSA to collect communications from hundreds of millions of user accounts, including those belonging to Americans. Over the following several weeks, several major U.S. technology companies, fearful that their networks were vulnerable, announced they were launching new efforts to encrypt their Internet traffic. Responding to this and other revelations about the NSA’s capabilities, lawmakers have pressed the top-secret agency for answers about its programs and advanced new legislation to curb its authorities.

For six months, The Post has been on the leading edge of reporting on the Snowden documents. It began by becoming the first news outlet to disclose PRISM, a massive program to vacuum up e-mails, documents and other electronic records from the largest U.S. Internet companies. Later, The Post revealed the NSA’s repeated violations of its own privacy rules; examined the workings of the secretive federal court overseeing surveillance activities; exposed the NSA’s clandestine collection of millions of e-mail address books globally; and broke the news that the agency was gathering nearly 5 billion records a day on the whereabouts of cellphones around the world.

The NSA had successfully shielded these programs from scrutiny for years. It did so, however, at the expense of any real discussion about the sometimes uneasy balance between individual privacy and national security, and about the power and responsibility the American people have invested in a relative few. The Post’s stories – based on reporting that went far beyond the documents themselves — were central to making that that debate possible.

The Post dug into the most complicated corners of the Snowden trove, putting a team of reporters with deep technical and legal expertise to work to break story after story. On their own, the NSA documents are enormously difficult to decipher. But The Post did, and then kept going beyond the documents, detailing the workings not only of the NSA but also of the broader intelligence community. 

Government officials have argued that the disclosures have undermined national security. But others— including those who are just as passionate about the need for transparency in a democratic system — have embraced The Post’s reporting as an opportunity to demand reform. Sen. Dianne Feinstein (D-Calif.) said on national television that a “front-page story in The Washington Post with respect to the Foreign Intelligence Surveillance Court probably put more transparency on that court than anything in the history of a secret organization.” A few months later, the general counsel for Microsoft compared The Post’s revelations about the NSA’s hacking capability to “an earthquake, sending shock waves across the tech sector.”

The Post’s stories have brought new attention not only to the NSA but also to the broader U.S. intelligence community. Since the attacks of Sept. 11, 2001, that intelligence community, flush with billions of dollars in taxpayer money and aided by technological advances that were once unthinkable, has built a virtual surveillance empire. It has done so, however, without any effort to increase transparency and with only limited accountability.

Before reporting based on the Snowden documents, few Americans had even the most basic understanding of how the intelligence community operated. For decades, for instance, the government refused to declassify even portions of its “black budget,” which outlined basic spending priorities on intelligence.

The Post shattered that secrecy, publishing an in-depth story based on the budget summary for fiscal 2013, disclosing unprecedented details about spending levels in graphics in print and online, and following up with additional articles, including one about previously unknown offensive cyberoperations.

Lee H. Hamilton, an Indiana Democrat who chaired the House Intelligence Committee and co-chaired the commission that investigated the Sept. 11 attacks, explained the importance of transparency in The Post’s budget story: “Much of the work that the intelligence community does has a profound impact on the life of ordinary Americans,” he said. They “ought not to be excluded from the process.”

That sentiment has been the driving force behind The Post’s reporting on the Snowden documents. Its careful, exhaustive and penetrating work has given meaning to arcane government programs that, at their core, affect Americans’ lives in fundamental ways, from their security to their right to privacy. For that reason among many, we are proud to nominate The Post’s coverage of the NSA and the intelligence community for the Pulitzer Prize for Public Service.

Sincerely,

Martin Baron

Executive Editor