Reporting from Washington — Internet entrepreneur Nicholas Merrill was working in his Manhattan office when an FBI agent in a trench coat arrived with an envelope.
It was fall 2004, and federal investigators were using new legal authority they had acquired after Sept. 11, 2001. Merrill ran a small Internet service provider with clients including IKEA, Mitsubishi and freelance journalists.
The agent handed Merrill a document called a National Security Letter, which demanded that he turn over detailed records on one of his customers. The letter wasn't signed by a judge or prosecutor. It instructed him to tell no one.
"Not even my lawyer? Not even my business partners?" Merrill asked.
The agent shrugged and left.
Merrill had gotten a rare glimpse of the secret domestic intelligence gathering that is one of the most significant legacies of Sept. 11. U.S. law enforcement and intelligence agencies now collect, store and analyze vast quantities of digital data produced by law-abiding Americans. The data mining receives limited congressional oversight, rare judicial review and almost no public scrutiny.
Thanks to new laws and technologies, authorities track and eavesdrop on Americans as they never could before, hauling in billions of bank records, travel receipts and other information. In several cases, they have wiretapped conversations between lawyers and defendants, challenging the legal principle that attorney-client communication is inviolate.
Advocates say the expanded surveillance has helped eliminate vulnerabilities identified after the Sept. 11 attacks. Some critics, unconvinced, say the snooping undermines privacy and civil liberties and leads inevitably to abuse. They argue that the new systems have weakened security by burying investigators in irrelevant information.
"We are caught in the middle of a perfect storm in which every thought we communicate, every step we take, every transaction we enter into is captured in digital data and is subject to government collection," said Fred H. Cate, a professor at the Indiana University Maurer School of Law who has written extensively on privacy and security.
A robust debate on the intelligence gathering has been impossible, for the simple reason that most of the activity is officially secret. In lawsuits alleging improper eavesdropping, the Justice Department has invoked state secrecy to prevent disclosure of classified information and systems.
In May, two members of the Senate Intelligence Committee said that Americans would be disturbed if they knew about some of the government's data-gathering procedures. But Sens. Ron Wyden (D-Ore.) and Mark Udall (D-Colo.) said they were prohibited from revealing the facts.
"When the American people find out how their government has secretly interpreted" surveillance law, "they will be stunned and they will be angry," Wyden said.
The National Security Agency, which eavesdrops on foreign targets, once had to get a court-approved warrant to monitor a U.S. citizen's communications over wires that traverse the United States. Now the agency is free to vacuum up communications by Americans and foreigners alike, as long as the target of the surveillance is a foreigner.
Exactly what records are kept and how they are used is not well understood even by lawmakers who oversee the intelligence agencies, said Rep. Rush D. Holt (D-N.J.), who chaired the now-expired Select Intelligence Oversight Panel.
"The NSA finds it pretty easy to snow members of Congress by confusing them," Holt said in an interview.
Officials from the FBI and NSA say they follow strict rules to avoid abuses. But in 2007, the Justice Department's inspector general found that the FBI had engaged in "serious misuse" of its authority to issue National Security Letters, claiming urgency in cases where when none existed
Such letters, a kind of administrative subpoena, are key to the increased surveillance.
10 Years After: Are We Safer?
So the government has used National Security Letters to get the data, issuing 192,500 of the letters between 2003 and 2006, according to an audit by the Justice Department inspector general. The numbers have dropped sharply since then, but the FBI issued 24,287 National Security Letters last year for data on 14,212 Americans. That's up from a few thousand letters a year before 2001.
"It used to be the case that if the government wanted to find out what you read and what you wrote, it would have to get a warrant and search your home," said Daniel J. Solove, a law professor at George Washington University and the author of numerous books and articles on privacy law.
Now, "it just obtains your Amazon purchase records, your Facebook posts, your Internet browsing history — without you even knowing."
There is nothing necessarily wrong with that, advocates argue.
"As we put more data in the cloud, as we share more data online, we become less shocked when the police have access to it," said Stewart Baker, a former NSA general counsel and policy chief at the Homeland Security Department.
Privacy activists disagree.
"I think it's a world of difference between what a person decides to post publicly and what the FBI collects about them secretly," said Gregory Nojeim, senior counsel at the Center for Democracy & Technology, a Washington-based civil liberties group.
U.S. intelligence officials insist that the new surveillance powers have been crucial to stopping terrorist plots.
They cite the case of Najibullah Zazi, an Afghan American who planned to bomb New York City subways in 2009. Warrants were obtained under the Foreign Intelligence Surveillance Act, or FISA, to search Zazi's vehicle and eavesdrop on his calls. The evidence was used to secure his guilty plea to terrorism charges.
Unlike a search warrant in a criminal case, obtaining a FISA warrant does not require convincing a judge that there is probable cause to believe a crime was committed. Instead, the government must show probable cause that the target is an agent of a foreign power. Because of the different legal standard, information gathered from FISA warrants tended not to be used in criminal cases a decade ago.
Now that line has been blurred. In the Zazi case, the wiretapped conversations were revealed during pretrial discovery and are believed to have helped persuade him to plead guilty.
"Zazi is a very good example of the melding of intelligence authorities and criminal authorities," said a senior law enforcement official, speaking on condition of anonymity. "We needed to move quickly, and we never could have done it like that" before Sept. 11.
The Zazi case revealed another new reality. Earlier this year, the government disclosed it had recorded 43 conversations between Zazi's codefendant, Adis Medunjanin, and his lawyer, Robert Gottlieb. With rare exceptions, such conversations are off-limits to investigators in criminal cases — unless they obtain a FISA warrant.
FISA warrants also enabled the FBI to bug the phones and break into the home of Oregon lawyer Brandon Mayfield, a convert to Islam, after a faulty FBI fingerprint analysis linked him to the 2004 Madrid train bombings.
The FBI initially refused to tell Mayfield or his family why or where he was being held. He wasn't released until Spanish authorities announced that the fingerprint belonged to an Algerian suspect.
Two years later, the U.S. government formally apologized to Mayfield and paid him a reported $2-million settlement.
A federal judge later ruled in Mayfield's favor that provisions of the Patriot Act, allowing the FBI to use FISA to conduct "surveillance and searches of American citizens without satisfying the probable-cause requirements of the 4th Amendment," were unconstitutional. The ruling was overturned on appeal in 2009.
By then, the Obama administration had largely embraced the surveillance strategies and systems developed under President George W. Bush.
Bush gave the NSA the authority to eavesdrop on Americans communicating with foreigners abroad without first obtaining a FISA warrant, deeming the process too slow.
As a U.S. senator, Obama condemned the so-called wireless wiretapping after the New York Times made it public in 2005. But when he ran for president in 2008, Obama voted for legislation that granted retroactive legal immunity to telecommunications companies that had secretly helped the government eavesdrop.
The law also retroactively legalized other forms of surveillance, former intelligence officials say, including "bulk" monitoring that allows the government to intercept all email traffic between America and a range of suspect email addresses in, say, Pakistan.
The government's goal is "to find the kind of patterns that maybe will lead them to evidence of some kind of terrorist plot, and maybe thereafter they can then zero in on a suspect," said Joel Margolis, a regulatory consultant for Subsentio, a Colorado firm that helps telecommunications companies comply with law enforcement requests. "It's just the opposite of what we've done in our tradition of law, where you start with a suspect."
Privacy advocates say the government should acknowledge how many Americans have had their communications intercepted in recent years. But after Democrats on the House Intelligence Committee requested that information, the Obama administration responded in July that it was "not reasonably possible to identify the number."
Merrill, the Internet entrepreneur, was so disturbed by the FBI's demand for his customer's records that he became an anonymous plaintiff in a legal challenge to the Patriot Act provisions on National Security Letters.
A federal judge in New York ruled parts of the law unconstitutional in 2004 and again in 2007, calling it "the legislative equivalent of breaking and entering." Last year, Merrill won the right to identify himself as the recipient of a letter, although he is still prohibited from saying much about it. But the FBI withdrew its request for his customer's data, so higher courts didn't rule on whether the request itself was constitutional.
"I want the America back that I was taught about in school," Merrill said. "The one where there's checks and balances, and where one branch of government can't do everything on its own."