Recent NSA Revelations

I haven't written much about the NSA scandals because there's so much written about it and I've fallen behind. Here's an attempt to catch up on some recent stuff. If you're only going to read one article, just jump to the bottom and read this one.

On July 31st, Glenn Greenwald wrote about XKeyscore: NSA tool collects 'nearly everything a user does on the internet'

"A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals, according to documents provided by whistleblower Edward Snowden."

"But training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed. XKeyscore, the documents boast, is the NSA's "widest reaching" system developing intelligence from computer networks – what the agency calls Digital Network Intelligence (DNI). One presentation claims the program covers "nearly everything a typical user does on the internet", including the content of emails, websites visited and searches, as well as their metadata."

"The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours." To solve this problem, the NSA has created a multi-tiered system that allows analysts to store "interesting" content in other databases, such as one named Pinwale which can store material for up to five years."

My sense is, they sweep up everything they can and then do searches when asked. They are legally limited in what they are allowed to collect (i.e., not from Americans) but they use the tree falling in the forest analogy. If they collect it and no one looks at it, is it really collected. So they say the information is only collected if it is ever returned in a search.

To me the problem is that they've basically constructed a time machine. They keep everything you do online for some period of time and can go back and look through it. Now the philosophical question is, do you change your behavior if you know it's being recorded? I think the answer is yes. On CBS This Morning today I saw a piece about how when businesses monitor employees performance with real metrics, the employees work harder (to avoid being fired). Lawyers sometimes prefer talking to their clients on particular topics rather than via email to avoid having a specific record or businesses do it to avoid secrets being found out in discovery.

On another point, based on how Snowden was able to get this info, it seems the NSA systems have crappy internal security. If you have some clearance you can query anything you want and system administrators have full access. It needn't and shouldn't be that way. There seem to examples where NSA analysts used the system to find out info about romantic interests. They were disciplined but a truly secure system wouldn't allow a user to do such a thing in the first place (imagine having to enter a subpoena number before doing a query).

John Naughton sums this up nicely. Edward Snowden's not the story. The fate of the internet is. "The press has lost the plot over the Snowden revelations. The fact is that the net is finished as a global network and that US firms' cloud services cannot be trusted"

So as a public service, let us summarise what Snowden has achieved thus far. Without him, we would not know how the National Security Agency (NSA) had been able to access the emails, Facebook accounts and videos of citizens across the world; or how it had secretly acquired the phone records of millions of Americans; or how, through a secret court, it has been able to bend nine US internet companies to its demands for access to their users' data.

Similarly, without Snowden, we would not be debating whether the US government should have turned surveillance into a huge, privatised business, offering data-mining contracts to private contractors such as Booz Allen Hamilton and, in the process, high-level security clearance to thousands of people who shouldn't have it. Nor would there be – finally – a serious debate between Europe (excluding the UK, which in these matters is just an overseas franchise of the US) and the United States about where the proper balance between freedom and security lies.

He goes on to ask bigger questions. Will the Internet break up into regional divisions? Given the revelations about what the US government is doing, will they continue to have as big a role as they do in Internet governance? Will foreign people and companies trust US cloud services (like Google, Facebook, Yahoo, Amazon, Apple, Microsoft, etc) with their information?

James Fallows correctly points out, Why NSA Surveillance Will Be More Damaging Than You Think. "The real threat from terrorism has never been the damage it does directly, even through attacks as horrific as those on 9/11. The more serious threat comes from the over-reaction, the collective insanity or the simple loss of perspective, that an attack evokes. Our government's ambition to do everything possible to keep us "safe" has put us at jeopardy in other ways."

So to try to improve their image the Director of the NSA spoke at the annual Black Hat security conference. That's a rare event. Also as Spencer Ackerman explains, US government declassifies court order on NSA surveillance as pressure builds.

"Clearly Congress intended for Section 215 of the Patriot Act to be a tool for investigating terrorists and spies, not for tracking the communications of all Americans. No prosecutor would seek a grand jury subpoena of this scope and no judge in a criminal or civil procedure would enforce one."

A proposed change to system sounds reasonable to me. US senators push for special privacy advocate in overhauled Fisa court

Senators Richard Blumenthal of Connecticut, Ron Wyden of Oregon and Tom Udall of New Mexico, all Democrats, want a special advocate for Americans' privacy to argue before the so-called Fisa court when the government seeks extraordinary surveillance requests. They also propose to diversify the powerful secret court ideologically and geographically.

A few days ago we found out about the secret budget of these agencies.

The Washington Post wrote, ‘Black budget’ summary details U.S. spy network’s successes, failures and objectives.

The $52.6 billion “black budget” for fiscal 2013, obtained by The Washington Post from former ­intelligence contractor Edward Snowden, maps a bureaucratic and operational landscape that has never been subject to public scrutiny. Although the government has annually released its overall level of intelligence spending since 2007, it has not divulged how it uses the money or how it performs against the goals set by the president and Congress. The 178-page budget summary for the National Intelligence Program details the successes, failures and objectives of the 16 spy agencies that make up the U.S. intelligence community, which has 107,035 employees.

Historical data on U.S. intelligence spending is largely nonexistent. Through extrapolation, experts have estimated that Cold War spending probably peaked in the late 1980s at an amount that would be the equivalent of $71 billion today. Spending in the most recent cycle surpassed that amount, based on the $52.6 billion detailed in documents obtained by The Post plus a separate $23 billion devoted to intelligence programs that more directly support the U.S. military.

They've got a big infographic. The Black Budget: Top secret U.S. intelligence funding. WonkBlog loves graphs so they broke down, America’s secret intelligence budget, in 11 (nay, 13) charts.

Via these documents we learned, U.S. spy agencies mounted 231 offensive cyber-operations in 2011. "Additionally, under an extensive effort code-named GENIE, U.S. computer specialists break into foreign networks so that they can be put under surreptitious U.S. control. Budget documents say the $652 million project has placed “covert implants,” sophisticated malware transmitted from far away, in computers, routers and firewalls on tens of thousands of machines every year, with plans to expand those numbers into the millions."

The administration’s cyber-operations sometimes involve what one budget document calls “field operations” abroad, commonly with the help of CIA operatives or clandestine military forces, “to physically place hardware implants or software modifications.”

Much more often, an implant is coded entirely in software by an NSA group called Tailored Access Operations (TAO). As its name suggests, TAO builds attack tools that are custom-fitted to their targets.

The NSA unit’s software engineers would rather tap into networks than individual computers because there are usually many devices on each network. Tailored Access Operations has software templates to break into common brands and models of “routers, switches and firewalls from multiple product vendor lines,” according to one document describing its work.

The implants that TAO creates are intended to persist through software and equipment upgrades, to copy stored data, “harvest” communications and tunnel into other connected networks. This year TAO is working on implants that “can identify select voice conversations of interest within a target network and exfiltrate select cuts,” or excerpts, according to one budget document. In some cases, a single compromised device opens the door to hundreds or thousands of others.

Sometimes an implant’s purpose is to create a back door for future access. “You pry open the window somewhere and leave it so when you come back the owner doesn’t know it’s unlocked, but you can get back in when you want to,” said one intelligence official, who was speaking generally about the topic and was not privy to the budget. The official spoke on the condition of anonymity to discuss sensitive technology.

Under U.S. cyberdoctrine, these operations are known as “exploitation,” not “attack,” but they are essential precursors both to attack and defense.

By the end of this year, GENIE is projected to control at least 85,000 implants in strategically chosen machines around the world. That is quadruple the number — 21,252 — available in 2008, according to the U.S. intelligence budget.

The NSA appears to be planning a rapid expansion of those numbers, which were limited until recently by the need for human operators to take remote control of compromised machines. Even with a staff of 1,870 people, GENIE made full use of only 8,448 of the 68,975 machines with active implants in 2011.

For GENIE’s next phase, according to an authoritative reference document, the NSA has brought online an automated system, code-named TURBINE, that is capable of managing “potentially millions of implants” for intelligence gathering “and active attack.”

The Washington Post followed up with The NSA has its own team of elite hackers.

But for all the reported secrecy surrounding TAO’s activities, a quick search of networking site LinkedIn shows a number of current and former intelligence community employees talking pretty openly about the exploits.

For instance, Brendan Conlon, whose page lists him as a former Deputy Chief of Integrated Cyber Operations for the NSA and former Chief of TAO in Hawaii, says that he led “a large group of joint service NSA civilians and contractors in executing Computer Network Exploitation (CNE) operations against target networks.” Barbara Hunt, who is listed as a former Director of Capabilities at TAO in Fort Meade, similarly claims she was “responsible for end-to-end development and capability delivery to build a versatile computer network exploitation effort.”

Dean Schyvincht, who claims to currently be a TAO Senior Computer Network Operator in Texas, might reveal the most about the scope of TAO activities. He says the 14 personnel under his management have completed “over 54,000 Global Network Exploitation (GNE) operations in support of national intelligence agency requirements.” Just imagine how productive the team in Fort Meade, rumored to have about 600 people, must be.

I wish Neal Stephenson would write a novel about these people.

Via one of the budget line items we've learned and Wired explores, Feds plow resources into “groundbreaking” crypto-cracking program.

The document goes on to reveal that something called the Consolidated Cryptologic Program has received more than $10 billion annually for the past four years, and it employs about 35,000 people. It also shows that 23 percent of this year's program funding supported collection and operations, 15 percent went to processing and exploitation, and 14 percent funded analysis and production. In addition to supporting cracking, the $10 billion presumably includes funding for so-called comsec, short for communications security, which is designed to prevent adversaries from accessing communications in an intelligible form.

One of the items computer security people are speculating about is Groundbreaking NSA Crypto-Cracking. The documents include a summary by Director of National Intelligence James Clapper.

“Also,” Clapper writes in a line marked “top secret,” “we are investing in groundbreaking cryptanalytic capabilities to defeat adversarial cryptography and exploit internet traffic.” The Post’s article doesn’t detail the “groundbreaking cryptanalytic capabilities” Clapper mentions, and there’s no elaboration in the portion of the document published by the paper. But the document shows that 21 percent of the intelligence budget — around $11 billion — is dedicated to the Consolidated Cryptologic Program that staffs 35,000 employees in the NSA and the armed forces.

Yesterday Bruce Schneier speculated, What Exactly Are the NSA's 'Groundbreaking Cryptanalytic Capabilities'?.

Honestly, I’m skeptical. Whatever the NSA has up its top-secret sleeves, the mathematics of cryptography will still be the most secure part of any encryption system. I worry a lot more about poorly designed cryptographic products, software bugs, bad passwords, companies that collaborate with the NSA to leak all or part of the keys, and insecure computers and networks. Those are where the real vulnerabilities are, and where the NSA spends the bulk of its efforts.

He speculates they might have some attacks against symmetric cryptography the way they had differential cryptanalysis before academia did but isn't too worried about that.

So while the NSA certainly has symmetric cryptanalysis capabilities that we in the academic world do not, converting that into practical attacks on the sorts of data it is likely to encounter seems so impossible as to be fanciful.

He thinks it's more likely they have some attacks against public-key crypto.

Breakthroughs in factoring have occurred regularly over the past several decades, allowing us to break ever-larger public keys. Much of the public-key cryptography we use today involves elliptic curves, something that is even more ripe for mathematical breakthroughs. It is not unreasonable to assume that the NSA has some techniques in this area that we in the academic world do not. Certainly the fact that the NSA is pushing elliptic-curve cryptography is some indication that it can break them more easily. If we think that’s the case, the fix is easy: increase the key lengths.

Today we learned that the NSA has apparently hacked something that lets them decrypt internet traffic. I'm guessing it's more likely they broke SSL/TLS rather than the underlying crypto. The story was published by both the NY Times, N.S.A. Foils Much Internet Encryption and The Guardian, US and UK spy agencies defeat privacy and security on the internet. The Guardian article includes some slides and some more tech talk but the NYT article puts things in perspective for a layman a little better. They summarize:

The N.S.A. hacked into target computers to snare messages before they were encrypted. In some cases, companies say they were coerced by the government into handing over their master encryption keys or building in a back door. And the agency used its influence as the world’s most experienced code maker to covertly introduce weaknesses into the encryption standards followed by hardware and software developers around the world.

“For the past decade, N.S.A. has led an aggressive, multipronged effort to break widely used Internet encryption technologies,” said a 2010 memo describing a briefing about N.S.A. accomplishments for employees of its British counterpart, Government Communications Headquarters, or GCHQ. “Cryptanalytic capabilities are now coming online. Vast amounts of encrypted Internet data which have up till now been discarded are now exploitable.”

The files show that the agency is still stymied by some encryption, as Mr. Snowden suggested in a question-and-answer session on The Guardian’s Web site in June. “Properly implemented strong crypto systems are one of the few things that you can rely on,” he said, though cautioning that the N.S.A. often bypasses the encryption altogether by targeting the computers at one end or the other and grabbing text before it is encrypted or after it is decrypted.

N.S.A. documents show that the agency maintains an internal database of encryption keys for specific commercial products, called a Key Provisioning Service, which can automatically decode many messages. If the necessary key is not in the collection, a request goes to the separate Key Recovery Service, which tries to obtain it. How keys are acquired is shrouded in secrecy, but independent cryptographers say many are probably collected by hacking into companies’ computer servers, where they are stored. To keep such methods secret, the N.S.A. shares decrypted messages with other agencies only if the keys could have been acquired through legal means. “Approval to release to non-Sigint agencies,” a GCHQ document says, “will depend on there being a proven non-Sigint method of acquiring keys.” Simultaneously, the N.S.A. has been deliberately weakening the international encryption standards adopted by developers. One goal in the agency’s 2013 budget request was to “influence policies, standards and specifications for commercial public key technologies,” the most common encryption method.Cryptographers have long suspected that the agency planted vulnerabilities in a standard adopted in 2006 by the National Institute of Standards and Technology and later by the International Organization for Standardization, which has 163 countries as members.

They also have two nice additions. They have the budget request and briefing sheet documents with some highlights and translations into regular english. Documents Reveal N.S.A. Campaign Against Encryption. They have an infographic trying to explain to the average user where the security protocols are used in the applications they run, Unlocking Private Communications.

So while some encrypted mail companies have shutdown, Developers Scramble to Build NSA-Proof Email.

Meanwhile the Patriot Act author says NSA’s bulk data collection is “unbounded in its scope”.

In one of the most prominent legal challenges to government intelligence gathering since the Edward Snowden disclosures, the American Civil Liberties Union (ACLU) has filed a lawsuit against four top Obama Administration officials. The case, known as ACLU v. Clapper, asks a federal judge to declare the entire metadata sharing program unlawful, halt it, and purge all related records.

On Thursday, Rep. James Sensenbrenner (R-WI), with representation from the Electronic Frontier Foundation (EFF), filed an amicus brief with the court. He noted that the vast data handover is not at all what Congress intended to happen. And Sensenbrenner should know, too, because he authored the Patriot Act in October 2001 and supported its subsequent reauthorizations.

Bruce Schneier outlines The Only Way to Restore Trust in the NSA.

The NSA has repeatedly lied about the extent of its spying program. James R. Clapper, the director of national intelligence, has lied about it to Congress. Top-secret documents provided by Edward Snowden, and reported on by the Guardian and other newspapers, repeatedly show that the NSA's surveillance systems are monitoring the communications of American citizens. The DEA has used this information to apprehend drug smugglers, then lied about it in court. The IRS has used this information to find tax cheats, then lied about it. It's even been used to arrest a copyright violator. It seems that every time there is an allegation against the NSA, no matter how outlandish, it turns out to be true.

All of this denying and lying results in us not trusting anything the NSA says, anything the president says about the NSA, or anything companies say about their involvement with the NSA. We know secrecy corrupts, and we see that corruption. There's simply no credibility, and -- the real problem -- no way for us to verify anything these people might say.

It's time to start cleaning up this mess. We need a special prosecutor, one not tied to the military, the corporations complicit in these programs, or the current political leadership, whether Democrat or Republican. This prosecutor needs free rein to go through the NSA's files and discover the full extent of what the agency is doing, as well as enough technical staff who have the capability to understand it. He needs the power to subpoena government officials and take their sworn testimony. He needs the ability to bring criminal indictments where appropriate. And, of course, he needs the requisite security clearance to see it all. We also need something like South Africa’s Truth and Reconciliation Commission, where both government and corporate employees can come forward and tell their stories about NSA eavesdropping without fear of reprisal.

Trust is essential for society to function. Without it, conspiracy theories naturally take hold. Even worse, without it we fail as a country and as a culture. It's time to reinstitute the ideals of democracy: The government works for the people, open government is the best way to protect against government abuse, and a government keeping secrets from is people is a rare exception, not the norm.

Schneier also wrote The US government has betrayed the internet. We need to take it back.

By subverting the internet at every level to make it a vast, multi-layered and robust surveillance platform, the NSA has undermined a fundamental social contract. The companies that build and manage our internet infrastructure, the companies that create and sell us our hardware and software, or the companies that host our data: we can no longer trust them to be ethical internet stewards. This is not the internet the world needs, or the internet its creators envisioned. We need to take it back. And by we, I mean the engineering community.

One, we should expose. If you do not have a security clearance, and if you have not received a National Security Letter, you are not bound by a federal confidentially requirements or a gag order. If you have been contacted by the NSA to subvert a product or protocol, you need to come forward with your story. Your employer obligations don't cover illegal or unethical activity. If you work with classified data and are truly brave, expose what you know. We need whistleblowers.

We need to know how exactly how the NSA and other agencies are subverting routers, switches, the internet backbone, encryption technologies and cloud systems. I already have five stories from people like you, and I've just started collecting. I want 50. There's safety in numbers, and this form of civil disobedience is the moral thing to do.

Two, we can design. We need to figure out how to re-engineer the internet to prevent this kind of wholesale spying. We need new techniques to prevent communications intermediaries from leaking private information.

Three, we can influence governance. I have resisted saying this up to now, and I am saddened to say it, but the US has proved to be an unethical steward of the internet. The UK is no better. The NSA's actions are legitimizing the internet abuses by China, Russia, Iran and others. We need to figure out new means of internet governance, ones that makes it harder for powerful tech countries to monitor everything. For example, we need to demand transparency, oversight, and accountability from our governments and corporations.

To the engineers, I say this: we built the internet, and some of us have helped to subvert it. Now, those of us who love liberty have to fix it.

And of course, after reading all of these articles and writing the above, the best article is the last one I got to. Of course it's also by Bruce Schneier, who apparently is now working with Greenwald. While it's titled How to remain secure against NSA surveillance it also provides the best description I've seen of what we know about the NSA's capabilities. I won't even try to summarize or pull quotes from it.