ODNI transparency report nerding: 151 million “call detail records”

Several people have blogged things trying to make sense of the disclosure this week that the USA Freedom Act system (which replaced the Patriot Act Section 215 bulk calling records program) collected 151 million call detail records in 2016, even though the intelligence court had approved two-hop collection surrounding only 42 suspects. To be sure, this number, 151 million, is small compared to the billions of records per day the old system was sucking in, but it is nevertheless surprisingly large on its face. I wanted to make two in-the-weeds points – one about the math in general, and one a response to Marcy Wheeler at Emptywheel.

I was told there would be no math

As people try to crunch the numbers of how to get to 151 million, a crucial thing to grasp is that a Freedom Act order is not merely a two-hop pen register, in which the N.S.A. gets prospective logs of all the new messages of its target and everyone in contact with him. Rather, it’s also a request for historical billing records still in the providers’ possession. So that’s potentially years of logs of phone calls (and probably SMS text messages) for each person in the suspect’s social circle, even though the government only collected those records during the calendar year of 2016. This factor will dramatically expand not just the number of calls a suspect would have, but also the number of social-link people who will contribute their own universe of second-hop records.

Another important insight is what it means that the government warns about duplication within the 151 million database: a lot of those 151 million records are redundant. For example, if the suspect, Joe the AT&T Customer, called his friend, Mary the Verizon Customer, the government would receive two records stemming from that single call – one from AT&T and one from Verizon. This problem extends to second-hop records: if Joe also called his other friend Fred the Sprint Customer, and Fred and Mary are also friends and separately called each other, the government would receive redundant records of Fred’s and Mary’s call from both Verizon and Sprint.

Another wildcard is that we don’t know is how much garbage is in the system from contacts with businesses and other entities that make a lot of phone calls to unrelated people, creating a potentially larger second-hop universe than an ordinary contact would – like if Joe called an auto body shop or a restaurant etc. which separately called or received calls from thousands of other customers over the years. Presumably the N.S.A. system is set up to invalidate the most commonly called numbers before requesting the second-hop records, lest they generate so much random noise that it would drown out the signal they are hunting for. But there must be some mid-sized entities that haven’t been added to the block list or that investigators wanted to keep for some particular reason – like, say the trunk line for the business where a suspect works. So this factor, too, could help get us to 151 million.

Response to Emptywheel

At Emptywheel, Marcy Wheeler has written an analysis of the 151 million number that has some elements I think are valuable contributions but also some that I am more skeptical about. Her introduction frames it as correcting misconceptions purportedly created in part by my New York Times article about the ODNI report. I reached out to her by email, but she wanted to have the conversation in public.

About half of her piece is devoted to showing how the math to generate 151 million call events within a year is implausible. Eventually, after hundreds of words, she reveals that this premise was a red herring because it is actually about historical records, not just prospective ones. Well, yes. My article said this was about “calling histories” involving “years” of phone records, so it created no such misconception, I hope.

Marcy also states that this is about more than just calls – it’s also about texts. I had only discussed “calls” in my article, but I think it’s likely right that SMS texts are also part of the mix since phone companies keep track of those for billing purposes and they serve the same purpose of identifying social links between people. Texts might also help get us to 151 million: a single conversation consisting of 10 SMS texts could be logged as 10 separate records, or 20 if duplicated between two carriers.

But Marcy then puts forward the idea that the 151 million message records in the ODNI report likely go beyond phone company records of calls and SMS texts and include other stuff, too, like websites visited on a cell phone’s browser and message logs from apps like WhatsApp and iMessage (both “certainly,” in her view) and Signal (“possibly” in her view). Indeed she says the latter is “necessarily true” for two reasons: because members of Congress have expressed concerns about electronic communications service providers that don’t keep records past 18 months, and because a lawmaker has said a large list of companies receive orders under the Freedom Act system.

Sometimes when Marcy speculates about things, she labels it a “wild-arsed guess,” but there is no such caveat here and she seems to be putting it forward as something her readers should treat as a fact. I am skeptical that this claim should be treated as a fact. Everything I have heard is that the Freedom Act system as it now exists, at least, is just about traditional telecom-based telephony (i.e. calls and SMS texts), echoing the predecessor Patriot Act program. I am aware of no evidence supporting the idea that the Freedom Act system has expanded to web browsing or app-based services from internet companies.

Importantly, the ODNI transparency report talks about the 151 million records coming from “telecommunications providers,” not electronic communications service providers generically speaking. Telecoms are a type of electronic communications service provider (defined here and here) that is generally understood to be phone and network companies, like AT&T, that transmit users’ signals and are regulated by the Federal Communications Commission under the Telecommunications Act of 1934. Messaging services that use the Internet but do not operate it, like WhatsApp, are a different type of electronic communications service provider and are generally not called telecoms.

I also do not see how the two pieces of purported evidence Marcy points to prove that metadata from WhatsApp-style services are nevertheless part of the 151 million records.

It is true that some members of Congress are interested in firms that do not keep their records longer than 18 months, but the context of that interest was rooted in traditional telephony: the F.C.C.’s regulation requiring phone companies to hold onto billing records for at least that time is understood to apply only to landline services, not cell phone services. Part of the debate about the Freedom Act was whether to impose a new data retention requirement on cell phone services to make sure relevant records would be there if the N.S.A. wasn’t storing its own copy.

It is also true that a lawmaker has said a sizable number of companies are receiving Freedom Act orders, but that also can be consistent with a telecom-only universe. As far as we know, only the three biggest telecoms were part of the old Patriot Act system – AT&T, Sprint, and Verizon – because the N.S.A. did not trust smaller telecoms to keep its existence a secret. Since the new system is not a secret, the government can obtain orders for all telecoms if it wants, and there are a ton – here’s a list – more than enough to make the number of those receiving Freedom Act system orders large and significant.

Does that mean the government could not ask the Foreign Intelligence Surveillance Court to interpret the words of the statute as justifying Freedom Act orders to internet messaging services? I would not rule it out as impossible. But I am aware of no evidence it has happened yet, and I don’t think it’s necessary to get to 151 million records collected in 2016.