Joseph Cox
There's just a dizzying array of technologies being used by these agencies that is legitimately hard for even me who covers this to keep track of everything that's going on.
Anna Diakun
Journalist Joseph Cox is one of the founders of 404 Media, a worker-owned website covering technology. And Joseph has written a lot about government surveillance. We reached out to him because he has managed to learn about social media monitoring programs that operate largely in secret. The federal government is not exactly forthcoming about these practices, but by paying attention to how agencies spend money, Joseph has noticed some patterns.
Joseph Cox
I just want to know what technology the U.S. government is buying. It turns out a lot of that technology is at the border.
Anna Diakun
As free speech lawyers, we worry about surveillance of expressive and associational activity, a lot of which takes place on social media. As a journalist, Joseph has used various methods to try to learn what he can about the government's surveillance of social media. Here's what he has been able to piece together.
Joseph Cox
What stood out to me, beyond the normal here's the social media profiles of the person, here's their connections, all of that sort of thing, was that it would surface content that it determines to be derogatory or not and there's a little thumbs up, thumbs down.
Anna Diakun
So a government employee somewhere sees your social media posts and their job is to measure your sentiment over time by clicking that little thumbs up or thumbs down button. More specifically, it's meant to measure “derogatory sentiment,” though what that means in practice is far from clear.
Joseph Cox
People's lives could be on the line here, either in that they could be captured and deported or maybe they'll be denied entry or whatever, and it gets boiled down to a Facebook-like icon. And that could have untold ramifications for their own life and that of their family. That is what stood out to me.
Anna Diakun
This is “Speech & the Border,” at the frontiers of censorship and surveillance. I'm Anna Diakun and I'm a lawyer here at the Knight First Amendment Institute at Columbia University. I've spent the last several years working on issues related to government surveillance of social media. During that time, social media surveillance has grown more and more pervasive, with new tools and technologies at the government's disposal every year. We'll get back to that in a minute, but one thing before we do. If you haven't listened to the first episode of this season, hosted by my colleague George Wang, I'd go back and do that before continuing on with this episode. And if you have, you know that U.S. courts have given the federal government a tremendous amount of power over immigration enforcement. When it comes to social media surveillance, the government has taken that power and run with it. As I mentioned earlier, there's a lot we don't know about these practices, but in this episode, we're going to tell you what we do know.
Barack Obama
Both Al-Qaeda, and now ISIL, pose a direct threat to our people. Because in today's world, even a handful of terrorists, who place no value on human life including their own, can do a lot of damage. They use the internet to poison the minds of individuals inside our country.
Anna Diakun
This is from President Obama's State of the Union Address in 2016. That year, the Obama administration added a field on visa waiver forms for people to voluntarily disclose their social media handles. This only applied to visitors from certain countries.
Faiza Patel
And at that time already people were concerned about the impact of asking individuals to disclose their social media handles, and particularly after the election, there was a great deal of advocacy and effort to try and get the government to back off.
Anna Diakun
This is Faiza Patel, a senior director of the Brennan Center's Liberty and National Security Program. The Brennan Center and the Knight Institute have worked together on this issue for years now, focusing on the serious chilling effects that social media surveillance has on speech and association online. Despite advocates' warnings, the government didn't back off on social media surveillance. Instead, it doubled down. In 2017, shortly after then-President Trump issued his so-called travel ban-
Faiza Patel
The president issued another executive order, which was what we sort of familiarly call the Extreme Vetting Order, which basically said, “Okay, all of this stuff is going on with respect to the ban, but I want you, State Department, to really up your efforts to vet and screen people trying to come to the United States.” And based on that order, the government put in place what we call the registration requirement, which essentially asked some, I believe, 14.7 million visa applicants each year, to provide their social media handles when they applied for a visa.
Anna Diakun
This is a dramatic shift. If you're applying for a visa from abroad, your application will usually be reviewed by a consular officer. Faiza says those officers have been able to ask for social media information in the past if they have reasonable suspicion about a specific individual. But this requirement is different. It's a total dragnet. It requires nearly all applicants for immigrant and non-immigrant visas to provide the social media handles they have used on 20 different platforms at any point over the last five years. We're talking about Facebook, Instagram, YouTube, X, and more.
Faiza Patel
What this requirement does instead is it just sweeps in everybody's social media information. Because once you give the social media handle to the government, it's not just going to keep it. It's going to use it in order to find out stuff about you and it's going to be able to see everything that you post online. So it's a very broad and sweeping requirement.
Anna Diakun
This means that if you post something publicly on one of these platforms, even under a pseudonym, the government can track it down and attribute it to you. So in 2019, the Brennan Center and the Knight Institute, working with law firm Simpson Thatcher, sued the government. This was the first case I worked on after joining the Knight Institute that year, and I'm still working on it today.
Our plaintiffs are Docsociety and the International Documentary Association, two organizations based in the U.S. that work to foster cultural collaboration across borders. These organizations do incredible work supporting documentary filmmakers around the world and they rely on social media to connect with filmmakers and audiences. In the suit, Docsociety and IDA explain that the social media disclosure requirement discourages filmmakers from abroad from speaking freely online and from traveling to the U.S. to meet with possible collaborators in person. We argued that the requirement violates the rights of visa applicants to speak anonymously and associate freely online, as well as the rights of their would-be U.S. audiences to hear from them online and in person. We put our arguments before the court and waited for the judge to rule. And then an election happened.
Faiza Patel
Then back in 2021, after the Biden administration took over, it actually rescinded the executive order, the Extreme Vetting Executive Order, that was the basis for the collection of social media handles and undertook a review of social media monitoring.
Anna Diakun
We hoped that during the review they would see that the cost of free speech and association far outweigh any benefits and that they would decide to abandon the practice. And then we waited and waited and waited. Finally, after more than a year, the Biden administration informed us that it had no intention of rescinding the social media disclosure requirement in the near term, but it didn't explain why. So we at the Knight Institute filed another lawsuit, this time to get access to the report produced at the end of the Biden administration's review process. We used the Freedom of Information Act, also known as FOIA. More on that lawsuit in a minute. But we're not the only group that's used FOIA to try to figure out what the government is doing and why in the realm of social media surveillance. So does journalist Joseph Cox, though of course his approach is different.
Joseph Cox
We rely on and use the Freedom of Information Act extensively.
Anna Diakun
Actually, he has this down to a science.
Joseph Cox
We file these FOIAs every single day. And I don't just file a random FOIA and see what comes back. What I will usually do is I will dig through contracting or procurement databases. This company was paid X amount by ICE in this year or this month or whatever. Let me file a FOIA to see if I can get more information on that because these databases do not include much information on what the tech's actually being used for. And at the end of the day, that's really what we want to understand. Even if I don't know everything about it, I want to at least understand the use case or the context in which it's being deployed. Sometimes you get that, sometimes you don't.
Anna Diakun
The government typically refuses to provide much detail about its social media monitoring practices. But by coming at the issue from this angle, scrutinizing how government agencies are spending their money, Joseph has been able to paint a picture of how mass social media monitoring is actually carried out. And the short answer is outsourcing.
Joseph Cox
So the government doesn't do an extreme level of vetting on the technology. It's not getting the source code typically of the tool. It's not getting the crown jewels of this technology. It is buying it off the shelf and it is using it and it is basically trusting the vendor on whether it's going to be accurate or not.
Anna Diakun
In short, the government is the customer and the product is software or systems that find and aggregate personal information and then marry it with social media information. Joseph says this is how vendors will sell these
kinds of tools to the government.
Joseph Cox
What data brokers will call it, they'd call it a 360 view of the person, it's a marketing term where they're like, “we're getting all of the data about the person.” So you have a 360 rotation and view of what this person's all about, social media, social security, emails, all of that.
Anna Diakun
So pretend for a minute that you're the government and you're walking down the aisles of the surveillance store. On that shelf over there is Giant Oak Search Technology, or GOST, the system we mentioned at the beginning of the episode. This is the system that promises to measure sentiment over time by having an analyst assign a thumbs up or thumbs down to different posts.
Joseph Cox
Someone will come into the country, they will obviously stay in the country longer than they're legally supposed to. This sort of system will kick in, or can kick in, when it perceives that somebody is doing that. So for example, and this is slightly reading between the lines, but from the documents which are in there, it looks like, “well, this person is supposed to have left the country.” Maybe they start tweeting that they're in, I don't know, Los Angeles or San Diego or wherever. Then we will automatically flag this, it will get vetted, and maybe that will be a piece of intelligence that then can be passed to ICE or whoever.
Anna Diakun
And that's just one surveillance tool. Continue moving down the aisles of the surveillance store and you'll see another tool with an ominous name, ShadowDragon.
Joseph Cox
So ShadowDragon is an open source intelligence tool, OSINT, which refers to anything that is openly available on the web.
Anna Diakun
ShadowDragon advertises that it can “map out identities and connections of target individuals,” and “visualize their networks using data from social media sites.” There's one part of ShadowDragon's methodology that really stood out to Joseph.
Joseph Cox
They will scrape data from BabyCenter, which is like a forum for pregnant people, and it will get their usernames, their photos, the groups they're members of. And there was also I think BlackPlanet, a Black-focused social media network. Again, getting their posts and their comments. And I had never seen these two particular sites, or a bunch of others in there, ever really come up in a government surveillance context or in a tool or anything like that. And of course, it just brings up additional questions about the context of that data. Are you looking at BabyCenter users in case there is an undocumented person who's going to give birth or something? That is me speculating because we don't know, but why are you looking at BabyCenter? Why are you looking at BlackPlanet? Especially when we've had the Black Lives Matter protests. It seems even more worrying to me than Facebook or Twitter.
Anna Diakun
It's hard to imagine any national security justification for gathering this kind of information. The same goes for things like political debates or memes. These online interactions are so subjective and context dependent. It's easy to imagine misunderstandings between family members and friends, and here we're talking about strangers evaluating strangers, often with different cultural and linguistic backgrounds. Yet that information is used by officials making immigration enforcement decisions. But is the picture it's painting even accurate?
Joseph Cox
That is the one key question that we don't have the answer to. We know what technology they're using. We know the capabilities of that technology. We sometimes know how it's being used and what use case and what's being used to enforce. We don't know how accurate or basically good this tech is. We don't actually know how good the tech is for the mission that ICE or CBP sets out to do and which it's mandated to do and we don't don't know actually how accurate some of the results are. It's basically a complete unknown at this point, at least based on the documents that I've seen.
Anna Diakun
And that's something that particularly worries Faiza Patel at The Brennan Center.
Faiza Patel
You need to be able to test what you're doing, see if it's producing good results, and then, only then, to deploy it. I don't think that mentality or that rigor has been used when agencies have acquired social media monitoring software.
Anna Diakun
Consider language, for example. We are talking about immigration after all,
Faiza Patel
You're talking about people across the world who are speaking in hundreds of different languages. That's not even counting different dialects or particular phraseology or terms that are common in particular communities. It's very difficult to translate what things mean in a particular language or context and have a kind of universal understanding of what that is. Jokes, for example, are very context specific, culture specific.
Anna Diakun
And then there's the other ways people communicate on social media, the likes, the re-posts, things like that.
Faiza Patel
The interactivity of social media is not only verbal, there's a lot of nonverbal cues, and they're not always easy to understand. What does a thumbs up mean? What does a frowny face mean? What does it mean when someone retweets something? You might retweet something because you think it's interesting and not because you agree with it.
Anna Diakun
And there's the risk that the government will draw conclusions based merely on the groups you've joined or the people you've followed or friended. Faiza points to the case of Ismail Ajjawi, a Palestinian set to begin his freshman year at Harvard. According to Ajjawi, after he landed at Boston Logan International Airport, he was subjected to hours of questioning. Immigration officers searched his phone and computer, then canceled his visa and deported him. The reason? Social media posts. But the posts? They weren't his. They were written by people on his friend list. This is guilt by association taken to the extreme. A person might have hundreds or thousands of people on their friend or follow list, ranging from their wacky uncle to a total stranger who once tweeted a meme they thought was funny. And yet the government is drawing conclusions about a person's beliefs based on these posts. All these issues call into question the larger point of mass social media monitoring. Does it even deliver on the promises of enhancing national security or combating terrorism?
Faiza Patel
We have a lot of information about what the government actually thinks of its own social media vetting program. So after the San Bernardino shooting, where one of the perpetrators had come over to the United States on a fiance visa, there was a lot of interest in social media monitoring including in Congress and DHS actually launched several pilot programs.
Anna Diakun
These pilot programs used social media as part of the vetting process for a few different visa categories.
Faiza Patel
And two years later, the department's own Inspector General issued a report, so this is back in 2017, which examined these pilot programs. And what the Inspector General found was that DHS actually had never measured whether or not these social media vetting programs were effective and went so far as to say that this is really not a basis for building out a broader social media monitoring program.
Anna Diakun
Faiza also points to another document from 2016. It's a Department of Homeland Security transition memo from the Obama administration to the Trump administration.
Faiza Patel
And there again, you have folks in DHS who are explaining that they've tried to use social media in visa vetting and it hasn't proved useful. People who are actually identified as being potential threats, outside of the social media context, don't show up when you do social media vetting. It was not a helpful situation.
Anna Diakun
And again, that document is from 2016. Fast-forward to today and all of the available evidence suggests that social media surveillance is ineffective. Remember when I mentioned that we filed a lawsuit to obtain information from the Biden administration about social media surveillance? They undertook a review early on but they've refused to share those findings with the public. However, in the course of this suit, the Knight Institute did get its hands on some other documents. In one, an internal email exchange from the Office of the Director of National Intelligence, an official says straight up that the collection of social media identifiers adds “no value to the visa vetting process.” When asked about those emails, a senior Biden administration official acknowledged that the collection of social media information had “yet to help identify terrorists among visa applicants.” But officials argued that they just needed more time to figure out how to use this information.
Faiza Patel
So here you have basically an unbroken line of six years of government evaluations, starting back in the pilot programs that DHS did 2015, 2016, all the way through at least 2021, which show that again and again when people looked at social media vetting, they didn't find that it added value. And nonetheless, social media vetting continues to this day and is often actually touted as being an important part of not just the visa vetting process, but other kinds of threat detection as well.
Anna Diakun
There is still so much we don't know, but we are certain that social media surveillance creates a significant chilling effect. Visa applicants know they're being monitored and they know their online speech and associations are a factor in decisions about their immigration status. But they have no idea how they factor in or to what extent. Even worse, they don't know what information the government has collected on them or who that information will be shared with. What is clear is that while it's easy for the government to install surveillance programs like this, it's incredibly difficult to give them up once they're already in place. Government officials don't want to look weak on national security and often bear no costs for keeping questionable flawed programs in place just in case they might bear fruit at some point in the future. And we haven't even started talking about AI. Joseph Cox told us about another tool sold by a company called Fivecast.
Joseph Cox
They collect all of this data online and then they use AI to put risk scores onto material and to say sort of how the person posting the material allegedly feels, so they will use their systems to gauge, well, how much emotion was there over time?
Anna Diakun
Remember at the beginning of the year when Google enabled AI search and it started telling users to eat rocks every day.
CNBC Newscaster
The AI overview telling users that you can eat a certain amount of rocks a day for your health, it's healthy to do so. Another one was suggesting using glue to thicken your pizza sauce.
Anna Diakun
These tools are largely still in the testing phase, but nonetheless, they're being sprung on the public in a variety of different ways, whether they do the job well or not. The government is working out rules for the adoption of AI tools by federal agencies. Faiza has been a vocal advocate for greater oversight on the use of AI tools. She's written extensively about how law enforcement agencies have used AI tools to target U.S. citizens, like racial justice protesters in 2020. When Faiza spoke with my colleague Carrie DeCell she said there's simply no excuse for these tools to be used by the government largely in secret.
Faiza Patel
I really think we need radical transparency because what we have right now is basically a patchwork of information. So you can read every document that the Department of Homeland Security puts out.
Carrie DeCell
And you have, to be clear. I know.
Faiza Patel
And you still won't know exactly what they're doing or even have a good sense of what they're doing. You will know pieces of what they're doing. You might get a fragment of a mention of a company or a program in a Congressional hearing. You might get a little piece of it from a FOIA request. There might be procurement records that tell you a little piece of the story. But I don't think we have a full sense of what exactly they're doing. I think we can say with confidence that social media tools, monitoring tools, are being deployed across the government at all levels, federal, state, local, in many different ways for many different purposes. And then most of these uses have not been properly vetted for accuracy and to ensure that they're unbiased.
Carrie DeCell
And what would you call for to increase transparency to kind of step towards radical transparency. Although one can only dream on that front, I would say.
Faiza Patel
Well, we wouldn't be in our jobs if we didn't dream, Carrie. I think two things. So one is Congress really needs to insist that agencies provide actual information on the different ways in which they are using social media monitoring tools, including which companies or products they're using. I think that is sort of baseline information. In addition to that, I think Congress must require these agencies to carry out the kind of risk assessment and audits that we have seen being developed in the context of AI for which this is very relevant before agencies use new tools and certainly also looking backwards at the tools that they already have.
Anna Diakun
The mandatory disclosure of social media information on visa applications is the casting point for a vast surveillance dragnet that extends far beyond visa applicants themselves and far into the future. This dragnet is certain to chill people from engaging in protected speech and association online and to deter some foreign citizens from visiting the United States, depriving U.S. residents of opportunities to meet with and hear from people who live outside the states.
At the same time, the public knows very little about the tools the government is employing and how closely it has vetted those tools. We also don't know the government's justification for keeping these tools around. And the problem is only going to get worse, as AI-powered tools proliferate, and the government's social media surveillance practices become even more of a black box. And for what? Everything we've learned so far suggests that social media surveillance hasn't helped the government identify threats. The U.S. government is compiling huge haystacks of sensitive information about millions of people each year, but by its own internal admission, finding no needles within them. We need a better approach, one that centers individual rights and democratic values. Government agencies need to be more transparent about their use of powerful surveillance technologies. And in the meantime, we're going to keep fighting to learn more about the tools the government is using and about how it is using them. Our work continues.
Akram Shibly
Three officers put their hands on me. One put their hand around my throat, another tied up my legs, and the third reached in to my pocket and took my cell phone.
Anna Diakun
And if you think being a citizen protects you from those kinds of searches, think again.
Anna Diakun
I'm Anna Diakun, a lawyer at the Knight First Amendment Institute. Thank you for listening to this episode. My colleague, Carrie DeCell conducted all of the interviews you heard, and she and I worked together on all of the litigation mentioned in this episode.
“Speech & the Border” is co-produced by Ann Marie Awad and Kushal Dev. Our executive producer is Candace White. Our engineer is Patrice Mondragon. Fact-checking by Kushal Dev. The art for our show was created by Nash Weerasekera. Our theme music was composed by Grayton Newman, with additional music from Epidemic Sound. “Views on First” is available on Apple, Spotify, and wherever you listen to podcasts. Please subscribe and leave a review. We'd love to know what you think. To learn more about the Knight Institute, visit our website knightcolumbia.org. That's Knight with a K. And follow us on social media. Bye.