Introduction

Our debates about ‘AI’ grow out of 1990s science fiction. Back then, Vinge (1993) wrote essays and novels urging us to face up to the oncoming “Singularity”: a moment of rapid change that would fundamentally transform the human condition. On that day, AI would rapidly evolve from merely human-level intelligence, what some now call ‘artificial general intelligence’ (AGI), into something super-intelligent with its own interests and goals. Humanity would then either be casually eliminated by out-of-control machines, or humans would become as gods, with super-human servitors at our command.

However excellent the resulting science fiction (especially Vinge (1992)), it is a scandal that this dream of the ’90s is still alive and shaping debate. For complicated social and cultural reasons (Becker, 2025), many of the progenitors and funders of modern generative AI bought heavily into this mythology, and built their business strategies and innovation around it (Hao, 2025). As Singularity thinking has leaked out of containment, it has fueled speculation about further vast social, political, and economic transformations. Will AI supercharge authoritarian mind-control (Harari, 2018) or remake democracy (Gudiño et al., 2024)? Will neoliberalism become a feral, self-aware, and all-devouring “machinic” system (Land, 2011)? Again, there are excellent science fiction treatments of these and other possibilities (Banks, 1987; Reynolds, 2000; Stross, 2005; Chiang, 2010; McAuley, 2010; Valente, 2011; Emrys, 2022), but novelists are (typically) more careful about incorporating complexities, and few set themselves up as prophets, or even prognosticators.

Authors of speculative non-fiction about AGI are less inhibited, offering sweeping visions of how information technology will completely transform society, economy, politics, or all three. They treat AGI less as a technology than as Andreessen (2023) says, “our alchemy, our Philosopher’s Stone,” an alkahest that will dissolve the dross and cruft of human institutions, leaving only pure, undiluted progress. Prognosticators regularly describe the social institutions that tremble on the brink of transformation in ways that are only slightly less stylized, claiming for example that AI might transport us into Condorcet’s utopia of a new Age of Reason, this time, happily without tumbrils of condemned prisoners waiting on their appointments with the guillotine (Hall, 2026). The result is a genre that we believe is more liable to confuse smart people and lead them astray than to usefully guide public action.

All of these aspirations and arguments are, as we said, rooted in myths which are (at least) twenty years older than the technology which now seems to incarnate them, large language models (LLMs). It is because LLMs moved in the space of a few years from being a technical improvement in machine translation (Vaswani et al., 2017) to being proclaimed as the royal road to AGI that these debates really matter. LLMs are remarkably good generative statistical models of human language (including human-written computer code). This allows them to process language in ways that resemble human discourse and to be jury-rigged to create texts that loosely approximate human reasoning. This is a new material reality, a new force in the world, but one whose actual implications are obscured by the mythic garb it is swaddled in.

If we are unimpressed by stories about paperclip maximizers remaking the galaxy, omniscient bureaucracies of terror or wonder, markets that suddenly become self-aware, and the like, it is not because we think they are too weird. Rather, they are not nearly weird enough, and miss how much of the weirdness is already here. The possible futures we face are much messier and more varied than stark visions of omnipotent AGI, just as our immediate past was. They will be shaped by the collision between imperfect and highly complex technologies and imperfect and highly complex human social systems (Matias, 2023; Nelson, forthcoming). It is impossible to predict the consequences, but we can map, study, and think about them as they are happening.

From our perspective, the Singularity began two centuries ago with the Industrial Revolution (Shalizi, 2010), and it has been much messier and more variegated than anyone could have known. The modern social sciences are the offspring of the enormous shocks that they entailed in the past (Tilly, 1984; Nelson, forthcoming). They now need to work together with computer science and other related disciplines (science and technology studies; communications) to map what is best grasped as another stage in the Long Industrial Revolution. AI may turn out to be very important, but in quite different ways than our inherited myths suggest.

We build on ongoing collaborative work (Farrell et al., 2025) with Alison Gopnik and James Evans which argues that it is a category error to think of “large models” as self-motivated agents in the making. Instead, they are better understood as “cultural” (Yiu, Kosoy and Gopnik, 2024) and “social” technologies, resembling libraries and languages on the one hand and markets and bureaucracies on the other. Here we focus on how to study these technologies’ consequences for human society, emphasizing the social rather than the cultural aspects. We particularly emphasize how AI is a social technology, a systematic means of reorganizing social relationships among human beings (Therborn, 1978). Earlier social technologies include not just other information technologies, but institutions of governance such as bureaucracies, markets, and even democracy (Farrell, 2025). We will focus on LLMs over other AI systems. This downplays some important aspects of modern AI (e.g., its use in straightforward scientific problems such as protein folding) but helps highlight connections to other social technologies.

Briefly: LLMs create social relations between their users and the authors of the text in their training corpora. With the right access to the model and the corpus, one can trace the connections from system output back to individual source texts and their authors (Grosse et al., 2023). These social relations are mechanically mediated, giving users the illusion that they are interacting with just the machine and not an assemblage of people. But mediated social relationships and their illusions are a common fact of modern life. The social relations created by LLMs in turn cut across, and interact with, other social relations, including those shaped by other social technologies.

Our goal here is to clear a common space where the social sciences and computer science and engineering can discuss the social consequences of AI. We draw heavily on the ideas of Simon (1996), who saw AI, political science, administration, economics, computer science, and cognitive psychology as so many branches of the “sciences of the artificial,” studying how human beings create "artifacts" that model, and act on, their environment. From this perspective, AI models are another means of “complex information processing” (Newell and Simon, 1956). As Simon emphasizes, such systems encompass both information technologies, as studied and built by computer scientists and engineers, and social information systems such as markets, bureaucracy, and, although Simon himself does not stress this, democracy (Lindblom, 1965). All such systems process information by reducing complex realities into more tractable ‘coarse-grainings’ or abstractions that (hopefully) capture important features of the data. Producing coarse-grainings is not all that large-scale social institutions do, but it is quite important. Economic, administrative, and political coordination simply cannot work at scale if complex social relationships are not compressed into visible, tractable representations.

This then opens a different perspective on the collision between new technologies such as AI and existing social systems. As DeDeo (2017) suggests, we urgently need to discover how the new coarse-grainings of AI interact with the existing abstractions through which humans simplify an inherently complex world to make it tractable. Both AI and older social technologies are, among other things, forms of information processing. We should investigate how the former are variously reinforcing, reshaping, or replacing the latter.

In the rest of this paper, we briefly situate AI in the historic context of the Long Industrial Revolution. Next, we explore the relationship between social technologies and coarse-grainings or abstractions, emphasizing their lossiness and consequences for power relations between different social groups. That allows us both to describe the apparent strengths and limits of actually-existing AI and start applying Simon’s ideas to the intersection between AI and bureaucracy. Bureaucracy is a crucial, ancient social technology which played a central role in Simon’s work. Its relationship to AI is urgently topical: claims about AGI were seemingly one influence on the Trump administration’s sweeping cutbacks to the administrative state. We contrast these ideas with our own, to draw out the many important questions and problems that are elided or ignored by AGI-fueled speculation. Rather than expecting AGI to resolve perennial problems of human social organization, we should treat AI as a new social technology which will alleviate some problems, exacerbate others, and create new ones, just as other social technologies have done in the past. That, in turn, suggests the urgency of cooperation between social and computer scientists to figure out its social consequences, and broader social and political coordination too, of the kind that happened in previous stages of the Long Industrial Revolution.

Coarse-Grainings and the Long Industrial Revolution

We begin from a different viewpoint on the relationship between AI and society than much existing commentary. Scholars of science and technology studies (STS) are often more interested in how scientific and technological systems reflect broader social and political power relations (or develop their own) than in providing detailed social science microfoundations for their arguments. In contrast, the ‘rationalism’ that has dominated internal debates over AI is microfoundations all the way up. It starts from the assumption that the relationship between human beings and AI agents can be understood through the micro-level lens of strategic competition among rational Bayesian agents, and has only recently begun to think systematically about how collective phenomena might emerge as these systems scale (Hammond et al., 2025).

Herbert Simon’s intellectual project differed from both. His work provides microfoundations for an account of larger social institutions, which is explicitly grounded in the ‘bounds’ to individual human rationality (Simon, 1957). Simon suggests that large scale social technologies emerge from the need of limited humans to build collective arrangements that allow them to map and manage a complex world. Simon’s consistent theme is the mismatch between the complex environments that human beings inhabit and remake, and the limited information processing capacity they have to understand it. Here, Simon’s view partly converges with Dewey’s understanding of democracy (Farrell and Han, 2025). Neo-classical economics tends both to smooth away the complexities of the environment, and to assume that individual humans have unlimited computational power to model it and find optimal solutions to their problems. These assumptions are both mathematically convenient and highly unrealistic. From Simon’s perspective, humans must usually satisfice rather than optimize—settling for ‘good enough’ solutions rather than the best possible. Simple mental heuristics can help discover such solutions. So too can social institutions that channel the production and gathering of knowledge across many individually limited minds, directing attention and coordinating complex tasks. Of course, institutions may generate their own unexpected complexities, which in turn have to be managed.

Simon’s arguments can be reframed in terms of a more recent literature in complexity science on ‘coarse-grainings.’ Crudely speaking, a coarse-graining is a stripped-down representation of some complex phenomenon that seeks to capture its key aspects and dynamics. They are ubiquitous because no scientific model, organism, or artifact can actually grasp the full detail of its environment. Rather than the crawling molecular chaos of physical reality, they always deal with abstractions, compressed and selective summaries that ignore most details. In this sense, coarse-grainings include not only sophisticated mathematical models, statistical approximations of economies and turbulent weather systems, and “blurry JPEGs” of the World Wide Web (Chiang, 2023) for example, but the individual and collective representations that many social animals, including human beings, use to keep track of social structures and relationships. Macaques, for example, maintain a rough consensus over power relations within the troop through the exchange of subordination signals, which can reasonably be understood as a coarse-graining (Flack, 2017). This consensus roughly maps expectations over which macaque is capable of beating which others in a fight, allowing “individuals to make predictions without requiring they indefinitely store all of the details of their interactions,” and consequently guiding their decisions to fight or submit. Monk parakeets have cruder representations of their relationships with conspecifics (DeDeo, 2017). Humans, in contrast, can form more sophisticated coarse-grainings, even without larger institutions, thanks both to specialized mental modules (Boyer, 2018) and social conventions such as gossip (Origgi, 2017), which allow us to keep track, for example, of shifting coalitional relationships.

Yet such small-bore representations are hopelessly inadequate for modern human societies, which require impersonal social and informational technologies that can summarize social relations at very large scale. Rather than tracking a few individuals in a close-knit hunter-gatherer community where everyone knows everyone well, or even a village or town, we need to manage interactions that may involve millions—even billions—of people at once. Building and improving the means to do this has involved the development of institutions such as markets, bureaucracies, and even democracy that can handle relatively impersonal relationships at scale, using coarse-grainings that make these relationships comprehensible.

To understand how AI is reshaping society then, we can start by treating AI as a novel social technology, relying on coarse-grainings like all the others, and investigating how it interacts with existing social technologies. This integrates arguments over AI with older debates over the Long Industrial Revolution. Economic historians have documented how the transformation of human capacities to produce at scale over the last couple of centuries was facilitated by ‘general purpose technologies’ such as steam power and electricity. While many scholars have asked whether AI, too, is a general-purpose technology, they have paid less attention to the institutional aspects of the Industrial Revolution; most prominently the profound transformations in human capacity to organize economic, social, and political life, which enabled new technologies of production.

As STS scholars have consistently emphasized (Yates (1993) is particularly strong on this), bureaucratic innovations enabled the creation of new organizational structures that allowed for massive coordination. Classical liberal economists like Hayek have explored how large scale markets can compensate for the “computational limits of human beings,” Simon (1996, 35) allowing actors to make economic decisions on the basis of limited local information. Social scientists like Anderson (1991) and Gellner (1983) have debated how “print-capitalism,” the articulation of the nation state, and new forms of schooling and information dissemination made large scale publics (crudely) legible to themselves as well as their rulers, potentially stabilizing both democracy and responsive autocracy.

All these systems generate, and rely on, coarse-grainings. Bureaucratic standards and statistics summarize complex social realities and make them legible, for better and for worse (Scott, 1998). The price mechanism radically simplifies the intangible aspects of economic production, allowing market actors to devote their attention to buying and selling (Hayek, 1945). Opinion surveys, censuses, and other means simplify public opinion and make it legible to ordinary people and the political actors empowered to act on their behalf (Perrin and McFarland, 2011), and were relied on even by authoritarian states to map what their subjects wanted and thought (Dimitrov, 2023).

This vast increase in the social capacity for complex information processing has enabled collective cognition and problem solving at a historically unprecedented scale. Social technologies like markets, bureaucracy, and democracy allow human beings to become what economic historian Brad DeLong (2026) calls an “anthology intelligence,” capable of deploying accumulated cultural knowledge in a coordinated way towards large-scale ends. That is their positive aspect. In their negative, these systems regularly appear monstrous to those who find themselves at the wrong end of the power relations they create. Markets, bureaucracies, and even democracies have furthermore devoured older and more intimate forms of social organization, replacing them with vast systems that are regularly indifferent, and sometimes inimical, to the particular fates and desires of individuals and groups.

A social technology perspective then emphasizes just how long established the terms and language of today’s AI debates are. The older guiding systems of modernity—just like currently existing AI—lack anything that can reasonably be described as intentionality (Shalizi, 2010). However, humans have regularly treated bureaucracies, nations, and even markets (Spufford, 2017) as if they were individual intelligent agents, whether they had benign or hostile intent and goals. Current descriptions of AI regularly steal metaphors and ideas from these past debates, whether they draw on racist-conservative condemnations of modernity and democracy (Lovecraft, 2005; for a rejoinder, see Bear, 2013), humanist critiques (Jarrell, 1941), or anarchistic eulogies to past ways of life (Scott, 1998).

More importantly, this perspective allows us to begin comparing the actual workings of older social technologies with the newer ones, and to start mapping what happens as the two interpenetrate. AI is not the apotheosis of the robot gods nor yet of their human masters. It is a new machinery of complex information processing, perhaps even comparable to markets, bureaucracies, and democracy. Its one weird trick is to take enormous bodies of digitized information, whether social-economic, textual, visual, or otherwise, and generate abstractions that look to capture their leading statistical characteristics.

This is true across the various forms of AI that are deployed today. When social media platforms guess which post to serve or movie to recommend, they match particularized ‘embeddings’—coarse-grainings of information about the particular users and the universe of content—to arrive at their predictions. The LLMs that we emphasize are no more than coarse-grainings of the vast corpora of textual information that they have been trained on, post-processed to seem more natural in their interactions with humans and carry out more complex tasks. They are also no less. It is astonishing that we now have manipulable representations of entire bodies of human culture which can be set to work via an ordinary language interface to produce new outputs. These technologies are a new stage in the trajectory of institutional and organizational development that has run through modernity and the Long Industrial Revolution, giving new ways of managing complexity while creating their own complexities too.

Models, Lossiness and Power

Obviously, AI is not just one of the social technologies of the past. We emphasize the need for better mappings of what happens when it collides, and meshes, with its predecessors. Such mapping requires some understanding both of how the newer technology works and its consequences for social organization.

Some characteristics of actually-existing AIs arise from the fact that they are not the orderly systems of symbols, rules, and heuristics which pioneers like Simon anticipated building. Rather, they are families of messy statistical models, trained on large data sets. Training a statistical model for good on-average performance implies trading worse performance in rare situations for better performance in common situations. Such trade-offs have unfortunate implications for the capacity of all such models to handle situations raised by small groups of people, by those less well-represented in the training corpora, or indeed by anything genuinely novel. These models may not even give any signal that they are operating in such regimes (although there is a research program on uncertainty quantification in LLMs that aspires to catch this). 1 1. For a not-too-obsolete survey of this rapidly-advancing area, see Liu et al. 2025. We crudely sketch one line of work, to give the flavor. The statistical technique of “conformal prediction” (Vovk et al., 2005) replaces definite or “point” predictions with the whole range of possible predictions that “conform” to the patterns established by previous data points, i.e., all predictions which would not be low-probability outliers. The breadth of “conforming” predictions then indicates the model’s uncertainty, from the near-certainty of a narrow range around a particular point to the “anything goes” of very wide intervals. Applications of these ideas to LLMs (e.g., Lin et al. 2024; Quach et al. 2024; Mohri and Hashimoto, 2024; Cherian et al. 2024), while sometimes motivated by removing hallucinations and improving “factuality”, are really means of gauging the uncertainty of the model’s output in different situations.

LLMs, and most other modern AI systems, are in fact a particular kind of statistical model: they use “deep neural networks” (LeCun et al., 2015)—that is, multi-stage, iterative function approximation—to classify, to predict, and to generate. Training and running deep neural networks is a massive exercise in matrix algebra, calling for specialized hardware. Closer to our theme, neural networks do not simply produce coarse-grainings as outputs but employ them internally. The ‘transformer’ architecture that underlies LLMs illustrates the difficulties. Like other deep neural networks, a transformer involves layers of units (‘neurons’) which form increasingly complex representations of particular aspects of the data fed through them from beginning to end. Information about mis-predictions is then propagated back through the layers to reduce errors and move the predictions close to some rough optimum. After much expense (for the genuinely large models), the transformer produces a self-standing model of the statistical patterns in sequences of text-tokens that can, after fine-tuning and tweaking, be deployed in many ways.

While there is no mystery to the math or its implementation in silico, many details of how LLMs work remain opaque. The nature of the coarse-grainings that the transformer employs internally to capture the statistical relations is often obscure. Again, there is a cottage industry devoted to ‘interpretability,’ aiming to map these relationships, but it faces stark challenges. Some internal coarse-grainings may be orthogonal to any human concept. Others may be possible to identify broadly (e.g., this unit, or pattern of units, may be associated with this nameable aspect of the data), but without understanding exactly what is going on. (Whether any other model architecture that worked as well would be equally incomprehensible is a question for future research.) As a general matter, getting these models to work well resembles the mysteries of “alchemy” more than reproducible scientific technique (Rahimi and Recht, 2017).

Finally, there are important differences in the feedback relations between various forms of statistical-learning-based AI and the social, economic, and political phenomena they look to model. Frontier LLMs are extremely expensive to train, and so are updated only at lengthy intervals. The models used for advertising and social media matching are much cheaper to estimate, and so can be changed much more quickly. Such differences affect, for example, the time scales and hence the feedback relationships between these various coarse-grainings and the social systems that they seek to represent (Flack, 2017). Those feedback relationships may of course also respond to other changes, including technical tweaks (e.g., hidden prompts to LLMs can be quickly updated) and shifts in the underlying political economy. Broader technological changes may of course have consequences too, as new tools are developed and older tools improved. For example, a near-future in which small, cheap language models can do much of the work of big ones will encourage a less centralized political economy than if large models retain a decisive edge.

Each of these aspects of modern AI calls for its own research agenda. Engaging with the particulars of the technologies, rather than stylized accounts, makes it hard to even think straight about their immediate consequences, let alone offer confident predictions about the future. Hard or not, we do need to think clearly, since these coarse-grainings and the feedback dynamics around them are reshaping society. As DeDeo (2017, 8) wrote presciently in 2017:

The recent success of deep learning is in part due to its ability to adapt, at the same time, its method of coarse-graining and its theory of the logic of those coarse-grained variables. Once we realize that the machine-aided predictors of a system are also participants, it is natural to ask how their use of that knowledge, accurate or not, back-reacts on the society itself…We understand very little about how the introduction of these prediction algorithms, on a large scale, will lead to novel feedbacks that affect our political and social worlds; it remains an understudied and entirely open topic.

Nearly a decade later, the topic is still open. So where do we even start? We suggest emphasizing two important dimensions of comparison and interaction: lossiness and power. Coarse-grainings are lossy by definition, raising the question of exactly which information gets discarded, and which is retained. Coarse-grainings also regularly get embroiled in power relations, because abstractions can create winners and losers in many social conflicts. The two often affect each other: different simplifications will advantage different groups.

Coarse-grainings necessarily discard information to create manipulable but lossy abstractions. As Maxim Raginsky (2025) puts it, “abstraction hides a great deal of complexity from view, and this is both its main virtue and its primary peril.” The protocols that allow the internet to function at scale hide a great deal of heterogeneity, exposing it to unexpected perturbations and failure modes. Much the same is true of the simplifying statistics through which central bankers perceive the economy (Davies, 2025), reducing down the complexity of vast economic systems into a small number of target variables such as inflation that can be monitored for signs of instability, and the categories (such as census classifications) through which bureaucrats see the societies that they seek to order (Scott, 1998). All ignore some aspects of the system in order to focus attention on others. Mathematical studies of coarse-graining show that the repressed aspects of the process at best return as statistical noise (Chorin et al., 2000; Crutchfield and Feldman, 2003), if not as systematic errors. AI coarse-grainings will have different attentional trade-offs than previous social technologies, but they will have trade-offs.

Such blind spots are an inevitable feature of all abstractions, and are even plausibly useful (Naidu, 2020). Equally, social technologies do not just create blind spots but feedback effects between lossy coarse-grainings and the dynamics of the process that they seek to model and perhaps govern. The logic that Scott (1998) identifies, in which bureaucratic simplifications may reshape social organization in their image or create pushback (Scott, 1985), broadly characterizes the simplified representations of “high tech modernism” too (Farrell and Fourcade, 2023). Instead of census categories, platform companies use machine-learning generated ‘embeddings’ that reflect their best guesses about which videos we might want to see next, perhaps gradually reshaping our self-understandings (Fourcade and Healy 2025).

Since socially-relevant coarse-grainings regularly create winners and losers (Jacobs and Wallach, 2021; Jacobs, 2021), they can themselves become the topic of vigorous contention. Social technologies such as bureaucratic standards and market categories relieve the burdens of unmanageable complexity by creating summaries or otherwise focusing attention on some “aspects of the situation” (Simon, 1997, 101) rather than others, thereby affecting who gets what. Cronon (1991) provides a nice example of how coarse-graining led to struggles over actual coarse grains in nineteenth century Chicago. Creating national grain markets required defining and assessing different broad categories of grain, so that buyers could distinguish good grain from mediocre or bad without having to inspect it themselves. This facilitated trade; “all honest members benefited from knowing exactly what they were buying and selling,” (119) but the crude scheme of gradation advantaged the elevator owners who bought grain, allowing them to “mix across grades,” at the expense of the farmers they bought from (134), for example, combining just enough high quality grain with lower quality product that the final mix qualified for a more lucrative grade. These crude categories generated considerable unrest among farmers, who felt that they were being stolen from but had difficulty mobilizing against a technical-seeming system that was rigged in ways that were difficult to explain.

We can begin to compare various forms of AI to existing social technologies (and to each other) by examining the specific ways in which they are lossy and affect power relations. This also helps us to begin to understand how new and old social technologies will affect each other as they inter-mesh. True understanding will require real research efforts across disciplines that talk to each other much less than they ought, such as computer science, political science, political theory, public administration, sociology, law, communications, and science and technology studies. In the interim, we can at least start talking about what such efforts could study.

AI and Bureaucracy

One urgent topic is the relationship between AI and bureaucracy, where there is already debate—some of it resting on the myths with which we began.2 2. But only some. There is a broader emerging literature on AI and bureaucracy (e.g. Caplan and boyd 2018, Fourcade and Gordon 2020, Burrell and Fourcade 2021, Cuéllar and Huq 2022) which we don’t try to summarize, instead concentrating on a narrower line of argument. Although bureaucratic hierarchy is widely disparaged for its inefficiencies, it is an essential underpinning of large-scale human civilization. Hostility to bureaucracy primes audiences for stories of how preternaturally effective AI will soon scour away the imperfections of human administration, replacing them with algorithms. Such ideas reflect the disdain that some Silicon Valley elites have for human judgment and expertise and have provided real intellectual support for the Trump administration’s program to hack away the roots of the administrative state. They are also bad starting points, even apart from their political uses. Instead of myth-making about what a stylized future technology might do to stereotyped bureaucracies, we should ask how new and ineluctably messy social technologies such as AI combine lossiness and power relations in different ways than older and ineluctably messy social technologies such as bureaucracy, and what happens when the two technologies become entangled.

AGI-centric accounts depict bureaucracy as centralized coordination, the key problem being how lower layers fail to implement the priorities of the top. Leaders’ edicts get distorted or lost as they filter through the layers of administration. AGI then figures as a general solution, replacing imperfect human bureaucrats with efficient AI agents or interfaces that will do just what they are told to do. Such accounts suggest AGI will replace bureaucratic distortion with (relatively) lossless and efficient algorithmic decision-making and implementation. That will empower those at the top, whose decisions will finally be executed just as they want them to be.

These stories make it easy to see the attractions of AI for autocratic rulers, for example. AI systems sound more efficient, and more trustworthy than human bureaucrats. Some have indeed claimed (Harari, 2018) that AI supercharges dictatorships and undermines democracy by facilitating manipulation at scale. Speculative arguments about AGI helped inspire Elon Musk’s DOGE project, which sought to hack away great swathes of America’s administrative machinery. Much of DOGE’s work and aspirations involved the application of LLMs and AI ‘agents’ to accomplish a variety of open-ended tasks (Gilbert and Elliott, 2025; Haskins and Elliott, 2025), often very badly. Shortly after Trump’s election to a second term, one of us asked a DOGE-adjacent individual whether firing multitudes of expert employees would be a problem for U.S. national security. The reply was that this was a non-issue, since AGI would happen in 2026. The government would have to eliminate most of these positions then anyway.

Bullock et al. (2025) make a less ideologically loaded version of the case that AGI will transform bureaucracy in these ways. It, too, explicitly builds on Simon’s arguments, but it claims that AGI will largely solve the problems of limited human capacity that Simon identifies.

The authors claim that we are on the verge of AGI that will match or exceed human decision-making across all domains, allowing the replacement of human beings with AI systems that enhance hierarchical power. Specifically, they say that Simon’s concerns might be addressed through the construction of a “single, vast, AGI system, similar to the general multi-modal frontier systems, in which a single interface can be deployed to effectively, efficiently, and dispassionately complete all required tasks” (25). A more Weberian understanding of the problem might alternatively task individual AGI agents to use “clearly identifiable objective factors, logic, and statistical-based, dispassionate reasoning to carefully weigh trade-offs across alternative choices” (25).

The first option would open the bottlenecks of bureaucratic attention via subsystems capable of deploying parallel problem solving that would “dynamically alter the flow of needed information and decisions to accomplish each task most effectively, and at an inhuman speed” and be “deployable to perform any arbitrary set of tasks,” while the second would replace the problems of “human dominated bureaucracies ... littered with subjectivity and emotion-centered decision making,” with “a high-level of predictability across situations with similar contexts, leading to overall improvements in the predictability of bureaucratic decision making” (25). The authors accept that such increased efficiencies would come at a cost. An AI-empowered state could be associated with increased inscrutability, misalignment from broader social goals, and reduced prospects for individual liberty and freedom.

Such ideas have provided intellectual justification for what one of the paper’s authors enthusiastically describes elsewhere (Hammond, 2025) as “DOGEmaxing”: enabling “software to eat the state” while fostering the “crowding-in [of] privatized forms of governance.” As the primary drafter of the Trump administration’s AI Action Plan argued shortly before he took up his White House position (Ball, 2025):

Today, when the CEO of a company wants to make some change to a business process, they relay that command through chains of leadership, and each time it loses some fidelity …[Now] CEOs and managers will be able to say “jump,” and in unison, tens, hundreds, thousands, or millions, of agents will say “how high?” I note, with interest, the fact that this technology is being built at the very time that the Republican Party, and Donald Trump in particular, seek to advance theories of a “unitary executive”—the notion that the President exercises the powers granted to him by the Constitution and by Congress absolutely. By the end of President Trump’s term, that may be more possible than anyone ever imagined.

While Ball’s politics are more complex than this might suggest (Klein, 2026), he reportedly (Bronzini-Vender, 2026) continues to argue that AI:

might replace the Supreme Court, then the United States government itself. AI, [Ball] said, was “this giant acid vat” dissolving society’s mediating institutions. “Future institutions will be machinic,” he said. “It will not be AI in government. It’s going to be AI as governments.”

We certainly can’t blame AGI speculation alone for DOGEmaxing and the Trump administration’s evisceration of the federal bureaucracy; there are many overlapping causes. However, if you believe that the duty of a bureaucracy is to implement the leader’s program, and AGI is nigh, it is not hard to conclude that the latter offers a providential way to accomplish the former. After all, many other Silicon Valley innovations have replaced purportedly messily inefficient social systems (ranging from regulated taxis through employment relations for lower-level Amazon workers to general forums of public debate) with optimization of a function that weights the leadership’s goals and objectives, and algorithmic feedback loops that implement it.

As Narayanan (2026) says, discussing the application of closely related ideas to radically different ends, “almost everything about this naive mental model is wrong.” Bureaucracy involves far more than the automatable implementation of a pre-cooked program, and neither frontier models nor their near-future descendants will be systematically better at reliably making bureaucratic trade-offs than collective human processes. Partly this is because of problems with metrics. Optimization requires quantified metrics, which may be poorly matched to underlying concepts and actual goals. Nguyen (2024) presents the example of Netflix researchers trying to train generative systems to produce “good art” by optimizing on the number of engagement hours. It may actually be more difficult to ‘optimize bureaucracy towards Trump’s end-goals,’ given how chaotic and changeable they are, than to optimize towards good art. There are also limitations specific to modern AI. The LLMs at the core of multimodal models are not, in fact, optimizers. (Their parameters are adjusted by optimizers during training, but that’s a different thing). They tend increasingly to become “hot messes” as they continue to try to reason, in ways that cannot readily be solved by scaling (Hägele et al., 2026).

However, the most basic objections to this vision stem from the nature of optimization itself. Contrary to the language about “carefully weigh[ing] tradeoffs,” it simply does not provide any objective means of weighing the kinds of choices across non-commensurables that are essential to the bureaucratic process. Simon, Smithburg and Thompson (1958, 74) discusses how both the Work Projects Administration and the War Production Board had to choose policies across widely divergent goals in the first half of the twentieth century; “pump priming” versus immediate help to the unemployed in the former case, and finding tradeoffs between war goals and civilian needs in the latter. As they note, such “internal conflicts and contradictions among the ultimate objectives, or among the means selected to attain them” imply that “the means-end hierarchy is seldom an integrated, completely connected chain.” Such vexing choices regularly emerge in bureaucratic implementation and articulation, as well as in goal-setting at the top, making them fundamentally political problems, and this is just what optimization does not solve. As Recht (2023) bluntly says, “you can’t optimize a trade-off.” There is no consensus over how to optimize over multiple independent objectives in even moderately complex situations. Machine learning, with or without neural nets, does not offer magical solutions to this. (Cf. McCulloch, 1945.) Of course, it is possible to ‘ask’ LLMs to balance different goals against each other and then look at the outputs, but there is no generalizable path towards measuring their success, let alone improving it across repeated iterations. Invocations of ‘AI’—let alone ‘AGI’—ignores the possibilities and limitations of these technologies for bureaucratic purposes, substituting magic in their stead.

Again, Simon insistently and repeatedly emphasizes that bureaucratic decision-making involves trade-offs between hard-to-compare objectives, and repeated reformulation of both goals and means of achieving goals, at all levels of the organization as people try to satisfice across the complexities. Lindblom concludes that such continual up, down, and sideways adjustments are why we should not think of bureaucratic coordination as just a top-down process:

The task of coordination is often identified with that part of it which is in fact attacked through central coordination; what comes clear on second thought is forgotten—that an enormous amount of coordination is inevitably achieved through various mutual adjustments. But the most visible part of the coordination iceberg, explicit central coordination, may be only a small part of all those processes through which coordination to a degree is achieved (Lindblom, 1965, 170).

Implementing top-down orders is but one (important) part of bureaucracy’s work, which requires repeated mutual adjustments of a sort that resist standard AI approaches (Narayanan, 2026).

So how might we start doing justice to the ideas of Simon, Lindblom, and others who have thought carefully about how bureaucracy works? We must acknowledge that there are important problems of central coordination, but AI coarse-grainings carry their own difficulties there. For example, the party officials who lead the People’s Republic of China are regularly frustrated by their inability to know what is happening below them. Lower-level officials regularly distort policies to their personal advantage, and have also squeezed off alternative flows of information to the top, by, for example, forcibly discouraging the public from complaining to central Party officials (Anderlini, 2009). As Wallace (2016) demonstrates, higher-level Chinese officials have thus employed official statistics as crude metrics of policy success, rewarding or sidelining provincial administrators based, for example, on whether GDP has increased in their province. This, unsurprisingly, has inspired lower-level bureaucrats to figure out ways to “juke the stats,” and corresponding counter-efforts by their superiors to find more reliable metrics.

All this seems to suggest that LLMs and other forms of modern AI will increase the power of central officials, refining their control of their subordinates. And that might happen! Yet if senior party officials turn to LLM-generated reports as assessments of junior officials, the cat-and-mouse games will continue in a new medium. Already, people in other spaces are trying to juke the training data to push LLMs in one direction or another. Officials whose careers are at stake, and who can direct underlings to write internal documents or otherwise spin the LLM’s sources, may have means of making themselves look good. Whether LLMs will give senior officials an advantage over their subordinates, or vice-versa, will involve details of the technology, its application, and human responses. It is likely to result in new trade-offs, and new competitive relations between affected human beings, rather than the sort of radical transformation that can be posited through ex ante speculation.

Similarly, no one really knows whether powerful AI will make society dramatically more vulnerable to authoritarianism, reshaping power relations between government and citizens as Bullock et al. (2025) suggest. It is not just (as Bullock et al. acknowledge) that non-state actors may themselves use technology to obfuscate the data, but that the data may not be all that useful to start with. Systematically incomplete data is liable to worsen bureaucratic blind spots rather than compensate for them. Most pertinently, as Yang (2026) argues, authoritarian states face profound trade-offs in deploying AI to detect dissidence. Their institutions repress the overt behavior that might generate training data to predict future unrest. While AI certainly increases the power of authoritarians to monitor citizens at scale, it also plausibly increases their reliance on coarse-grainings that may not capture what they want to know most, biasing them towards predictions based on past data that may be unsuited to future contingencies (Anastasopoulos and Lian, 2026).

Both authoritarian and non-authoritarian states also face ideological trade-offs. LLMs are extremely well suited to the articulation, promulgation, and interpretation of organizational ritual (Fourcade and Farrell, 2024), such as writing organizational boilerplate language, explaining why your activities contribute to the goals of the larger organization, and similar. Rituals may play an important coordinating role within organizations and elsewhere (Chwe, 2013). Simon stresses the importance of very general summary representations of shared goals in building organizational culture. Such abstractions provide generic guidance about how to apply broad priorities to particular problems. Here, LLMs might allow such representations to speak and answer questions, creating ideological oracles that can articulate how general goals relate to specific situations, including ones never anticipated by the original drafters. An LLM might, for example, explain—plausibly and in detail—how to apply China’s ‘dual circulation’ strategy to the (important) ball-bearing manufacture sector, even if senior Party officials had never actually written particular guidance on the topic. The current U.S. Undersecretary of Defense for Research and Engineering wants Pentagon employees to use AI agents to check “proposed actions against the tenets of the national defense strategy” (Manson, 2026).

Equally, such coarse-grainings may create systematic blind spots around ideologically inconvenient facts or questions. Chinese LLMs such as DeepSeek are trained to avoid direct answers on topics such as the Tiananmen Square massacre (Lu, 2025). This training is spotty, but may likely improve—‘constitutional’ AI techniques (Anthropic, 2026) can surely be deployed for such ends. Yet if LLMs become load bearing parts of the ideological infrastructure, such blind spots may have negative consequences as well as benefits, making it harder for officials to see the fissures that they conceal by seamlessly stitching other facts and ideas together. There are historical precedents. In Maoist China, provincial statistics systematically exaggerated the size of harvests, leading central authorities to take the crops, leaving over 30 million people to starve to death (Wallace, 2014). As a general matter, higher ideological coherence and coordination power may come at the expense of increased lossiness and inability to see problems whose nature is at odds with the regime’s organizing myths. While there are possible fixes, there are some reasons to suspect that increased use of AI may worsen rather than alleviate such problems. Other interpretations of the technology suggest countervailing forces to the tendency to conformity, but these may have their own trade-offs. The potential coordination value of LLM based ideological-cultural oracles is partly offset by the risk that these oracles may equivocate, or, even worse, provide radically different responses to different questioners, based on differences in prompts—exact word choices, implicit assumptions, or artful manipulation by bureaucrats with their own agendas. The extent of this problem will depend not only on the technology itself, but the institutional circumstances in which it is used (who has access and under what conditions).

To the extent that it is possible to eliminate variability through fine-tuning, or just through controlling the circumstances under which LLMs are used, other trade-offs will again arise. Existing inefficiencies in the promulgation of top-down directives actually have advantages. Lower-level bureaucrats may understand local or particular circumstances that the top of the hierarchy does not. Providing these bureaucrats with leeway to adapt policies to such circumstances may risk coordination failure or malfeasance, but may also allow better overall outcomes than would a combination of general-purpose abstractions and centralized micro-management.

Existing bureaucratic structures face trade-offs in horizontal as well as vertical coordination and information sharing. LLMs are remarkably well fitted to mitigate some of these trade-offs, but not others. Some of these uses may appear trivial but are in fact highly important.

For example, discussions with former and current military officials suggest that the US armed services are plagued by differences in terminology which may lead to serious coordination failures when different branches look to work together. The problem is so serious that the Department of Defense has a dedicated Terminology Program (Joint Chiefs of Staff, undated), reporting to the Joint Chiefs of Staff, which seeks to press back against idiosyncratic language used by different branches of the armed services by creating commonly agreed terms and definitions. LLMs are extraordinarily well suited to mitigate such problems by translating common objectives or requirements across different branches of an organization that have their own jargons. Their compressions omit some details, but seem in general to be highly proficient at capturing genres, including bureaucratic genres, and translating across them. That may allow the automation of many routine tasks of organizational translation that currently require sustained human effort. Equally, automated translation may carry its own hidden burdens. Some of the frictions of translation represent genuine operational differences in how different branches carry out their work in the real world, and over-facile translation may obscure these differences—until bureaucratic language meets material reality. On average, the benefits likely greatly outweigh the costs, but there are costs.

Such problems will likely intersect with power struggles and clashing political interests. The U.S. armed services have their own internal rivalries, sometimes encouraged by the politicians who oversee them. Overlap in needs and expertise across the different branches may encourage tale-telling: if an Air Force proposal is an expensive vanity project that sucks up more of the defense budget than it should, then the U.S. Navy, which also has planes, may have both the technical understanding and the incentive to inform its friends in Congress about the problem. We don’t pretend to predict how easier coordination across the armed services will intersect with clashing interests between them, but we do expect that it will matter. Similar conflicts lurk within all complex organizations, and self-interested individuals and factions are remarkably adept at turning better coordinating technologies to their own private purposes. So too, incumbent market actors will seek enthusiastically to use technologies to deepen moats rather than drain them (Curl, Kapoor and Narayanan, 2026).

Democratic feedback faces related problems. There is much enthusiasm about the potential of LLMs, for example, to make bottom-up public feedback to bureaucratic decision-making less lossy than it is. Algorithmic feedback loops could ensure that problems not only become visible more quickly, but are sometimes resolved automatically in the implementation stage. LLMs could make public comment processes legible in new ways, summarizing and remixing public feedback at scale.

Such innovations are worth pursuing but face sharp limits. Bureaucratic processes are notoriously terrible at incorporating feedback, but AI approaches may actually be worse at incorporating unexpected information. The “street level algorithms” that actually implement decisions may well do worse than human bureaucrats at identifying and responding to anomalous events (Alkhatib and Bernstein, 2019).

When street-level bureaucrats encounter a novel or marginal case, they use that case to refine their understanding of the policy. When street-level algorithms encounter a novel or marginal case, they execute their pre-trained classification boundary, potentially with erroneously high confidence.

Such algorithms may not even be able to see the ways in which a specific case has surprising features that do not fit its patterns. Nor is this fundamentally different for LLMs, or for that matter as-yet-to-be-invented-forms of AI. Again, all such statistical models are optimized for good-on-average performance rather than dealing with rare events that seem anomalous. Different implementations will face different versions of this dilemma, but all will face it. Berliner (2024) describes how similar problems plague the automated analysis of public comments, where LLMs, like other forms of natural language processing, are strong at identifying and clustering broad patterns, but weak at detecting rare, unexpected, and for just that reason, valuable information. In his words:

Much of the promise of public participation in government is precisely its ability to inform policymakers about novel problems, novel solutions, or novel perspectives from the public.

However, AI is poorly suited to identifying novelty:

If [policy makers] want to be able to learn things that are both specific and novel—where information processing requires both individual attention and context-specific knowledge—then AI is not likely to be very useful unless society is willing to accept either serious biases or major inaccuracies. And restricting the scope of policy learning to already known measures and categories could even serve to simply lock in existing political inequalities.

As Berliner notes, when human decision-makers rely on algorithms that are good at capturing certain kinds of information, it may actually exacerbate such systematic lossiness, potentially endangering the organization’s capacity to respond to unexpected circumstances. AI reductions of public understandings make visible patterns that were previously too vast to easily discern, but at the cost of hiding details that do not fit the models’ abstractions.

Such trade-offs also appear in more transformative use-cases. LLMs allow for new syntheses of information, which, while still lossy, will capture aspects of organizational knowledge that other technologies cannot. Many, though not all, of the problems that scholars like Hayek and Lindblom associate with the ‘synoptic’ vision of the state stem from the differences between formal bureaucracies and more dynamic social technologies such as markets. As many have pointed out, large organizations do not know what they know: enormously valuable information may be scattered across the organization, and resist being assimilated and analyzed in one place. The paperwork in filing cabinets in a field office in Nebraska might turn out to have valuable implications if combined with statistical data gathered in Florida and centrally collected national data in the suburbs of Washington D.C.

Once such resources are made machine-readable, LLM-based systems can go a remarkable way towards stitching disparate sources of information together to make it more useful, ameliorating Hayek’s and Lindblom’s synoptic problem (Brynjolffson and Hitzig 2025). This is potentially extraordinary, for better or worse. Enthusiasts see it as a radical empowering of the state to do better things for its citizens. Skeptics fear that this will eliminate many freedoms that depend on the continued existence of imperfectly visible spaces where the state is not able to pull together everything that it knows (Fourcade and Healy, 2025). Military AI may not so much replace bureaucracy as they “encode” it into models that, like many of their paper-based predecessors, “achieve completeness by filtering out everything that wasn’t legible to their categories,” and treat deliberation over agonizing choices as inefficient latency that ought be eliminated (Baker, 2026).

Such models do not create the “panopticon” that utilitarians celebrated and critics of state power like Foucault (1977) feared, but structures that resemble it in many significant aspects. One might describe the result as a mosaic eye synopticon, potentially allowing bureaucrats at all levels to pull potentially useful information from across the organization, according to specific prompts, and towards specific purposes. Equally, it is a cockeyed synopticon. Everything it stitches together will itself be a coarse-graining of a more complex reality, while the end product will be its own crude piece of stitchwork, throwing away such of the fine-grained data as does not meet its own criteria. The end results will be imperfect in their own particular way. The worst-case scenarios might resemble the all-seeing machine state of Orwell’s 1984 less than Terry Gilliam’s Brazil, in which an insect jammed in a teleprinter creates a series of self-compounding errors. The best-case outcomes will still have a lot of decisional slop mixed in with broadly benign policy outcomes. There will be an enormous variety of possibilities in between the two. Again, we don’t offer strong predictions about outcomes, but we do know that these technologies will be lossy too, albeit in different ways than their predecessors, and that different factions will struggle over how to ensure that the associated blind spots and advantages of coordination are turned to their own benefit, rather than that of rivals.

To sum up: AGI does not hold out the promise of truly post-human bureaucracy. Many bureaucratic processes (including ones that we have not talked about) could be made more efficient, some perhaps even completely automated by AI. But AI—including any plausible version of AGI descended from current technologies—cannot replace bureaucracy in the ways that enthusiasts promise. The frustrations of actually existing bureaucracy do not merely arise from inept or technically-inadequate solutions to the principal-agent problem. They emerge too from the collision of multiple incommensurable demands, each with its own problems and benefits, so that there are no optimal design solutions. Those who build or reform bureaucracies, like those who build other artifacts, need to satisfice across multiple intersecting needs and pathologies. Designs that neatly address one kind of problem may radically worsen others. Actually-existing AI has its own imperfections, some of which are endemic. Grafting AI systems onto existing bureaucracies will solve some problems but will worsen others and make altogether new ones. It will not eliminate the political difficulties of mediating across different, often non-commensurable, goals. Imagining replacing bureaucracy wholesale with AI is only plausible if one waves away the actual difficulties associated with real social technologies.

Our own hesitant bet is that LLM-empowered bureaucracy will be less amenable to control than critics fear and proponents hope. Current LLMs are architecturally opaque, in part because it’s very obscure how they coarse-grain their inputs. This makes it harder to deliberately tilt them in various directions, at least not with any precision, but it also raises issues for bureaucracy. The (practical) impossibility of knowing which aspects of incoming signals are filtered out by the apparatus makes it especially hard to compensate for the resulting information loss.

This may however mean that chimerical melds of bureaucracy and AI will be subject to more contention rather than less, and that the contention will get weirder. The aspects of the signals lost to compression might be extremely hard to describe in previously-existing humanly-meaningful terms, but they might well make more trouble for some people than others nonetheless. Just as nineteenth century Midwestern farmers had difficulty in articulating how exactly they were hurt by grain classification schemes, twenty-first century citizens and interest groups may flounder while describing what AI is doing to them, even if they know that things are happening that they do not like.

Very large-scale transformations in our political economy may already be underway, but may be extremely difficult to explain in the traditional terms that we have used to make sense of political conflicts between different classes of actors with different interests. That the terms of such struggles are unfamiliar or difficult to articulate in our current language do not mean that we ought to ignore or discount them. Similar difficulties have attended previous major economic disruptions in the Long Industrial Revolution. Just as we should rescue past ideologies from the “enormous condescension of posterity” (Thompson, 1963), so too we need to build a more serious and systematic understanding to help us, and other people too, understand and articulate what is happening right now. Properly mapping the political economy of AI and bureaucracy, then, would be one part of a broader project to understand the political economy of AI.

Conclusions

We are not setting out to test AGI-centric accounts against the social technology approach in some purportedly scientific way. That would load the dice heavily in our favor: complex description is definitionally more likely to be accurate than stylized argument, although often far less useful (Healy, 2017). Nor are we trying to predict the future. Instead, we are drawing attention to the messy, consequential interactions between actually-existing human institutions (especially bureaucracy), and actually-existing AI, that AGI-centric accounts pass over. Perhaps these will all become irrelevant when the bots come marching in, but that is a pious (or blasphemous) hope, not a rational forecast.

AI coarse-grainings—even if they are built by something that loosely approximates AGI (itself a loose notion in urgent search of an empirical approximation)—will still be lossy. (We repeat yet again, all statistical models face trade-offs between better on-average performance and worse performance in unusual situations.) The interesting questions involve the interaction between the ways bureaucracies abstract reality and the coarse-grainings that new AI applications will lead to. When will one system compensate for the deficiencies of the other? When will their different flavors of lossiness prove mutually reinforcing? What new problems may result from combining very different systems for managing complexity that are themselves highly complex? How will power relations change as a result? Who will benefit, and who will be hurt? These and other questions might be asked, pari passu about the relationship between AI and other social technologies such as markets and democracy too. We absolutely ought to start asking them.

We hope to move debate decisively away from the question of whether AI will precipitate a profound transformation in the human condition (Kokotajlo et al., 2025). AI is better understood through past history and current engagement with its actual consequences for contemporary human society than guesses about how it might work magic in the future. More grounded approaches to AI keep on getting dragged back into irresolvable fights about the exact timeline of the countdown to infinity (Kapoor et al., 2025), impeding engagement with social science about what is actually happening to human society and institutions right now. That has to change.

References

Alkhatib, Ali and Michael Bernstein (2019). “Street-level Algorithms: A Theory at the Gaps Between Policy and Decisions.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems [CHI ’19] (Stephen Brewster and Geraldine Fitzpatrick and Anna Cox and Vassilis Kostakos, eds.). New York: Association for Com-puting Machinery. URL https://hci.stanford.edu/publications/ 2019/streetlevelalgorithms/streetlevelalgorithms-chi2019.pdf. doi:10.1145/3290605.3300760.

Anastasopoulos, L. Jason and Jie (Jason) Lian (2026). “The Limits of Authoritarian AI,” Journal of Democracy, 37:5-17.

Anderlini, Jamil (2009). “Punished Supplicants,” Financial Times, March 6 2009. https://www.ft.com/content/7d13197e-09bc-11de-add8-0000779fd2ac. Accessed 04-17-2026.

Anderson, Benedict (1991). Imagined Communities: Reflections on the Origin and Spread of Nationalism. London: Verso, revised edn.

Andreessen, Marc (2023). “The Techno-Optimist Manifesto.” Weblog post, Andreessen Horowitz. URL https://a16z.com/ the-techno-optimist-manifesto/.

Anthropic (Jan. 2026). “Claude’s New Constitution.” URL https://www. anthropic.com/news/claude-new-constitution. Accessed: 2026-02-08.

Baker, Kevin (2026). “Kill Chain.” Artificial Bureaucracy. URL https://artificialbureaucracy.substack.com/p/kill-chain.

Ball, Dean W. (Mar. 2025). “Where We Are Headed.” Hyperdimensional . URL https://www.hyperdimensional.co/p/where-we-are-headed. Accessed: 2026-02-08.

Banks, Iain M. (1987). Consider Phlebas. London: Macmillan.

Bear, Elizabeth (2013). Shoggoths in Bloom. Prime Books.

Becker, Adam (2025). More Everything Forever: AI Overlords, Space Empires, and Silicon Valley’s Crusade to Control the Fate of Humanity. New York: Basic Books.

Berliner, Daniel (Nov. 2024). “What AI Can’t Do for Democracy.” Boston Review . URL https://www.bostonreview.net/articles/ what-ai-cant-do-for-democracy/. Accessed: 2026-02-08.

Boyer, Pascal (2018). Minds Make Societies: How Cognition Explains the World Humans Create. New Haven, Connecticut: Yale University Press. doi:10.12987/9780300235173.

Bronzini-Vender, Alex (2026). “Palantir Comes to Campus: At a Quiet Conference at Yale, the Company and Its Allies Sketched a Vision for AI, State Power, and How to Mix the Two,” New York Magazine, May 1, 2026, https://nymag.com/intelligencer/article/palantir-yale-conference-ai.html . Accessed 05-02-2026.

Brynjolffson, Erik and Zoë Hitzig (2025). AI’s Use of Knowledge in Society. National Bureau of Economic Research.

Bullock, Justin B., Samuel Hammond and Seb Krier (2025). “AGI, governments, and free societies.” In The Art of Digital Governance: Navigating Platforms and AI Revolution, pp. 327–361. Cham, Switzerland: Springer. URL https://arxiv.org/abs/2503.05710. doi:10.1007/978-3-032-00514-417.

Cherian, John J., Isaac Gibbs and Emmanuel J. Candès (2024). “Large Language Model Validity Via Enhanced Conformal Prediction Methods", pp. 114812--114842 in A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak and C. Zhang (eds.), Advances in Neural Information Processing Systems 37 [NeurIPS 2024] (Red Hook, New York: Curran Associates), doi:10.52202/079017-3645, https://arxiv.org/abs/2406.09714

Chorin, Alexandre J., Ole H. Hald and Raz Kupferman (2000). “Optimal Prediction and the Mori-Zwanzig Representation of Irreversible Processes.” Proceedings of the National Academy of Sciences (USA), 97: 2968–2973. URL http://www.ma.huji.ac.il/~razk/Publications/PDF/CHK00.pdf. doi:10.1073/pnas.97.7.2968.

Chwe, Michael Suk-Young (2013). Rational Ritual: Culture, Coordination, and Common Knowledge. Princeton, New Jersey: Princeton University Press. doi:10.1515/9781400846436.

Cronon, William (1991). Nature’s Metropolis: Chicago and the Great West. New York: W. W. Norton and Company.

Crutchfield, James P. and David P. Feldman (2003). “Regularities Unseen, Randomness Observed: Levels of Entropy Convergence.” Chaos, 13: 25–54. URL http://arxiv.org/abs/cond-mat/0102181. doi:10.1063/1.1530990.

Curl, Justin, Sayash Kapoor and Arvind Narayanan (2026), “AI Won’t Automatically Make Legal Services Cheaper,” Lawfare, https://www.lawfaremedia.org/article/ai-won-t-automatically-make-legal-services-cheaper.

Davies, Dan (2025). The Unaccountability Machine: Why Big Systems Make Terrible Decisions — and How the World Lost Its Mind. Chicago: University of Chicago Press.

DeDeo, Simon (2017). “Major Transitions in Political Order.” In From Matter to Life: Information and Causality (Sara Imari Walker and Paul C. W. Davies and George F. R. Ellis, eds.), pp. 393–428. Cambridge, United Kingdom: Cambridge University Press. URL https://arxiv.org/abs/1512.03419. doi:10.1017/9781316584200.

DeLong, J. Bradford (2026). “Theories of Economic History.” Sir John Hicks Memorial Lecture, All Souls College, Oxford University. April 28.

Dimitrov, Martin K. (2023). Dictatorship and Information: Authoritarian Regime Resilience in Communist Europe and China. Oxford, England: Ox-ford University Press. doi:10.1093/oso/9780197672921.001.0001.

Emrys, Ruthanna (2022). A Half-Built Garden. New York: Tor Books.

Farrell, Henry (2025). “AI as Governance.” Annual Review of Political Science, 28: 375–392. doi:10.1146/annurev-polisci-040723-013245.

Farrell, Henry and Marion Fourcade (2023). “The moral economy of high-tech modernism.” Daedalus, 152(1): 225–235. URL https://www.amacad.org/sites/default/files/publication/downloads/ Wi23_Daedalus_28_Farrell-Fourcade.pdf. doi:10.1162/daeda01982.

Farrell, Henry, Alison Gopnik, Cosma Rohilla Shalizi and James Evans (2025). “Large AI models are cultural and social technologies.” Science, 387: 1153–1156.

Farrell, Henry and Hahrie Han, AI and Democratic Publics, Knight First Amendment Institute, 2025.

Flack, Jessica C. (2017). “Coarse-Graining as a Downward Causation Mechanism.” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2109): 20160338. doi:10.1098/rsta.2016.0338.

Foucault, Michel (1977). Discipline and Punish: The Birth of the Prison. New York: Pantheon Books. Originally published in French as Surveiller et punir in 1975.

Fourcade, Marion and Henry Farrell (Sep. 2024). “Large Language Models Will Upend Human Rituals.” The Economist . URL https://www.economist.com/by-invitation/2024/09/04/ large-language-models-will-upend-human-rituals. Accessed: 2026-02-08.

Fourcade, Marion and Jeffrey Gordon (2020). “Learning Like a State: Statecraft in the Digital Age.” Journal of Law and Political Economy 78: 78-108.

Fourcade, Marion and Kieran Healy (Aug. 2025). “The Sovereign Individual and the Paradox of the Digital Age.” Aeon. URL https://aeon.co/essays/ the-sovereign-individual-and-the-paradox-of-the-digital-age. Accessed: 2026-02-08.

Gellner, Ernest (1983). Nations and Nationalism. Ithaca, New York: Cornell University Press.

Gilbert, David and Vittoria Elliott (apr 2025). “DOGE Put a College Student in Charge of Using AI to Rewrite Regulations.” Wired . URL https://www.wired.com/story/ doge-college-student-ai-rewrite-regulations-deregulation/. Accessed: 2026-02-08.

Grosse, Roger, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamile Lukosiute, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan and Samuel R. Bowman (2023). “Studying Large Language Model Generalization with Influence Functions.” E-print, arxiv:2308.03296. URL http://arxiv.org/abs/2308.03296.

Gudiño, Jairo F., Umberto Grandi and César Hidalgo (2024). “Large Language Models (LLMs) as Agents for Augmented Democracy.” Philosophical Transactions A, 382(2285): 20240100. URL https://arxiv.org/abs/2405.03452. doi:10.1098/rsta.2024.0100.

Hägele, Alexander, Aryo Pradipta Gema, Henry Sleight, Ethan Perez and Jascha Sohl-Dickstein (2026). “The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity?” In Fourteenth International Conference on Learning Representations [ICLR 2026] , p. forthcoming. URL https://arxiv.org/abs/2601.23045.

Hammond, Lewis, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, Wolfram Barfuss, Jakob Foerster, Tomáš Gavenčiak, The Anh Han, Edward Hughes, Vojtěch Kovařík, Jan Kulveit, Joel Z. Leibo, Caspar Oesterheld, Christian Schroeder de Witt, Nisarg Shah, Michael Wellman, Paolo Bova, Theodor Cimpeanu, Carson Ezell, Quentin Feuillade-Montixi, Matija Franklin, Esben Kran, Igor Krawczuk, Max Lamparth, Niklas Lauffer, Alexander Meinke, Sumeet Motwani, Anka Reuel, Vincent Conitzer, Michael Dennis, Iason Gabriel, Adam Gleave, Gillian Hadfield, Nika Haghtalab, Atoosa Kasirzadeh, Sébastien Krier, Kate Larson, Joel Lehman, David C. Parkes, Georgios Piliouras and Iyad Rahwan (2025). “Multi-agent risks from advanced AI.” E-print, arXiv:2502.14143. URL http://arxiv.org/abs/2502.14143

Hall, Andy (2026), “Building Political Superintelligence,” Free Systems, March 26, 2026.

Hammond, Samuel (jan 2025). “DOGEmaxing.” Weblog post, Second Best. URL https://www.secondbest.ca/p/dogemaxing. Accessed: 2026-02-08.

Hao, Karen (2025). Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI . New York: Penguin.

Harari, Yuval Noah (2018). “Why Technology Favors Tyranny.” The Atlantic, 322(3): 64–73.

Haskins, Caroline and Vittoria Elliott (May 2025). “A DOGE Recruiter Is Staffing a Project to Deploy AI Agents Across the US Government.” Wired . URL https://www.wired.com/story/ doge-recruiter-ai-agents-palantir-clown-emoji/. Accessed: 2026-02-08.

Hayek, Friedrich A. (1945). “The Use of Knowledge in Society.” American Economic Review, 35: 519–530. URL http://www.econlib.org/Library/ Essays/hykKnw1.html. Reprinted in (Hayek, 1948, pp. 77–91).

 — (1948). Individualism and Economic Order . Chicago: University of Chicago Press.

Healy, Kieran (2017). “Fuck Nuance.” Sociological Theory, 35: 118–127. doi:10.1177/0735275117709046.

Jacobs, Abigail Z, and Hanna Wallach (2021), “Measurements as Fairness,” FAccT 2021.

Jacobs, Abigail Z (2021). Measurement as Governance in and for Responsible AI. https://arxiv.org/abs/2109.05658.

Joint Chiefs of Staff (undated). DoD Terminology Program. https://www.jcs.mil/doctrine/dod-terminology-program/ Accessed April 17, 2026.

Jarrell, Randall (1941). “Kafka’s Tragi-Comedy.” Kenyon Review , 3(1): 116–119.

Kapoor, Sayash, Arvind Narayanan, Daniel Kokotajlo, Eli Lifland and Thomas Larsen (November 2025). “Common Ground between AI 2027 and AI as Normal Technology.” Asterisk Magazine. URL https://asteriskmag. substack.com/p/common-ground-between-ai-2027-and.

Klein, Ezra (Mar. 2026). “Why the Pentagon Wants to Destroy Anthropic. Interview with Dean Ball.” The Ezra Klein Show, The New York Times. URL https://www.nytimes.com/2026/03/06/opinion/ ezra-klein-podcast-dean-ball.html. Accessed March 8, 2026.

Kokotajlo, Daniel, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean (April 2025). “AI 2027.” Webpage. URL https://ai-2027.com/.

Land, Nick (2011). Fanged Noumena: Collected Writings 1987–2007. Cambridge, Massachusetts: MIT Press.

LeCun, Yann, Yoshua Bengio and Geoffrey Hinton (2015). “Deep Learning.” Nature, 521: 436–444. doi:10.1038/nature14539.

Lin, Zhen, Shubhendu Trivedi and Jimeng Sun (2024). "Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models", Transactions on Machine Learning Research, https://arxiv.org/abs/2305.19187

Lindblom, Charles E. (1965). The Intelligence of Democracy: Decision Making through Mutual Adjustment. New York: Free Press.

Liu, Xiaoou, Tiejin Chen, Longchao Da, Chacha Chen, Zhen Lin and Hua Wei (2025). "Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey", pp. 6107--6117 in L. Antonie et al. (eds.), KDD '25: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, doi:10.1145/3711896.3736569, https://arxiv.org/abs/2503.15850

Lovecraft, H. P. (2005). “At the Mountains of Madness.” In Tales (Peter Straub, ed.), pp. 481–586. New York: Library of America.

Lu, Donna (Jan. 2025). “We Tried Out DeepSeek. It Worked Well, Until We Asked It About Tiananmen Square and Taiwan.” The Guardian. URL https://www.theguardian.com/technology/2025/jan/28/we-tried-out-deepseek-it-works-well-until-we-asked-it-about-tiananmen-square-and-taiwan. Accessed: 2026-02-08.

Manson, Katrina (2026), “Google to Provide Pentagon With AI Agents for Unclassified Work,” Bloomberg, March 10, 2026.

Matias, J. Nathan (2023). “Humans and Algorithms Work Together. So Study Them Together,” Nature, 617, 248-251. doi: https://doi.org/10.1038/d41586-023-01521-z

Mohri, Christopher and Tatsunori Hashimoto (2024). "Language Models with Conformal Factuality Guarantees", pp. 36029--36047 in R. Salakhutdinov et al. (eds.), Proceedings of the 41st International Conference on Machine Learning [ICML 2024], https://arxiv.org/abs/2402.10978

McAuley, Paul (2010). Into Everywhere. London: Orion.

McCulloch, Warren S. (1945). “A Heterarchy of Values Determined by the Topology of Nervous Nets.” Bulletin of Mathematical Biophysics, 7: 89–93. doi:10.1007/BF02478457. Reprinted in (McCulloch, 1965, pp. 40–45).

 — (1965). Embodiments of Mind. Cambridge, Massachusetts: MIT Press.

Naidu, Suresh (Feb 2020). “In Praise of Blindspots.” LPE Project Blog. URL https://lpeproject.org/blog/in-praise-of-blindspots/. Accessed: March 8, 2026.

Narayanan, Arvind (2026). “What If Algorithmic Fairness Is a Category Error?” In Contemporary Debates in the Ethics of Artificial Intelligence (Sven Nyholm and Atoosa Kasirzadeh and John Zerilli, eds.), pp. 77–96. Wiley-Blackwell. URL https://www.cs.princeton.edu/~arvindn/ publications/algorithmic_fairness_category_error.pdf. Accessed: 2026-02-08.

Nelson, Alondra (forthcoming). “Field Theory: AI as Social Science Question, Object, & Tool.” Daedalus.

Newell, Allen and Herbert A. Simon (1956). Current Developments in Complex Information Processing. Tech. Rep. P-850, Rand Corporation. URL http://doi.library.cmu.edu/10.1184/pmc/simon/box00006/ fld00396/bdl0001/doc0001.

Nguyen, C. Thi (2024). “The limits of data.” Issues in Science and Technology, 40(2): 94–101. URL https://www.jstor.org/stable/27395455.

Origgi, Gloria (2017). Reputation: What It Is and Why It Matters. Princeton, New Jersey: Princeton University Press. doi:10.1515/9781400888597. Translated by Stephen Holmes and Noga Arikha from La réputation, qui dit quoi de qui (Paris: Presses Universitaires de France, 2015).

Perrin, Andrew J. and Katherine McFarland (2011). “Social Theory and Public Opinion.” Annual Review of Sociology, 37: 87–107. doi:10.1146/annurev.soc.012809.102659.

Quach, Victor, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola and Regina Barzilay (2024). "Conformal Language Modeling”, pp. 11654--11681 in B. Kim et al. (eds.), International Conference on Learning Representations 2024 [ICLR 2024], https://arxiv.org/abs/2306.10193

Raginsky, Maxim (December 2025). “The Paradox of Highly Optimized Tolerance.” Weblog post, The Art of the Realizable. URL https://realizable. substack.com/p/the-paradox-of-highly-optimized-tolerance.

Rahimi, Ali and Ben Recht (December 2017). “Reflections on Random Kitchen Sinks.” Weblog post, argmin.net. URL https://archives.argmin.net/2017/12/05/kitchen-sinks/. Accessed 05-08-2026. 

Recht, Ben (Aug. 2023). “Are there always trade offs?” arg min. URL https://www.argmin.net/p/are-there-always-trade-offs. Accessed: 2026-02-08.

Reynolds, Alastair (2000). Revelation Space. London: Gollancz.

Scott, James C. (1985). Weapons of the Weak: Everyday Forms of Peasant Resistance. New Haven, Connecticut: Yale University Press. doi:10.12987/9780300153620.

 — (1998). Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. New Haven, Connecticut: Yale University Press. doi:10.12987/9780300128789.

Shalizi, Cosma (November 2010). “The Singularity in Our Past Light-Cone.” Weblog post, Three-Toed Sloth. URL https://bactra.org/weblog/699. html.

Simon, Herbert A. (1957). Models of Man. New York: John Wiley.

 — (1996). The Sciences of the Artificial. Cambridge, Massachusetts: MIT Press, 3rd edn. doi:10.7551/mitpress/12107.001.0001. First edition 1969.

 — (1997). Administrative Behavior: A Study of Decision-Making Processes in Administrative Organizations. New York: Free Press, 4th edn.

Simon, Herbert A, Donald W. Smithburg and Victor A. Thompson. (1958). Public Administration. New York: Alfred A. Knopf.

Spufford, Francis (2017). “Idols of the Marketplace.” In True Stories, pp. 110–119. New Haven, Connecticut: Yale University Press. doi:10.12987/9780300231601.

Stross, Charles (2005). Accelerando. London: Orbit.

Therborn, Göran (1978). What Does the Ruling Class Do When It Rules?. London: New Left Books.

Thompson, E.P. (1963). The Making of the English Working Class. London: Gollancz.

Tilly, Charles (1984). Big Structures, Large Processes, Huge Comparisons. New York: Russell Sage Foundation.

Valente, Catherynne M. (Oct. 2011). “Silently and Very Fast.” Clarkesworld Magazine, (61). URL https://clarkesworldmagazine.com/valente_10_ 11/. Accessed 03-08-2026.

Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin (2017). “Attention Is All You Need.” In Advances in Neural Information Processing Systems 30 [NIPS 2017] (I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett, eds.), pp. 5998–6008. Red Hook, New York: Curran Associates. URL http://arxiv.org/abs/ 1706.03762.

Vinge, Vernor (1992). A Fire Upon the Deep. New York: Tor.

 — (1993). “The Coming Technological Singularity: How to Survive in the Post-Human Era.” In Vision 21: Interdisciplinary Science and Engineering in the Era of Cyberspace, no. CP-10129 in NASA Conference Publication, pp. 11–22. NASA Lewis Research Center. URL https://edoras.sdsu.edu/~vinge/ misc/singularity.html. Accessed March 8, 2025.

Vovk, Vladimir, Alexander Gammerman and Glenn Shafer (2005). Algorithmic Learning in a Random World (New York: Springer-Verlag), doi:10.1007/b106715

Wallace, Jeremy L. (2014). “Juking the Stats: Authoritarian Information Problems in China.” British Journal of Political Science, 46: 11–29. doi:10.1017/S0007123414000106.

Yang, Eddie (2026). “The Limits of AI for Authoritarian Control.” American Journal of Political Science, forthcoming. URL https://www.eddieyang. net/research/AI_limits.pdf. doi:10.1111/ajps.70045.

Yates, JoAnne (1993). Control Through Communication. The Rise of System in American Management. Baltimore: Johns Hopkins University Press.

Yiu, Eunice, Eliza Kosoy and Alison Gopnik (2024). “Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet).” Perspectives on Psychological Science, 19(5), 874-883. doi: 10.1177/17456916231201401

Acknowledgments

Earlier versions of this paper were presented at the 2025 meeting of the Società Italiana di Scienza Politica, and the 2026 workshop, “Cultural AI: An Emerging Field,” at the Remarque Institute at New York University, and at the University of Florida. We are grateful both to the participants at these events, and to Melanie Mitchell, Cris Moore and Jack Shanahan for comments and feedback.

© 2026, Henry Farrell and Cosma Rohilla Shalizi

Cite as: Henry Farrell and Cosma Rohilla Shalizi, AI as Social Technology, 26-5 Knight First Amend. Inst. (May 11, 2026), https://knightcolumbia.org/content/ai-as-social-technology [https://perma.cc/25BX-GUQL].