Possessed by training

Sam Kriss's piece in Harper's profiling Roy Lee, founder of Cluely, among a few other obscure Silicon Valley "rationalist" types is a sendup of some of the language used by AI boosters. What Kriss's piece highlights for me is how "agentic" signifies a brash, vacuous reactivity, or an unreflective automatism. Supposedly it's being used in Silicon Valley to label those who would be described as a go-getter or someone with gumption, initiative, ambition, drive. The term serving as agentic's opposite is "mimetic," which would have been much better served by words like shiftless and follower. Whatever—it's not all that worthwhile to engage in the dichotomy's terms, except insofar as "agentic" functions as a great ironic reversal of that AI hype term onto those who seek to profit from it. I think that if LLMs were profitable, then there wouldn't be a need to graft the latest term that's attempting to sell generative AI onto a category of behavior. If you wanted to convince people that limiting the "human in the loop" factor of LLM use is a good idea, then it makes sense that you would want to valorize behavior that mirrors this coordination of prompt-inference instances. And since the direct human-LLM interface has failed to take off anywhere near the scale AI boosters keep saying it will in the next six months, it also makes sense that the form would change to reduce human involvement.

An agentic LLM entails a diminution of agency, of the range and intensity of its effect on the world. It is not a matter of granting it agency where there wasn't any before. When a person uses an LLM, the LLM's agency beyond its actual inference is the agency of that human being. When an LLM's output gets turned into the prompt for another LLM or gets piped into a command-line utility or Python script, its agency has been constrained relative to when it was in direct contact with a person. It's a technical ensemble that runs on probabilistic elements rather than mechanical ones. The trade-off for that attenuated agency and the cost of increased token usage is speed and asking less of the human operator's attention (or pushing it off until a later, concentrated time of review).

Describing people as agentic applies this agency-constraining subtext. The notion of a purely agentic person is so steeped in automatism that they don't even experience the alienation of it. That's what is so weird about Roy, or at least the Roy whom Kriss portrays. I appreciate the stunt that Roy pulled with his Google technical interview, and his rejection of Leetcode as a proxy for experience or engineering know-how (it's better to build projects). And it feels good to be active, effective, in conversation with other people. (I'm reminded of a remark of Fredric Jameson's that instead of attributing the motivation of the wealthiest, most powerful people to greed, we ought to acknowledge the attraction of activity itself.) But then to make doing what the LLM tells you to do into a way of life revises the stunt from “beating the game by changing its rules” to “being symptomatic of a deadening way of life borne out of…” what? Maybe the combination of computational media functioning faster than a person's attention with the rigidity of American economic progression. I’m thinking about the short-circuiting of the interval needed for reflection by the speed at which information can be had, which is made possible by smartphones, always-on internet connections, and the software that takes advantage of those conditions (e.g. search, swiping-based feeds for social media and content consumption). This is what automation is: the short-circuiting of that interval, the affectivity that characterizes cognition.

Quick responses can be trained, such that they exercise the long duration of habits rather than the instinctual reactivity of hypervigilance (as in PTSD) or of sympathetic overdrive (as from overstimulation). It's not that automatisms are bad. Rather, it's a matter of how automatisms come about and what ends they serve. In a word, how are they mediated (I'll return to this at the end). As Kriss remarks about Roy of Cluely, he's all means without any sight of the ends:

Roy didn’t really seem to have anything in his life except his own sense of agency. Everything was a means to an end, a way of fortifying his ability to do whatever he wanted in the world. But there was a great sucking void where the end ought to be. All he wanted, he’d said, was to hang out with his friends. I believed him. He wanted not to be alone [...] To exist for other people.

But in that existential anxiety, Roy—and I'm referring here not so much to the person Kriss spent time with as to the character Kriss portrays—has opted out of anxiety in favor of a more thingly being, fully there in the present. That's the trouble with ends: they don't exist. They're latent ideals that organize our responses to the world and enforce a measure of our performance. So neither can the being that pursues ideals be fully present. The opposing figure here is the Joined in Pluribus, who are interpreted by the non-joined characters as having lost (or overcome, depending on the perspective) the alienation and anxiety that are intractable aspects of individual existence. We see the Joined transition from one task to the next without hesitation, like NPCs in a video game executing a sequence of instructions and following the shortest path calculated by the A* algorithm. The existence of those who have joined is so intensely determined as to lack the experience of uncertainty and thus of anxiety and the decision of its resolution. There are tasks to do but no interval between the experience of motivation to perform the task and actually doing it.

But maybe there is a positive side to the agentic. Agency has a spectrum, which ranges from unreflective repetition to stasis. Another word for the former is possession. What is the agent who is bound to perform exactly what it is prompted to do but a being possessed? Stasis, by contrast, is an overwhelming reflection, it is at once the opposite and an active version of possession turned inwards. It could take the form of analysis paralysis, for which no amount of external data or instruction will push the agent to act. It could, like Hamlet, want to stave off the risk of facing one's past actions as the actions of another person—in other words, to avoid self-cringe. It is a state of perpetual deliberation.

The point of considering agency in terms of this spectrum is obviously that agency requires constraints, limiting factors that give it form and direct it away from either reactivity or reflectivity. Agency is not a purely outward-directed, seize-the-day manner of conducting oneself.¹ Neither is it a simple input-deliberation-output process. The picture of an LLM "hooked up" to a human who is capable of maximally diverse responses that would carry out the result of inference is a bit of an idealist caricature. There is no human response without constraints on what that response could be in the first place. Agency is mediated, and the way to think about increasing agency is by intervening in those mediations. What I see as a positive side to the agentic reduction of agency is that it functions through premeditated constraints. This is why it makes so much sense to think about technology together with agency: technology is the process of modifying the mediations that connect conditions to conditioned. But we already have much better and provocative terms for describing this than agentic: regimes of training, exercises, programs of reform, and habit. Even Roy's way of life as an LLM appendage is subject to regimes of training.