Can a tool have agency? And who answers when it hurts someone?

By rainwiz on 24 October 2026 — 6 mins read

Most public debate treats these questions as a philosophical puzzle about machine consciousness. I would treat them as a design problem instead, and psychology has a body of theory that gives us a sharp vocabulary for it.

A theory from the 1920s

Activity Theory was nowhere near the AI debate. Lev Vygotsky developed it in the 1920s around one argument: human action is always mediated by tools and cultural signs. A child does not learn to count “in their head.” They count on something the culture gives them: fingers, an abacus, a number system. And the tool shapes the thinking as well as the output (remember the dispute with Piaget over constructivism).

Article content

Aleksej Leont’ev later organized the framework into three layers. An activity is driven by a motive, often not fully conscious. Actions sit inside it, driven by goals you are aware of. Operations are the last inner layer, the things you actually execute under a given set of conditions. Scrolling instagram can be an action inside the activity “checking the competition,” or inside the activity “killing ten minutes.” The behavior is the same, while the meaning and the intention behind it are completely different.

Article content
Source: Victor Kaptelinin and Bonnie A. Nardi (Acting with Technology, 2006, p.64)

Then Yrjö Engeström, in Learning by Expanding (1987), turned these elements into a model we can still use in organizational studies. He added three elements to Vygotsky’s triangle. Rules are the governing norms that the activity follows. Community is who else is involved or affected. Division of Labor is how tasks, authority, and responsibility are actually distributed.

Article content

That lower side is the part I want to spend time on, because it answers several of the questions people keep asking the wrong way.

Agency without a subject

Victor Kaptelinin and Bonnie Nardi addressed this directly in Acting with Technology (2006), and their answer is still the clearest I know. A tool can produce effects its designer never anticipated. It can even take over operations, or whole actions, that a human used to perform. None of that makes it a subject. An artifact cannot decide what matters to it or form an intention. It can act, in a limited sense, without being an agent in the full sense.

The word “agency” does a lot of unexamined work in AI debates, so it helps to pin down what it claims. Luciano Floridi and Jeff Sanders, in On the Morality of Artificial Agents (2004), define an artificial agent by three properties: interactivity (the ability to respond to a stimulus by changing state), autonomy (the ability to change state without an external stimulus), and adaptability (the ability to change the rules by which it changes state). Plenty of software meets that definition, and none of it needs intention, consciousness, or free will to do so. Read this way, agency describes what a system does and says nothing about an inner life.

Norbert Wiener made a similar move decades earlier. In Behavior, Purpose and Teleology (1943 with Arturo Rosenblueth and Julian Bigelow) they argued that a machine governed by feedback shows purposeful behavior in a rigorous, observable sense. A servomechanism correcting its course towards a target is the example. Nobody has to claim the machine intends anything, because in their account purpose is a property of the feedback loop.

Floridi and Wiener alongside Kaptelinin and Nardi seem to arrive at the same place. A tool can be interactive, adaptive, and purposeful (all perfectly defensible uses of agency) without ever becoming a subject with intentions of its own.

The distinction sounds academic until you apply it to a case of harm. When people ask “did the AI decide to deny that loan,” they’re smuggling in a subject that isn’t there. The tool executed operations. Training data shaped those operations, and so did a product decision about what counts as risk. All of it ran inside rules nobody wrote down clearly. It also ran inside a division of labor where three different people built the model, deployed it, and live with the consequence, and they may never speak to each other. A decision was made, but the tool didn’t make it. It’s spread across the other corners of Engeström’s triangle, and most organizations only ever look at the top one.

Where the harm lives

Engeström’s concept of contradiction helps here. For him, tension inside one part of a system, or between two systems, is what drives change, so fixing it throws away the information it carries.

Most of the AI harm we perceive around is originated from a contradiction between elements belonging to the bottom layer of the triangle. The Instrument is a model optimized for engagement or efficiency. The Rules are a compliance framework written for a world without generative AI. The Division of Labor leaves nobody owing the outcome once it crosses a team boundary. The contradiction between those three elements shows that a proper design work was never done.

Lucy Suchman took this a step further in Human-Machine Reconfigurations (2007). She argued that the tool’s capability never fixed the boundary between human action and machine action. People renegotiate it moment by moment, in the actual interaction. So every product decision about what the AI does automatically, and what it hands back to a human, is a decision about that boundary. Somebody drew the line. If nobody drew it deliberately, it still got drawn by default, by whoever found it easiest to ship that way. An undrawn boundary is still a boundary, and nobody owns it.

Why this new life to activity theory

I reached for a 1980s framework instead of this year’s agentic AI vocabulary for a specific reason. Most of that vocabulary makes you settle an unresolvable question first (does the AI “really” have intentions?) before you can talk about responsibility at all. Activity Theory skips that step. It lets you say, rigorously, that a tool produces real effects, including effects nobody planned, without promoting it to a subject.

It isn’t the only framework that sidesteps intentionality. Actor-network theory, from Bruno Latour, does something similar and then goes further. It treats humans and nonhuman “actants” symmetrically, and it refuses to rank a person’s agency above an object’s inside a network of relations. That’s a coherent position, and it’s useful for plenty of questions. On responsibility, though, it makes things harder. If the algorithm, the loan officer, and the compliance policy are all nodes in the same network, the theory gives you no way to decide where accountability goes when the network produces harm.

Wrap up and a reflection

Activity Theory keeps an asymmetry on purpose. The subject position is reserved for something that can hold a motive. Instrument, Rules, Community, and Division of Labor are what the subject acts through and within. That asymmetry is what lets you trace a decision nobody designed back to the people who were supposed to design it.

So I would set aside “who’s responsible when the AI messes up”, because it’s still looking for a subject at the top of the triangle. The more useful question is which corner of the activity system absorbed the decision nobody designed, and who was supposed to own that corner. Design, in this sense, means authoring the whole triangle deliberately, before the contradiction surfaces as harm to someone.

Self-reflection: in your organization, if an AI system caused harm tomorrow, could you point to who owns the Rules, the Community, and the Division of Labor around it? Or would everyone point at the tool?

Leave a comment