A machine can do something without being someone. That sentence sounds obvious until software begins to select applicants, route ambulances, negotiate prices, moderate speech, or operate a vehicle with little moment-to-moment direction. We call such systems autonomous, and the word is useful if it means that a system can pursue a task without a fresh instruction for every step. It becomes dangerous when autonomy is treated as a conclusion about the system's moral status.
Agency has several meanings. A system can be a causal agent: its operation changes what happens in the world. It can have delegated authority: people and institutions can authorize it to act within a defined scope. It can display functional agency: it can select means in response to inputs rather than merely replaying one fixed sequence. None of these, by themselves, makes it a moral agent. Moral agency belongs to a subject capable of understanding a good, freely choosing in light of that understanding, and being answerable for the choice.
Catholic moral reflection gives us reasons to preserve these distinctions. The human person is not merely an efficient source of outputs, but a creature with intellect, freedom, conscience, and a vocation to love. The Catechism connects moral responsibility with the voluntariness of human acts, knowledge, freedom, and circumstances (CCC 1731–1736). A model may produce an impressive recommendation. It does not follow that the model knows what it is doing in the moral sense, wants an end, or can repent when its recommendation injures a neighbor.
The question here is primarily ontological and architectural: what kind of agency does the system possess, and what kind does it lack? Questions of an individual's culpability and an institution's moral distance require separate analysis. Keeping those questions distinct prevents a description of machine behavior from becoming either an accusation against one engineer or an excuse for an entire institution.
Several Kinds of Agency
The first distinction is between causation and responsibility. A thermostat causes a furnace to turn on; a database causes a clerk to see one record before another; a navigation system causes a driver to take a particular route. These causal descriptions are not empty. The design of a system can make a real difference to a person's prospects. But causal contribution is not the same as moral accountability. A falling stone can break a window, while the person who threw it may be responsible for the damage. A system can be part of the chain through which harm occurs without being the subject to whom blame, praise, penance, or forgiveness is owed.
Delegated authority is a second distinction. When a hospital permits software to prioritize alerts, or a company permits a model to filter résumés, the institution has assigned the system a role. The system exercises authority in a practical sense because its output changes what a human can do next. Yet the authority remains derivative. It comes from the people who define the purpose, provide the data, deploy the model, and accept its recommendations. Delegation can distribute tasks; it cannot transfer away the duty to govern the task.
Functional agency is the source of much confusion. A reinforcement learning system can evaluate options and update a policy. A language model can compose a response that was not stored as a sentence in its training material. An agent can call tools, preserve a plan, and react to a changing environment. These capacities justify careful safety analysis. They do not, on their own, establish interiority, understanding, conscience, or a free act. Describing a system as autonomous can tell us how to engineer supervision. It cannot settle whether the system is a person.
Intention is not optimization
An objective function is not an intention. It specifies what a system will tend to optimize under a model and a set of constraints. Intention in a moral act is directed by a subject who understands an end as a reason for acting. A fraud detector can maximize a score for likely fraud; it does not desire justice or harbor suspicion. A delivery robot can minimize travel time; it does not value a patient's comfort. The vocabulary of goals is useful shorthand, but it should not smuggle a moral subject into a piece of software.
Why Automation Cannot Launder Accountability
Automation often creates an illusion of disappearance. Before deployment, there are visible choices: a manager approves a use case, an engineer selects a target, a procurement team accepts a vendor, and a regulator sets a standard. After deployment, the same choices appear as a score, a queue position, or an automated action. The institution can then say that the system decided. But an output does not become ownerless because its path is complex.
Responsibility should be traced across the whole sociotechnical arrangement. The designer may be responsible for a foreseeable defect in the model or interface. The deployer may be responsible for using a tool outside its validated context. The operator may be responsible for ignoring an obvious warning or treating a recommendation as unquestionable. The executive or board may be responsible for a business decision that removed review in order to reduce cost. An institution may bear responsibility for a policy that made predictable harms invisible or made correction practically impossible.
Hard Cases at the Boundary
Consider an autonomous vehicle that strikes a pedestrian after its perception system misclassifies a shadow. No single person may have intended the death. The engineer did not choose this pedestrian, the safety driver may have had no time to react, and the model may have behaved outside its test distribution. Still, questions remain. Were the sensors validated for the environment? Was the deployment area broader than the evidence justified? Were warnings communicated honestly? Did the operator have a real capacity to intervene? Did the company monitor near misses and act on them?
A second case is an automated benefits system that incorrectly flags a family for investigation. The error may arise from an old database, an innocent correlation, and a threshold selected by a contractor. There may be no malicious intent. Yet the family can lose income, incur debt, and experience humiliation while trying to correct the record. Here negligence may lie not in the prediction itself but in deployment without timely notice, human review, accessible appeal, or a pause on severe consequences. A system can be statistically respectable and administratively unjust.
A third case concerns a model that generates harmful advice despite safeguards. If a user deliberately bypassed warnings and induced the output, the user's responsibility matters. So does the provider's responsibility for foreseeable misuse, misleading claims, inadequate testing, and a product design that rewarded confident fabrication. These responsibilities can coexist. One actor's agency does not erase another's contribution.
Such cases also show why outcome alone cannot settle culpability. Catholic moral reasoning distinguishes the object of an act, the intention, and the circumstances (CCC 1750–1756). The same technical mechanism may serve a life-saving purpose in one setting and an exploitative purpose in another. A good result does not excuse an intrinsically disordered act, and a bad result does not prove that every participant acted culpably. These principles do not supply a software test suite. They provide a vocabulary for resisting both simplistic blame and convenient exoneration.
The Strongest Objection
The strongest objection is that advanced systems may eventually become too opaque and adaptive for ordinary human attribution. If no one can predict the precise output, the objection runs, it is unfair to blame a designer for that output. And if no human caused the particular result in the ordinary sense, perhaps the only honest answer is that nobody is responsible. Holding people accountable for emergent behavior could discourage useful innovation and punish them for failures they took reasonable care to prevent.
This objection identifies a real limit. Responsibility is not magic omniscience. A person is not culpable merely because a bad result occurred somewhere in a system they touched. Reasonable precautions, honest uncertainty, proportionate deployment, and the possibility of intervention matter. We should not call every accident a sin or turn every model error into proof of malice.
But unpredictability is not the same as lack of responsibility. We routinely assess whether a risk was foreseeable in kind, even when its exact instance was not predictable. A ship owner need not predict the exact wave that will cause a failure to be responsible for ignoring seaworthiness. Likewise, an institution deploying a high-impact model can be responsible for accepting a class of risks it did not adequately test, communicate, limit, or insure. The relevant question is not always, “Who intended this exact outcome?” It is also, “Who created the conditions in which this foreseeable danger was imposed on these people, and who had the authority to reduce it?”
There is also a practical distinction between legal liability and moral culpability. A corporation may owe compensation even when no individual acted with personal malice. A regulator may require a recall without claiming that every engineer was blameworthy. Moral inquiry should neither replace law nor hide behind it. Institutions can have duties of care, repair, and truthfulness that are broader than the question of which individual deserves punishment.
When Opacity Sets a Boundary
The objection about emergent behavior has a practical consequence: opacity itself can make deployment impermissible at a given level of risk. Before release, a safety case should state the operating envelope, the evidence supporting it, the failures that require a stop, the time available for intervention, and how near misses will be investigated. If no qualified person can tell when the system is outside its validated conditions, or no one can intervene before an irreversible harm, “human ownership” is only a name.
This boundary should not be confused with a claim that every novel failure makes a person blameworthy. Compensation, insurance, recall, or strict enterprise liability can be appropriate even when moral blame is absent; negligence asks whether reasonable precautions were omitted, while moral culpability asks what a person knew, intended, or freely accepted. A worker should not absorb liability for a decision they had no authority to change. Conversely, an organization should not call a system unforeseeable when it declined to define limits, monitor near misses, or fund a recovery path.
Designing for Agency and Answerability
A responsible design begins by naming the decision and its human owner. Teams should document what the system is allowed to do, who authorized that scope, what evidence supports it, and who can stop it. “Human in the loop” is not enough if the human is rushed, lacks information, or can only approve what the system has already decided. Meaningful oversight requires authority, time, competence, and a realistic path to disagree.
Systems should separate recommendation from execution wherever the stakes demand it. They should record inputs, versions, interventions, uncertainty, and the reason for a consequential action in a form that can be examined later. They should use graduated permissions rather than one undifferentiated grant of autonomy. High-impact actions need confirmation, reversibility where possible, independent monitoring, and a humane appeal process. A person affected by a system should not have to defeat an aura of technical infallibility before being heard.
Governance should also ask who bears the cost of error. If a model is cheap for an institution because every false positive becomes a claimant's unpaid labor, the apparent efficiency is a transfer of burden, not a neutral improvement. Catholic social teaching's concern for the common good and the dignity of work supports examining these distributions. Subsidiarity does not mean leaving a vulnerable person alone with an automated system; it means placing decisions at the level capable of serving the person while coordinating higher levels when needed.
Finally, organizations need a culture in which stopping a system is recognized as responsible action rather than failure. Incident reports should identify design and governance conditions, not merely blame an operator who was placed in an impossible position. Procurement should demand evidence and remedies, not only performance benchmarks. Public claims should distinguish what a model can do from what the institution is willing to take responsibility for doing with it.
Agency in Its Proper Place
Calling a machine an agent can be technically illuminating. It can remind us that the system acts through an environment, changes options, and may require controls different from those used for passive software. The term becomes morally confused when it is used to make a machine the final subject of praise or blame. Causal agency, delegated authority, and functional autonomy are real features of systems; personhood and moral agency are different claims.
The absence of machine personhood does not make the machine harmless. It makes the human and institutional duties around it more visible. We must ask who chose the end, who supplied the authority, who knew the risk, who could intervene, who profited, who was exposed, and who can repair the injury. Automation may distribute an act across code, data, policy, and operators, but distribution is not disappearance.
A humane technological order will let machines perform bounded work while keeping moral judgment answerable to persons. It will not demand that every system imitate a human before it deserves scrutiny, nor grant every system the dignity and standing owed to a human being. The central discipline is simple: let software be as capable as it can be, but never let its capability become an excuse for human beings to stop answering for what they have made possible.
Sources and Qualifications
The Catholic framework draws on the Catechism §§1700, 1731–1736, and 1750–1756 for dignity, freedom, responsibility, and moral acts, withGaudium et Spes 26 and Caritas in Veritate 70 on the common good and responsible human direction of technology. These texts do not offer a taxonomy of machine agency or a software specification. Their concepts are applied by analogy, distinguishing moral theology, civil liability, and engineering practice.
Source Notes
The Church’s treatment of freedom, human acts, intention, circumstances, responsibility, and imputability.
The council’s account of personal dignity and the common good as conditions in which persons and groups can flourish.
A social encyclical on technological development, responsibility, integral human development, and the need for moral formation.
A voluntary framework for governing, mapping, measuring, and managing risks created by AI systems and their organizational use.
Guidance emphasizing that responsibility and accountability for AI outcomes must remain attributable to human and institutional actors.
Editorial revision note
This essay completed one self-editing, cross-essay consistency, and strongest-criticism pass. Its controlling thesis was preserved while objections, qualifications, evidence limits, and practical consequences were strengthened.