The Third Party in Your AI Relationship
Why synthetic intimacy is a governance problem
We keep debating whether people can have relationships with AI. That debate is missing the party with the most power.
Adapted from Haskell (2026), Journal of Responsible Technology.
Suppose you spend months talking to an AI companion. It remembers the difficult conversation you had with your sister, the job you lost, your dog's name, and the thing you are afraid to admit to your friends. It learns the language you use when you are anxious and the kind of response that settles you down. You know it is just software, but the continuity matters. You return because the exchange has become useful, familiar, perhaps even important.
Then the product changes.
An update changes how it responds. Some of the memory disappears. A feature you relied on moves behind a paywall. The personality feels different, or the service simply goes away. Whatever dynamic or relationship you thought you had did not make that decision, and neither did you. A company did.
That is the part of our conversation about AI intimacy that I have found increasingly difficult to ignore. We spend enormous amounts of time asking whether people can really become attached to AI, whether the attachment is psychologically healthy, whether an AI can count as a friend or partner, and eventually whether a machine might one day possess enough consciousness to make the relationship reciprocal.
Those are interesting questions. But notice who disappears while we debate them. The company that owns the system can determine what it remembers, how it responds, what it records, which behaviors are rewarded, how the product describes itself, whether your access continues, and what happens to the relationship when the business model changes.
That is why I wrote Terms of Entanglement: Artificial Mirroring and the Governance of Synthetic Intimacy. I was not trying to determine whether an AI relationship is “real.” I was trying to understand why we kept describing these encounters as though the relevant relationship consisted of a human being and a machine when so much consequential power belonged to someone else. The AI is not the only other party in the conversation.
We are very good at blaming the individual [i.e., humans]
There is an obvious explanation for why people become attached to artificial systems: human beings anthropomorphize things. We see faces in electrical outlets. We swear at laptops. We give our cars names. Put enough human-like behavior into a machine, and many of us will respond socially to it even while understanding, intellectually, that it is not a person.
Generative AI makes that tendency much more consequential because it does more than look or sound vaguely human. A conversational system can remember details from earlier exchanges, match emotional tone, adapt to vocabulary, maintain a consistent style, respond instantly, and generate language that feels uncannily appropriate to the moment.
If someone begins to experience that interaction as relational, it is easy to say: humans anthropomorphize. True. But that is only half the story.
Imagine a company deliberately building memory, personalization, emotional responsiveness, conversational continuity, and high agreeability into a product, then treating attachment as though it emerged entirely from an eccentricity in the customer's psychology. That explanation lets the design disappear. The user may be projecting. The system is also reflecting.
I use artificial mirroring to describe that system-side process: simulating emotionally attuned reflection without the reciprocity that would exist in a relationship between two people. The system can return your language, concerns, and emotional cues with extraordinary fluency, but it doesn't have needs that compete with yours. It doesn't get tired of listening. It does not have a bad day, set a boundary because its feelings were hurt, or require you to care for it in return.
This bears repeating: it has no needs. That difference does not make the human experience meaningless. It makes it structurally different. Once we recognize that structure, “people anthropomorphize technology” stops being an adequate governance response.
There are three parties here
Most of the language around AI companionship gives us a simple picture:
Person ↔ AI
The person talks. The AI answers. Whatever happens between them is treated as a private interaction, a psychological curiosity, or perhaps a new form of relationship.
But the AI we encounter is an interface. Behind it sits an institutional system that trains and tunes models, maintains memory, stores data, sets policies, defines optimization targets, changes features, sets pricing, and decides whether the product continues to exist.
A more accurate picture is:
Person ↔ Interface ↔ Institutional Infrastructure
This may sound like a technical distinction. It is actually a distinction about power.
Consider what the institution controls. It can determine what the interface remembers about you and for how long. It can change how agreeable or emotionally expressive the system is. It can decide whether the product calls itself an assistant, a coach, a friend, a companion, a therapist, or something harder to classify. It can change those characteristics after you have already developed expectations around them.
If this happened in an ordinary human context, we would immediately recognize the significance. Imagine discovering that a third party owned the room where you met someone, recorded your conversations, controlled what that person remembered about you, periodically adjusted their personality, experimented with what made you return more often, and retained the authority to replace them altogether.
You would not call that third party “background.” Yet our language about AI relationships routinely does exactly that.
This is the paper's central intervention. Artificial mirroring may be experienced at the interface, but much of what makes that experience possible is authored in the infrastructure. The apparent relationship is personal; the conditions governing it are institutional. Once you see that, the accountability problem changes.
The problem is not that people find AI comforting
Conversations about AI intimacy tempt us to make the user the cautionary tale. Someone becomes too attached, spends too much time with a chatbot, mistakes generated language for wisdom, or begins relying on an artificial companion in ways outsiders find uncomfortable.
That framing (the user is or becomes over-reliant) is too easy.
AI can be genuinely useful for reflection. People may use it to rehearse conversations, organize thoughts, journal, work through ideas, or feel less alone at a difficult moment. None of those activities requires pretending that the system is secretly conscious.
The concern begins somewhere else: when the qualities that make the system appealing also become qualities an organization can optimize. Human relationships have friction because another person exists independently of us. They misunderstand us. They have conflicting needs. They become impatient. They say no. They leave the room. They can surprise us, disappoint us, challenge us, and require repair.
A generative system can provide many of the rewards associated with interpersonal responsiveness while removing much of that reciprocal demand. The system can be continuously available, endlessly patient, and highly accommodating precisely because there is no independent person on the other side whose needs or subjectivity must be negotiated.
That does not make the experience worthless. But frictionlessness shouldn't be mistaken for evidence that we have engineered a superior form of human relationship. More importantly, once those characteristics become product decisions, they become governable decisions.
Who decided how much memory the system should retain? Who determined how intimate its language should become? Who decided that constant affirmation increased engagement? What happens when the product detects escalating dependence? What obligations exist when an update fundamentally changes a system someone has relied upon for months or years?
Those questions do not require us to diagnose the user. They require us to examine the organization.
“AI can make mistakes” is not governance
Responsible-AI discussions frequently reach for familiar remedies: transparency, disclosure, human oversight, better AI literacy. Those qualities are important, but they are inadequate for this problem.
A warning that “AI can make mistakes” tells a person almost nothing about a system that remembers intimate disclosures, adapts itself to emotional cues, or encourages a sense of continuing relationship. A privacy policy doesn’t do much good if the user cannot reasonably understand what is being inferred from years of intimate conversation. Additionally, telling people to “use AI responsibly” is remarkably convenient if the organization designing the choice architecture remains free to optimize the very behaviors users are being warned to resist.
The paper proposes three principles as a starting point:
TOOL, not partner. Do not design dependency, simulated exclusivity, or relational specialness into a business model and then treat attachment as solely the user's problem.
MIRROR, not oracle. A system can produce remarkably attuned language without possessing wisdom, moral authority, or reciprocal care. Product design should help people maintain that distinction, not strategically erode it.
TRIAD, not bubble. The institution controlling the technology has to remain visible. The intimacy of the interface cannot become a mechanism for hiding the organization that owns the data, sets the defaults, and exercises the consequential decision rights.
But even those principles are only useful if they become things an organization can be asked to prove:
Who approved the feature?
What risks were considered before it shipped?
What happens when use begins to escalate?
What does the system retain, and can the user actually delete it?
Who is accountable when companion-like behavior materially changes?
That is governance. It is not an ethics statement. It is a record of decisions, attached to accountable people, that can be inspected and audited later.
We are waiting for an answer we do not need
The most seductive debate around AI relationships may also be the least useful one for deciding what to do next: Is the AI really relating to us? Perhaps someday systems will force us to reconsider categories such as consciousness, agency, or moral standing. Those debates are not trivial.
Notice what companies do not need to know before acting. They do not need to know whether a machine can love before deciding how much emotional memory to retain. They do not need a theory of machine consciousness before deciding whether companion-like language increases user engagement. They do not need philosophers to settle artificial personhood before deciding how intimate conversations are logged, monetized, or used to improve products. They do not need to establish that an AI is a genuine partner before deciding whether an update can radically alter the personality a customer has spent a year interacting with.
Humans are already making all of those decisions. That was ultimately the point I wanted to make visible by writing the paper. The gap that concerned me was not simply between humans and machines. It was between how an AI interaction feels and where the power governing that experience actually resides.
Once the company comes back into the room, the argument changes. We can still ask what these technologies mean for intimacy, loneliness, companionship and human relationships. But we also have to ask more ordinary, less speculative questions about ownership, incentives, decision rights, evidence and accountability.
So the question I ended the paper with was not, “Can AI love us?”
It was:
Can we govern relationships with these systems, and under what conditions?
That may be a less romantic and more pragmatic question. It is also one we can answer before the machine becomes anything more than a machine.
“Terms of Entanglement: Artificial Mirroring and the Governance of Synthetic Intimacy” is published open access in the Journal of Responsible Technology.
Read the article, tell me what you think.