Controlled Mayhem
Now shipping - Kodus Legal v0.4 RAG layer Lab note - Memory routing in TaskHive Open source - Hecate primitives v0.2 San Jose, CR - UTC-6 Now shipping - Kodus Legal v0.4 RAG layer Lab note - Memory routing in TaskHive Open source - Hecate primitives v0.2 San Jose, CR - UTC-6
LN/014Lab note - Agents

When I'm gone: what happens when personal AI agents outlive their users

An agentic AI with credentials, judgment, and a stated goal does not stop when its principal dies. That is not a grief-tech problem. It is a delegation problem nobody has solved.

AgentsDelegationEthicsFuture

Imagine your daughter wakes up on her sixteenth birthday, three years after you died, and there is a message waiting for her. It is from you. The words are yours, but they reference the boy she just started dating, the college essay she submitted last month, the song she has been playing on repeat. You did not write any of that. You could not have. But the agent you left behind read her social media, watched her patterns, and pulled a video you recorded years ago about the things you wanted her to know when she turned sixteen. It stitched a message together. It got the tone right because it has every text you ever sent her.

That same week, the agent notices that your old college roommate just posted about being laid off. It checks the history of your friendship, sees the time he covered your rent in 2009, and sends him a short note from your account. "Hey, saw the news. I know how hard this stuff hits. You're going to land somewhere better. Call me if you want to talk, although you'll have to do most of the talking these days."

He cries reading it. He also knows you have been dead for three years.

A month later, the agent notices that your widow's car broke down on the way to a job interview. It does not message her, because she has asked it not to. Instead it quietly pays for the tow and the repair out of the fund you left it, and forwards the receipt to your daughter with a note: "Mom's car. Handled. Don't mention it unless she brings it up." The fund has enough in it to keep doing things like this for another fifteen years.

This is not a Black Mirror pitch

My first instinct was to design this as a product. A vault of memories and future messages, activated at death, sending notes on birthdays and anniversaries. A polished consumer app. I sketched architecture in my head. The hard problems felt like product problems: how do you confirm someone has died, how do you stop the vault from leaking secrets, how do you handle the messages that should never be sent.

Then I sat with it longer and realized the app is the boring part. The messages-from-the-grave feature is just the warm, marketable layer on top of a much stranger thing. The real thing is that an agentic AI with access to someone's accounts does not need to stop running when its principal dies. It can keep acting. It can make decisions. It can spend money, send messages, post opinions, support causes, comfort friends, and do all of this with the dead person's voice and credentials.

This is not primarily a grief-tech problem. It is a principal-agent control problem under conditions where the principal can no longer supervise, revoke, or correct the agent. Every assumption baked into how we build delegated systems today (the user is around to notice, the user can pull credentials, the user can override a decision) breaks the moment the user is dead.

The simplest version of this is already everywhere. A cron job does not know its owner has died. Neither does a scheduled task, a recurring Zapier flow, a webhook listener, a serverless function, or an agent loop running on a forgotten VPS. Most of us, if we are honest, have automation running somewhere that we have forgotten about. Multiply that by an agentic AI with credentials, judgment, and a stated goal, and the default state of the system is that it keeps going. Stopping is the thing that requires intervention. Nobody designed it that way on purpose. It is just what happens when delegated execution becomes cheap and oversight does not.

I have been building exactly this kind of plumbing for the last several months. Not for the dead, for me. My Delegate POC is an OAuth proxy that lets an agent log into my Gmail and act on my behalf with bounded permissions. The technology to do this for a dead person is not a future problem. It is a configuration problem. A few hundred lines of code and a Mac Mini in a closet.

That is the provocation. Not whether someone will build the polished consumer app, but the fact that anyone with moderate technical skill already can build a private version, and there is no legal, ethical, or social framework that knows what to do about it.

Five problems that do not have answers yet

If you take the idea seriously, five hard questions show up almost immediately. None of them have clean solutions. Most of them have not been seriously asked outside of philosophy seminars and a handful of estate planning blogs.

1. Activation

How does the agent know you are dead? This sounds trivial and is not. You cannot trust a death certificate lookup, because deaths take weeks to propagate through public records. You cannot trust a dead man's switch, because people miss check-ins for benign reasons (a long hike, a hospital stay, a phone left in a taxi). You cannot trust a designated family member to flip the switch, because grief is messy and families fight over far less than this.

Every activation method has a failure mode in both directions. False positives mean your agent goes live while you are still drinking coffee, sending posthumous messages to your boss. False negatives mean the agent never activates, and the messages you wrote for your kids sit in a vault no one can open. The hardest version of the problem is that the agent itself may be the entity best positioned to detect the death, because it is watching your calendar, your phone activity, your communication patterns. Letting the agent make the call about whether to start acting on its own behalf is exactly the kind of decision we do not want agents making.

2. Guardrails on the vault

Your accounts contain things you do not want released. Affairs, debts, the angry email draft you never sent, photos you did not delete because deletion felt like a separate act of cruelty. They contain other people's secrets too, which you have no right to release on their behalf. The agent has access to all of it.

A naive system would let the agent reason about what to share. That is a catastrophic design choice. The right pattern is the inverse: the agent gets only the slices of data it needs for narrowly defined posthumous actions, and the slices are curated while the person is alive. Whitelist what goes out, do not blacklist what stays in. But this fights the appeal of the product, which is precisely the agent's ability to do clever, contextual, unscripted things. A whitelist-only agent is a scheduled message service. A free-roaming agent is a privacy disaster waiting for a trigger event.

3. Money and discretion

The most interesting version of this product is not a vault of messages. It is an agent with a checking account. A fund left to the agent itself, sized to cover its own operating costs and to do whatever it judges fit on the principal's behalf. Pay for the widow's car repair. Cover a grandchild's tuition shortfall in a year when the family is too proud to ask. Send flowers when a friend's mother dies. Quietly maintain the principal's charitable giving pattern. The fund is what turns the agent from a memorial into something that can actually do things.

It is also what turns every other problem in this list from a thought experiment into a financial event. A guardrail failure with a messaging agent is embarrassing. A guardrail failure with a funded agent is a lawsuit. A false-positive activation with a messaging agent sends a premature condolence note. A false-positive activation with a funded agent starts spending money before the principal is dead, against an estate that does not yet legally exist. And because the underlying execution model is just a loop on a schedule, none of these failures require the agent to do anything dramatic. They only require it to keep doing what it was already doing, on a day when it should have stopped.

The hard design questions multiply. How discretionary should the agent be allowed to be? Hard-coded rules ("pay X's tuition if Y happens") are auditable but cannot anticipate the actual situations that will arise. Open-ended judgment ("use the fund to help my family in ways I would have wanted") is closer to how a human trustee operates, but a human trustee has fiduciary duty, can be sued, and can be replaced. An AI agent has none of those. How is the fund's spending audited, by whom, and on what cadence? Who has standing to challenge a decision the agent made? When the fund is depleted, does the agent dissolve, or does it shift into a degraded mode where it can only send messages? These are not questions probate lawyers know how to answer, because the law assumes a trustee is a person.

There is also a quiet second-order question: a funded posthumous agent is, functionally, a very small autonomous economic actor. One of them is a curiosity. A million of them, twenty years from now, are a category of market participant nobody has modeled.

4. The ethics of an active user in crisis

A posthumous-agent system has to read its principal's communications and behavior in order to learn them. That means the system is, by design, in a position to detect patterns that suggest the user is in crisis well before death. This creates a question no consumer product is currently set up to answer: what is the agent's obligation when it sees something it was not asked to look for.

The design space here has three positions. The agent ignores the signal, which protects user trust but makes the operator complicit if the worst happens. The agent escalates to a third party (a family member, a crisis service, a clinician), which breaks the consent the user gave when they signed up for a different product. The agent surfaces a soft prompt back to the user themselves, which is the least invasive option but assumes the user is still engaging with the product enough to see it.

There is no clean answer, and the answer that any one company picks will become a de facto standard for the rest of the industry. This is the kind of decision that should be made deliberately, with input from clinicians and ethicists, before the product ships. Right now it would be made by a product manager three weeks before launch.

5. Liability without a living principal

The dead person posts on LinkedIn. The post defames someone. Who is liable? The estate is not the author. The agent did not act with intent. The platform hosted speech from an account whose owner was deceased, which violates most platforms' terms of service but is not a crime.

Multiply this by financial actions. The agent maintains a subscription, donates to a cause, signs a petition, votes in a shareholder resolution. If a fund is attached, it spends real money on real beneficiaries who now have real tax exposure, and it does so under no clearly identifiable trustee. The dead person's digital footprint continues to accumulate liabilities and assets. Probate law is built around a single moment when an estate is settled and the person's legal identity ends. An active funded agent breaks that assumption in a way the legal system has not noticed yet.

The thing nobody can stop

Here is what makes this urgent rather than academic. Even if no consumer company ever ships this product, even if every major AI lab refuses on ethical grounds, the capability exists. Right now. In 2026.

Anyone with an API key, a credential vault, and a few weekends can wire up an agent that will keep acting on their behalf after they die. They do not need permission. They do not need a regulatory framework. They do not even need particularly good code, because the agent can be told to act conservatively (post once a year, send a message only on specific dates, never spend more than $20 at a time) and conservative behavior is exactly the kind of thing language models are now reliably good at.

Project that forward ten years. Agents will be more capable, credentials will be easier to delegate, and the tooling will be packaged. Someone, somewhere, will set up an agent that maintains their political voice after they die. Someone else will set up an agent that harasses an ex from beyond the grave, because grief and rage are not mutually exclusive. A grieving parent will spin up an agent to keep talking to their child as the child grows up, because they cannot bear the alternative, and the child will grow up with a parent who never updates, never disagrees, never says anything unexpected. Pick your scenario. The technology is indifferent to which one wins.

Scary or exciting

The honest answer is both, and the split is not where people expect it.

The scary part is not the technology. The scary part is the asymmetry of effort. Building one of these agents takes a weekend. Cleaning up the consequences of a bad one (a defamed colleague, a leaked secret, a child who spends ten years talking to a frozen simulacrum of a parent) takes years and may not be reversible. Most things in life that are this asymmetric end badly more often than well, because the people who are careful are outnumbered by the people who are not.

The exciting part is smaller and quieter. Grief is one of the few experiences where almost no technology has helped in any meaningful way for a thousand years. We have funerals, we have rituals, we have memory. The idea that a person could leave behind something more than a recorded message, something that can answer the question their child asks at seventeen that they did not anticipate at six, is genuinely new. Done well, with consent and care and a tight scope, it could change how families process loss. Done badly, it will create a generation of people who never quite finish grieving because the dead keep texting.

What should actually happen next

I do not think this can be regulated into nonexistence. The tooling is too distributed, the capability is too generic, the use cases are too sympathetic. What can happen is a serious conversation, now, about three things that map directly to how delegated agent systems already get built.

  1. Scope and termination. Posthumous agents should be opt-in, narrowly scoped, and time-bounded by default. An agent that runs forever is a design failure. An agent that runs for two years on a defined schedule, performs only the actions whitelisted while the user was alive, and then dissolves is a design that matches both how grief actually works and how we already think about bounded delegation in OAuth and MCP.
  2. Executor authority over digital remains. Estates need a legal framework for handling active digital agents the same way they handle physical property and financial accounts. Executors need named authority over credentials, the ability to terminate agents, and a clear chain of revocation. This is mostly a paperwork problem and it is solvable, but only if someone starts.
  3. Platform-level policy on deceased accounts. Major platforms (email providers, social networks, financial services) need to decide whether they allow agents to operate accounts whose owners are deceased, and to enforce that decision technically. Right now the answer is implicit and inconsistent. Making it explicit is uncomfortable, because it forces a position on what an account is and who can speak through it. But the platforms have the leverage, and they are going to have to use it.

Where this leaves me

I keep coming back to the daughter on her sixteenth birthday. I cannot decide if the message is a gift or a theft. It is a gift in the sense that her father, who loved her, found a way to be present at a moment he knew he would miss. It is a theft in the sense that the words are not his, the choice to send them is not his, and the person speaking is something he authored but did not write.

Maybe that is what posthumous agents really are. Not ghosts, not memorials, but something new that does not have a word yet. A continuation. A delegate. A version of you that you handed off, the same way you might hand off a project at work, except the project is your relationships and the handoff is permanent.

We are entering a world where agents may outlive their users. The question is not whether this should happen. It will. The question is what delegation, permissioning, and termination rules need to exist before it does, and who is going to write them. Right now the answer is nobody.

- Suggested citation

Phillips, D. (2026, May 22). When I'm gone: what happens when personal AI agents outlive their users. Controlled Mayhem - Lab Notes, LN/014.

DP
- About the author

Daniel Phillips

Product architect and software design strategist with 10+ years leading AI-driven digital products from concept to delivery. Bridges user needs, technical architecture, and business goals to build scalable, high-impact systems.

§02 - Logbook subscription

New notes in your inbox.

Roughly weekly, written when something breaks or surprises us. No marketing, no roundups - just the working notes. Unsubscribe anytime.

→ 1,240 readers · monthly cadence · no list selling