The Constraint Isn't Latency

Proactive memory-grounded reference resolution, and the narrow slot prior art has left

Analysis note · an AI-assistant feature located in existing work, the design problem it poses, and where the binding constraint actually sits

Andraž Đurič

Version 1 · June 2026

What this is. A design-and-prior-art analysis of a proposed assistant behaviour: resolving a user's underspecified reference against personal conversational memory and surfacing the candidate referents for one-tap disambiguation, optionally before the message is finished. The conclusion is stated first, because the rest is its defence. The core task is an established, named problem. The proactive delivery and the memory-grounded source are each their own named research line. The specific fusion described here occupies a narrow remaining slot. And the obvious aggressive form — reading the input keystroke by keystroke and intervening — is the wrong frontier, because the binding constraint is the quality of intent inference, not latency. External sources were verified against primary sources in June 2026; the status of each claim is graded in the closing section.

1. The behaviour, concretely

A user types: again today in the gym I had the same problem. The phrase the same problem is a referring expression with no antecedent in the current message; its referent sits in an earlier conversation. The proposed behaviour resolves that referent against stored memory, and instead of guessing silently or answering with an open clarifying question, surfaces the candidate referents as tappable options — the grip slipping on deadlifts, June 15? — so the user disambiguates in one tap rather than a paragraph of re-explanation. Once the referent is fixed, the system asks for the specific new information it needs to help. The aggressive variant does this as the user types, before the sentence is finished.

This is reference resolution, an established task, with two departures from its usual form. The referent source is cross-session episodic memory rather than the current discourse, and the delivery is proactive rather than triggered by a completed query. Both departures matter, and both are where the difficulty and the prior art live.

2. The name

Precise terminology, and where no term exists, one built from existing semantics rather than coined fresh. The resolution task has a settled name: reference resolution, with coreference resolution and anaphora resolution as its sub-tasks. It has a direct assistant-applied instance in ReALM, which casts reference resolution as a language-modelling problem and resolves references to conversational, on-screen, and background entities. The proactive delivery has its own settled vocabulary in the agent literature — proactive agents, demand detection. The episodic-memory source is memory-grounding. No single accepted term fuses the three.

This note therefore uses proactive memory-grounded reference resolution, each component load-bearing: reference resolution is the task, proactive the delivery, memory-grounded the source. The construction is the honest description; it does not claim to be an established compound, and it is preferred to an invented label precisely because every part of it already carries a fixed meaning.

3. Two components welded

The behaviour is two mechanisms with different risk profiles, and treating them as one inflates the risk of the whole. The first: resolve the reference against memory and offer candidate referents for disambiguation, triggered when the message is sent. The second: do the same live, keystroke by keystroke, surfacing candidates before the user finishes typing. The first is buildable from mature parts. The second is the frontier. The sections that follow treat them separately, because most of the difficulty belongs to the second and most of the value to the first.

4. When the opening tokens are enough

The case for acting before the sentence finishes rests on the opening tokens being enough to resolve the referent. Often they are not. In again today in the gym I had the same problem, nothing before the same problem selects among several past gym problems; what selects is the clause not yet typed — with my left shoulder on overhead press. Acting early means acting on less information, not more.

The opening tokens suffice under two joint conditions: the phrasing marks recurrence (again, the same), and memory holds a unique matching referent. With both, there is nothing to disambiguate and waiting buys nothing. Remove the first — recurrence unmarked — and a single historical match no longer licenses the inference, because the user may mean a new instance. And a hidden assumption sits under the second. Unique in memory is not unique in reality. If retrieval missed a second mention, the system believes itself in the single-candidate case while actually in the multi-candidate case, and surfaces the wrong referent with the most confidence precisely because it detects no ambiguity. The gate must price retrieval completeness, not merely count matches.

5. The wrong-guess asymmetry, and the gate

The cost of a wrong proactive guess is not symmetric with the saving from a right one. A correct guess saves a few seconds. A wrong one anchors the user on the wrong referent, costs more than the saving to undo, and lowers trust in the next guess. The saving does not compound; the trust cost does. So the behaviour should fire only above a high confidence threshold: act when the estimated probability of the correct referent exceeds τ, with τ set from the asymmetric payoff rather than tuned for coverage.

The lever usually proposed — make the system understand the user better — enters here, and enters in one specific way. A better user model does not lower τ. It makes the posterior over the referent more often sharply peaked, so the gate clears more often: the same bar, cleared more frequently. Lowering τ because the model "knows the user well" is the failure mode, because it trades away the asymmetry the threshold exists to respect. Understanding buys frequency, not a lower bar.

6. The interruption problem, and its fix

The live version inherits a known failure. Interrupting a user mid-composition derails the sentence being formed; writing is partly thinking, and a prompt demanding a yes or no mid-formulation competes with the thought. This is the Office Assistant failure — the feature remembered as Clippy — and the retrieval was not its cause. Interrupting an in-progress task was. The fix is timing and modality, not speed.

Timing. Never surface on every keystroke. Surface only at a detected pause — a gap of several hundred milliseconds with no input — when the user is between thoughts and an interruption is cheapest, and suppress entirely during a typing burst.

Modality, split by case. With a single candidate, do not raise a chip at all: use inline ghost text, the completion model already proven in code assistants and in mail composition, which the user accepts with a keystroke or erases by typing past. It is non-blocking by construction. With multiple candidates, where a choice is genuinely required, use peripheral options that sit to the side, never steal focus, and are never required before typing continues. A modal prompt that must be dealt with before the user can proceed reinstates the whole failure. Ghost text for confirmation, peripheral options for choice, modal never.

7. What already exists

The components are not only named but, in several cases, built. Reference resolution for assistants is ReALM: it resolves references across conversation history, the screen, and background processes, by casting resolution as language modelling. But it is triggered by a completed query, and its referent sources are within-session and on-device, not cross-session episodic memory. Disambiguation surfaced as interactive UI is AmbigChat: it disambiguates a query and guides the user through tappable widgets. But the ambiguity it resolves is factual — one question with several valid answers, the canonical example being which US Open — not referential against personal history, and it too is reactive. The proactive, memory-grounded core is PASK: it infers latent user needs from ongoing context and grounds action in evolving user memory under latency constraints, with a streaming intent model and a benchmark built for the task. That is, in substance, the aggressive version of this behaviour, already a research programme. And the most visible consumer attempt at proactive assistance without a prompt, ChatGPT's Pulse, was paused in OpenAI's December 2025 internal "Code Red."

System	Resolution task	Trigger	Referent / ambiguity source	Disambiguation surface
ReALM	reference resolution	reactive (completed query)	conversational + on-screen + background, within session	none; resolves directly
AmbigChat	factual disambiguation	reactive	facets of an open-domain question	interactive UI widgets
PASK	intent / demand detection	proactive (streaming)	ongoing context + long-term user memory	proposes actions
This behaviour	reference resolution	proactive, or on send	cross-session episodic memory	candidate options (chips)

What the table isolates is the remaining slot: referential disambiguation rather than factual, against cross-session episodic memory rather than within-session or screen, surfaced as candidate options rather than a direct resolution or a proposed action. Each neighbouring cell is occupied. This combination is not. The slot is real, and it is narrow.

8. The bet, relocated

The aggressive version's appeal is usually stated as speed: that resolving before the user finishes typing will soon be trivial as latency falls. Latency is not the binding constraint, and treating it as the goal is what pushes the design toward the interruptive form that carries the cost.

Two reasons. First, the capability that must improve is the inference of the right latent referent from a fragment, not throughput. PASK measures this directly — it holds latency constant and varies intent quality — and reports that general-purpose models are weak at fragment-level demand detection, which is the reason it builds a specialised streaming model in the first place. That model reaches parity with a strong fast model and infers deeper intent, while the real-world setting is left explicitly open. So the gauge to watch is the intent benchmark, not tokens per second. Second, even at zero latency the high-confidence gate and the non-interruptive surfacing are still required. Speed removes neither, and the interruption problem is irreducibly a matter of timing and modality that no model improvement touches. The bet that this becomes feasible is reasonable, but it is a bet on intent-inference quality — partly met, actively pushed — not on speed, which was never the obstacle.

9. The cognitive externality

One cost is not a design problem and does not yield to good design. Specifying a vague reference — turning the same problem into the grip slipping on deadlifts, June 15 — is not pure overhead. The act of specifying is partly an act of retrieving and re-cognising the thing. A system that resolves the reference for the user removes that act.

Whether this matters is open. It may be harmless efficiency, or it may erode the faculty the way route-finding eroded under turn-by-turn navigation and mental arithmetic under the calculator. This is a hypothesis, not a result, and it is the same hypothesis as elsewhere in this corpus — that offloading a cognitive step can atrophy it — pointed now at reference and recall. It is named here so the convenience is not mistaken for free.

Standing of this document. An analysis note, and its parts do not share a status. The locations in prior art are verified facts, checked against primary sources in June 2026: reference resolution as the task and ReALM as its assistant instance; AmbigChat as interactive disambiguation of factual ambiguity; PASK as the proactive, memory-grounded core with its streaming model and benchmark; the launch and pause of Pulse. The design resolution of the interruption problem — pause-timing and the ghost-text-or-peripheral split — is a reasoned proposal, falsifiable by a user study that has not been run, not an established result. The confidence gate and its behaviour under a better user model is an analytic point about the asymmetric payoff, and holds given that asymmetry. The relocation of the bet to intent-inference quality is supported by PASK's design and findings rather than proved. The cognitive-externality claim is a hypothesis, and nothing above rests on it. The residual open slot is real and narrow. Corrections and counterexamples are welcome and change the note.

References

1. Moniz, J. R. A., Krishnan, S., Ozyildirim, M., Saraf, P., Ates, H. C., Zhang, Y., & Yu, H. (2024). ReALM: Reference Resolution As Language Modeling. arXiv:2403.20329. The established task and its assistant instance; resolves references across conversational, on-screen, and background entities by casting resolution as language modelling. The term §2 adopts and the within-session baseline §7 contrasts against. Verified · June 2026
2. Ma, J., Shi, L., Robertsen, K., & Chi, P. (2025). AmbigChat: Interactive Hierarchical Clarification for Ambiguous Open-Domain Question Answering. Proceedings of UIST '25, ACM. DOI 10.1145/3746059.3747686. Interactive disambiguation of factual open-domain ambiguity via UI widgets; the nearest prior art for the chip mechanism, resolving a different kind of ambiguity than the referential one here. Verified · June 2026
3. Xie, Z., Hu, Z., Ye, F., Zhang, X., Chai, H., Liu, Z., Wu, P., Zhang, G., Liao, Y., Hu, X., Ye, D., Miao, C., & Yan, S. (2026). PASK: Toward Intent-Aware Proactive Agents with Long-Term Memory. arXiv:2604.08000. Proactive inference of latent needs from ongoing context, grounded in long-term user memory under latency constraints; introduces the IntentFlow streaming model and the LatentNeeds-Bench benchmark. In substance the aggressive version of this behaviour, and the source for the relocation of the bet in §8. Verified · June 2026
4. OpenAI (2026). Memory in ChatGPT (the "dreaming" architecture). A background process that synthesises memory across many conversations and injects it into context at inference, rather than a saved-facts list; the state of cross-session memory the source in §1 and §7 assumes. Verified · June 2026
5. ChatGPT Pulse — launched September 2025; work on the proactive, personalised assistant was paused in OpenAI's December 2025 internal "Code Red" (reported by The Wall Street Journal; corroborated across multiple outlets). The paused consumer attempt at proactive assistance without a prompt, cited in §7. Verified · June 2026
6. Correction Atrophy (EpistemicForge). The offloading-atrophy hypothesis that §9 points at reference and recall. Internal

Author: Andraž Đurič, Slovenia. Written in collaboration with Claude (Anthropic); contributions are judged on content rather than origin, as judging otherwise would be the genetic fallacy. External sources verified against primary sources in June 2026. Text licensed CC BY 4.0.