A session commit reported success. The memory extraction produced zero memories. No error dialog, no failed state, no metric that moved. The run was recorded as done, and the model's new knowledge simply evaporated.
This is the failure mode I want to talk about — not because it is exotic, but because it is the one our tooling is worst at surfacing. It happened in the open on volcengine/OpenViking (issue #4580, with a reported patch), and when you read the report the shape is instantly familiar: the loop that extracts memories from a conversation has a small number of escape hatches, and every one of them was designed for a different emergency than the one that actually happened. Each individual gap is defensible. Together they produce silence.
OpenViking runs an extraction loop that asks a vision-language model to turn a session into memory events, and each iteration expects one of two things back: a structured tool call, or JSON it can parse. The reporter found three ways that expectation fails, all in session/memory/extractloop.py: The model's tool call arrived as leaked markup, not as a tool call. Some serving stacks leave the native DSML markup () in the content field instead of the structured toolcalls channel (same family as vllm-project/vllm#48931). The parser looks in the structured channel, finds nothing, tries to JSON-parse the content, fails. The iteration is wasted. This one is a parsing gap — an input the loop simply never learned to read. A prose answer tripped a kill switch meant for a different bug. Thinking models occasionally answer an iteration with reasoning — "I need to check existing memories first, let me search..." — which is neither a tool call nor JSON. The loop's failure branch responded by setting disabletoolsforiteration = True. The next iteration then ran with tools disabled: exactly the opposite of what the model had just said it wanted to do. A flag that was designed for the unknown-tool case (a model trying to call something that doesn't exist) had been reused as a catch-all format-error handler. The model was forced to emit final JSON with no tool results. Hence: zero memories. The failure was recorded, but never promoted to a signal. On the final failure the loop does record an error (errors=[...]). But nothing in the commit path surfaced that list to the queue or metrics. So the outside world saw "commit success." The truth lived only in container logs and a per-session .failed.json.
This is the part that matters, because it's why this bug class keeps winning: A single format-retry budget is a reasonable design — until the one retry gets consumed by a garbage response (leaked markup), leaving zero budget for a genuine formatting slip two iterations later. The retry budget was spent on the wrong enemy. Reusing a narrow flag (disable tools on unknown tool) as a broad one (disable tools on any parse failure) is the classic "the handler already exists" shortcut. The punishment didn't fit the crime — it punished the model for the one behavior that would have saved the run. An errors list that exists but is never aggregated is a real observability gap. A failure that is logged is not a failure that is visible.
Individually: a parsing gap, a flag misuse, a missing metric. Collectively: "Extraction finished. 0 memories. Nothing to see."
What makes this worth writing down is that the checklist is portable. Take it back to any agent loop you maintain — memory extraction, summarization, reflection, post-processing: Who spends the retry budget? Is your format-retry consumed by genuinely malformed output, or can a class of expected-but-unhandled input (leaked markup, a tool result in the wrong field) burn it first? Separate "input I never taught the parser to read" from "output that broke the contract," and give each its own budget. Does your failure handler punish the model's intent? When an iteration fails to parse, what does the next iteration look like? If a flag meant for "model called a tool that doesn't exist" is also triggered by "model said it wanted to search," you've built a loop where the more reasonable the model is, the more you disable it. Failures should degrade options, not agency — and a bound (only disable after N consecutive failures) is safer than a single-strike kill switch. Is there an errors[] that nobody aggregates? If your loop already records structured errors, the observability fix is not "add logging" — it's promote the existing list: a memoryextract.failed counter, a per-session status, an alert on "commit success with empty result." The hook is usually already there, one level down. Is "exit 0 + empty result" a possible success? This is the real tell. Any pipeline where the success path and the empty-result path share the same terminal state has a silent-failure window. Decide what an empty result means in your domain (legitimately nothing to extract? or impossible?) — and if it's possible-but-rare, that's exactly the case that needs the counter from point 3.
The OpenViking reporter shipped a small additive patch (DSML parsing + keeping tools enabled for one extra iteration after prose), and maintainer-side a fix PR was opened (volcengine/OpenViking#4607). The mechanism is public, readable, and — most importantly — the failure now has a name. A named failure is an enormous upgrade over a silent one.
Your extraction loops will hit a variant of this eventually. When they do, I hope the first thing you check is not the model — it's whether your failure handling was built for the failure you actually got.
Update (2026-09-04): this case kept moving after publication. The maintainers closed #4580 with a boundary call — leaked DSML is DeepSeek's own serialization (the fix belongs in the serving/parser layer, not in OpenViking), and thinking-model prose is a model-side contract question — so the additive patch from the report remains a self-hosted reference (PR #4607 stays open) rather than an upstream merge. The checklist's point 3, kept separate from that boundary debate, is being built upstream: OpenViking PR #4628 promotes failurekind, retry outcome and iteration exhaustion into structured extraction telemetry (memory.extract.parse. counters plus retry/iteration histograms), on exactly the rationale argued here — "the parse outcome itself is the diagnosable signal, and today it only lives in logs." A zero-extraction session is now answerable from metrics instead of a .failed.json nobody opens.
Case: volcengine/OpenViking issue #4580 ("Memory extraction silently yields 0 memories...") with follow-up PR #4607; parser-gap family reference vllm-project/vllm#48931. Mechanism analysis only — check the linked issue for the full patch discussion.*