A Vendor Retired Our Model Without Notice —
We Found Out 4 Days Later

Background

One of the core pipelines in a data platform I run uses an LLM to parse legal documents (case notices, rulings and the like) into structured data — extracting case-information fields out of non-standard formats for downstream search and business systems.

The parsing engine ran on the Moonshot (Kimi) v1 series API under a commercial contract, and had been stable for about two years. The pipeline also carried a token-length tiering mechanism: short documents went to the 8K tier, longer ones escalated to 32K / 128K, and a truncated response triggered an automatic retry on a higher tier.

The Incident

On 2026-08-31, Moonshot retired the v1 series. What matters is how: a post on official channels — no email, no formal notice under the contract. For a system that depended on it, the result was that document parsing began failing silently — and it took us 4 days of routine monitoring to notice, by which point more than 2,000 documents had failed.

We responded on two tracks at once: the business side opened a conversation with the vendor about the contract, and engineering rushed to adapt to the replacement model, kimi-k2.6.

Model Selection: Three Candidates, One Elimination Question

When v1 was retired, Moonshot had three possible replacements online:

We went with kimi-k2.6. The criterion deserves its own line: for structured-extraction tasks, choose the model by measured accuracy on your own workload, not by its rank on a leaderboard — don't pay a premium for capability you won't use.

Migration: Not a One-Line Model Swap

The expectation going in was "change a model string." The actual commit touched 4 files and 93 lines, and hit three layers of problems that had nothing to do with each other:

Layer 1: parameter semantics changed. max_tokens became max_completion_tokens; chain-of-thought output had to be explicitly disabled (thinking: disabled) and tool selection turned off (to avoid burning tokens on nothing); and whether temperature / top_p still meant anything on the new model had to be re-evaluated from scratch.

Layer 2: output behavior drifted. This was the subtle one. Even with response_format: json_object explicitly set, kimi-k2.6 still returned content wrapped in Markdown code fences, sometimes with explanatory prose mixed in. The old code parsed the raw string directly, so it threw reliably on the new model.

Layer 3: resource tiers had to be rebuilt. The v1-era logic that dynamically picked between the 8K / 32K / 128K tiers was entirely invalid in a world with a single target model. The context budget was recalculated and maxTokens went from 50000 to 32768 — keep the old value and the server rejects the request outright.

Hardening the Parser: Never Trust LLM Output Format

For layer 2, we added a guard at the parser's entry point:

1. Strip leading and trailing Markdown code-fence markers;
2. If the result still doesn't start with a brace or bracket, fall back to slicing from the first brace (or bracket) to the last;
3. Only persist on a successful parse — and persist the re-serialized, normalized JSON, so downstream always receives clean data.

Data Repair

The 2,000+ failed documents were repaired through manual triage and batch re-extraction. Thanks to the parser hardening upstream of persistence, the re-run wrote data in exactly the same format as freshly parsed data — downstream never noticed.

Results

Model replacement, testing and rollout were all completed the same day we found the problem. Three outcomes:

• Parse success rate went up, not down — the new model is genuinely stronger than v1;
• Cost is essentially flat — the v1 commercial contract included data-protection terms and was priced higher; k2.6 carries a higher unit price, but against the old contract total it comes out even;
• Data compliance became a new follow-up item — we now run on a public API whose data-usage terms differ from the commercial contract. Whether to re-sign a commercial contract is under discussion with the vendor's business team.

Lessons

1. Vendor lock-in is a business risk and a technical risk. Technically: keep model calls behind a single service layer — the reason this took only 4 files is that the line had stayed clean. Commercially: the contract needs explicit model-lifecycle notification obligations and transition terms. The lesson is "announced, therefore offline."
2. Detection costs more than repair. Fixing, testing, and launching took only two hours. Noticing took four days. Any LLM pipeline needs failure-rate monitoring and alerting — you cannot rely on routine inspection. This is the biggest lesson in this post, and it isn't close.
3. API compatibility ≠ behavioral compatibility. Parameter names you can fix against the docs; output-format drift only surfaces under real calls. The first step of a migration should be a smoke test on production samples, not a full cutover.
4. Failed jobs must be replayable. Full automation isn't required, but the "never persist a failed parse, persist only what can be replayed" design is what turned the repair of 2,000+ documents into a batch operation instead of a disaster.
5. Data compliance is its own workflow. The gap between a public API and a commercial contract on data usage should be assessed separately from the technical migration — protect business continuity first, follow up on the compliance terms, but follow up.

Open Items

The v1-era dynamic tiering logic is currently commented out wholesale, to be restored or rewritten once we add multi-tier models again; the commercial contract is still under discussion. Finishing a migration doesn't mean the debt is paid — writing it down beats forgetting it.