The number said "-50%." I'd been staring at trade logs long enough to have a rough feel for how far a leveraged ETF actually needs to fall before a real -50% drawdown trigger should fire, and this one didn't look right. When I actually checked the math behind that specific row, the trigger had fired at roughly -2.9% below the average cost basis -- nowhere near the -50% the log claimed. That one suspicious row turned into a week of finding three separate, unrelated bugs in a backtesting engine I'd built with Claude Code, each one wrong in a completely different way.
The project, briefly Outside of this blog, I've been using Claude Code to build and iterate on a backtesting simulator for a rules-based, dollar-cost-averaging-with-leverage trading strategy on leveraged ETFs -- buy more as price drops in defined stages, sell in defined stages as price recovers, all governed by explicit percentage thresholds and cooldown timers. None of the specifics matter for this post. What matters is that the simulator has to replay years of daily price data and decide, on every single day, whether a threshold condition is true. That's a lot of surface area for a rule to be implemented slightly differently than it was specified, and unlike an API call failing, a wrong threshold doesn't throw an exception -- it just quietly produces a plausible-looking wrong number.
Bug one: the trigger price formula didn't match the trigger rule The rule, as written in my own spec, was simple: a buy-the-dip trigger at "-50% below average cost" should fire when price crosses averagecost 0.5. What the code actually computed used a different reference point entirely, one that happened to produce trigger prices only a few percent below cost instead of half:
what the code actually computed -- a fixed step down from cost, which lands only a few percent below it, not half triggerprice = averagecost - fixedstep_amount
The formula was internally consistent -- it always fired at the same (wrong) distance -- which is exactly what made it hard to catch by eyeballing outputs. It looked like a real trigger doing real work. It just wasn't the trigger the spec described. Every backtest run before this fix used trigger points that didn't match the strategy's own documentation, which meant weeks of "results" needed to be thrown out and rerun once the formula was corrected.
Bug two: a reactivation timer that was secretly a different mechanism The second bug took a specific, hard-to-describe symptom to surface: a sell trigger reactivating suspiciously soon after its cooldown period should have still been active. The cooldown was supposed to be pure time-based -- N trading days pass, the trigger rearms, unconditionally. What was actually implemented was closer to hysteresis: the trigger would only rearm if price dropped back below the threshold line within that window; otherwise it stayed permanently disabled for the rest of that cycle. Those two behaviors look identical in the common case and diverge only in specific price paths, which is exactly why it survived several earlier rounds of eyeballing. Fixing it changed the simulated results substantially -- selling more often, it turns out, meaningfully changes how much of a multi-year uptrend a strategy captures, so the "wrong" and "right" versions of this rule didn't just differ by a rounding error, they told different stories about which strategy variant was better.
Bug three: a structural assumption that only broke on specific days The third bug was the quietest of the three. The simulator originally computed everything on split-adjusted historical prices -- the standard convention, where past prices are rescaled so that today's share count lines up with history. Re-verifying against raw, unadjusted historical prices (to check whether the strategy's "only buy whole shares within a daily budget" rule would have actually been achievable in real life) surfaced a case where a stock split day wasn't handled at all: share counts weren't scaled up on the split date, so portfolio value briefly appeared to be cut in half on paper. It's the kind of bug that only exists on a handful of specific calendar days across sixteen years of data, invisible unless you specifically go looking at those days.
What ties these together None of these three bugs would show up as a stack trace. They'd show up, if you were unlucky, as a strategy decision made on subtly wrong numbers -- the kind of bug that's genuinely dangerous specifically because the output still looks like a normal number in a normal range. The common thread across all three fixes was the same: stop trusting that a formula which "runs without error" is the formula that was actually specified, and go check a handful of individual data points by hand against the written rule instead of trusting the aggregate output. That's a much slower way to debug than reading an exception traceback, but for anything that computes silently-wrong numbers instead of throwing, it's the only way that actually works.
Related reading The Silent Scheduled-Task Failure Nobody Warns You About (And How I Caught It)My OAuth Tokens Kept Expiring Every 7 Days, and the Reason Was a Dropdown Labeled 'Testing'