1 September 2026

Twelve days early

A trading system asked FRED for the economy as it stood on 31 March 2023. FRED answered with the economy as it stands now, and nothing failed.

Ask FRED for the consumer price index as of 31 March 2023 and you get a figure for March. It looks correct. The observation is dated 1 March, comfortably inside the window you asked for, the units are right, the series title is right, and nothing in the response suggests a problem.

March CPI was published on 12 April.

So a backtest deciding what to do on the last day of March, written by somebody who had carefully bounded the request to end on that day, was reading a number that would not exist for another twelve days.

I know this one from the inside. I shipped it into my own production system: macro series read by observation date, when the only thing actually on my screen on the morning of a decision was the previous day's publication. It sat there for weeks. Nothing about it looks wrong, which is the entire difficulty.

Two windows, and only one of them was set

Economic data has two dates and most code only thinks about one. There is the period the number describes, and there is the day the number was published. March CPI describes March and arrives in April. Payrolls describe a month and arrive in the first week of the next one. Quarterly GDP describes three months and arrives about four weeks after they end. Every series from every statistical agency works this way, and the gap runs from days to a quarter.

FRED models this properly, which is more than most vendors do. observation_start and observation_end bound the period the data describes. realtime_start and realtime_end bound the vintage: what was known, and when. The documentation says plainly that the second pair is optional and defaults to today's date.

Set only the first pair and you have asked a precise question about March and accepted today's answer to it.

A window that ends on the decision date is not the same thing as data that existed on the decision date.

What it hands you, measured

In August I audited the FRED path in TradingAgents, an open source multi-agent trading framework with over a hundred thousand stars on GitHub. Its macro vendor bounded the observation window and left the vintage unset.

I measured it against the live API for a decision date of 31 March 2023 over a one year window, fetching each series twice: once with the parameters the code actually sent, and once with the vintage pinned to the decision date. The difference is the leak.

Every series tested handed over exactly one release that had not happened. An agent reasoning on 31 March had March CPI, March payrolls, March unemployment and first quarter GDP in front of it.

And the numbers it was entitled to had moved

The unpublished releases are the obvious half. The quieter half is that the observations it was legitimately allowed to see had all been revised since. Comparing only the dates present in both vintages, so this is revision effect on its own:

First quarter 2022 real GDP reads about ten percent higher today than it did in the March 2023 vintage, most of that rebasing rather than genuine revision. January 2023 payrolls have been revised down by 263,000 people. Industrial production for December 2022 has moved by nearly two points.

A backtest reading those is not quite being handed the future. It is being handed a tidier past than anybody actually had, which is a subtler advantage and a harder one to notice, because every individual number looks reasonable.

Nothing failed

No exception. No warning. No empty result. The response is well formed, the table renders, the units and the series title are correct, and the values are real values that the Federal Reserve genuinely publishes.

The function even carried a docstring describing the behaviour it was supposed to have: no later observations are returned, so a past date never leaks future data. That sentence is true about observations and false about information. Somebody had thought about this problem, reached the right conclusion, written it down, and then bounded the wrong window.

The correct intention was already in the file. It was the request that was wrong.

This is why the class survives review. A reviewer reads the docstring, reads the request, sees a date bound that matches the docstring's promise, and moves on. The two windows are one word apart in a parameter list.

Whether you have it

The question to ask of any input to a backtest is not whether it ends on the right date. It is whether it was knowable on that date. Those are different questions and in most code only the first one has been asked.

Some tells, in rough order of how often I find them:

• Your data source has a concept of revisions or vintages and you have never used it. Then you are reading the latest one, everywhere, for every historical date.
• Anything from a statistical agency. CPI, GDP, payrolls, unemployment, industrial production and their relatives are all revised, some of them for years afterwards, and benchmark revisions move them a long way.
• Fundamentals taken from a current snapshot. Restated earnings are the same defect wearing an accounting costume.
• Index membership applied backwards, so the backtest trades the companies that turned out to be worth including.
• An adjusted close column. A split or dividend adjustment applied across the whole history is this same shape again, and it is the single most common one I see.

None of these throw. That is the property they share and the reason they are worth paying somebody to look for. A backtest with a genuine bug usually announces itself, because it crashes or returns nothing or produces something absurd. A backtest with look-ahead returns a number that is plausible, defensible, and higher than it should be, attached to a mechanism you can explain to somebody at a party.

Where it landed

I reported it. The maintainers had a fix committed within two hours and pinned the vintage in the next release, which is faster and more gracious than the average and worth saying out loud.

That first pin was unconditional, which broke live runs for anyone whose clock sits ahead of Chicago, and a second release fixed that. I am not raising it to score a point. My own first draft of the same fix carried the identical defect, and it only got caught because somebody else read it before it went anywhere. Which is the observation one more time: the distance between the date a number describes and the date it became knowable is easy to lose hold of, and you can lose hold of it while you are in the middle of fixing it.

What found it was not cleverness. It was reading the vendor's documentation next to the request the code actually sent, noticing that one pair of parameters was missing, and then going and asking the live API twice to check whether the difference was real. That is an afternoon of unglamorous work, and there is no step in most people's process where it is anybody's job.

The number comes out. It is plausible. Nobody is assigned to try to break it.

Being assigned to try to break it is the whole of what I do. Here is what that involves.