Bob Browning's blog

Our website fintechbenchmark.com is a large directory of fintech products and services. The information on the site was seeded from the internet, and in this dynamic industry things change. To keep it current, we run an agent whose job is to keep the site up to date. It runs overnight, processing a batch of companies and their products, using AI to check that the information is accurate — and updating the site when things have changed.

Over time, this job will get smaller. We give vendors the opportunity to claim their pages and update them directly, and we encourage it — that’s the only way we can be sure the information stays truly current. So if you’re a fintech product vendor: please find your entry and claim it.

Part of the payload we send to the AI service (currently Claude) is a large amount of data that doesn’t change from request to request — including our full list of product categories, which we need so new products get allocated correctly and miscategorized ones get fixed. Claude lets us mark that fixed data so it can reuse a pre-processed version of it on the next request, instead of reprocessing it from scratch, as long as the two requests are within five minutes of each other (that window can be extended). This cuts the cost significantly.

The saving varies. Running with no tools at all, we see something like a 65% reduction. Once we add tools like web search or web fetch, the cached data becomes a smaller share of the total request, so the percentage saving goes down — even though the tools themselves are essential and add meaningfully to the cost. In absolute terms we’re talking about small numbers of cents per company, but across 4,000 companies and 7,000 products, it adds up.

The Claude console conveniently shows total prompt-caching savings for the current month, so we can track it. We also produce our own overnight report identifying what changed and what it cost.

Eventually, I hope to decommission this program once every page is maintained by the real experts — the vendors themselves. So: give us a hand. Claim your company.