2026-09-01 — views
Agentic data cracking reduces token costs for unstructured data reasoning
Read this because Enterprise AI teams face prohibitive token bills. This approach turns reasoning overhead into reusable infrastructure.
A new method structures unstructured data adaptively during reasoning to cut query costs by 53 percent.
The Cost of Unstructured Reasoning
Valuable data remains embedded in unstructured sources such as web pages, reports, contracts, filings, earnings calls, and PDFs. The primary bet in enterprise AI is deploying large language model agents that reason over this data to answer complex questions for knowledge workers. While current agents can perform this task, the cost is prohibitive. Each question often requires repeatedly opening large documents to recover scattered evidence, a process that can consume up to a million tokens. If the data were already structured, the same question would reduce to a cheap database lookup. On the FanOutQA benchmark, reasoning over an ideal pre-structured store is 28 times cheaper. This gap grows to orders of magnitude as questions fan out over more documents.
Adaptive and Speculative Structuring
Pre-structuring all data in advance is not viable because documents hold vastly more possible structure than any workload will use. Furthermore, the useful structure and documents are unknown until queries arrive. To address this, researchers propose agentic data cracking. This method structures unstructured data adaptively and speculatively as a byproduct of reasoning itself. The process is adaptive because observed queries decide when structuring happens and what matters. It is speculative because it goes beyond the current question. Whenever an agent opens a document to answer a query, a cracking sub-agent forks from the already-loaded context at marginal cost. This sub-agent extracts grounded structure likely to serve related future queries.
Performance and Infrastructure Implications
Over time, an increasing share of queries is fully covered by structured data. These queries are answered without opening a document, keeping agentic accuracy at close to retrieval-augmented generation cost. On the FanOutQA benchmark, extended with merely one related question per test question, cracking cuts cost by 53 percent while preserving accuracy. This approach represents a first step toward next-generation data infrastructure for agentic reasoning over unstructured data. It creates a shared substrate beneath the model where knowledge that reasoning already paid to uncover accumulates.
Practitioner note
Developers should monitor token consumption patterns to identify high-frequency document access points. Implementing a sub-agent that extracts structure during initial document loads can reduce long-term inference costs without sacrificing answer quality.