Unstructured

OpenAI shipped a new, fast, and inexpensive vision model last week.  Look at which benchmarks made the launch story: • Computer use: up 15 points • Terminal tasks: solve rate more than doubled • Table structure: not reported. Forms: same. Documents do get a mention: “great for invoices, document extraction." But they didn’t share any relevant benchmark stats. We're starting to see VLMs split into two different jobs: • Screen vision — buttons, menus, terminals. The pixels an agent needs to click next. • Paper vision — 400-cell tables, nested forms, handwriting, stamps, reading order across a 200-page scan. The pixels an enterprise needs to trust. It's not hard to see why frontier labs are prioritizing screens. Agents operating software are a massive market, and coding and computer-use environments give researchers something they love: feedback loops where you can increasingly verify whether the model actually succeeded. Document processing is a different problem. The layouts have an incredibly long tail, the benchmarks are contested (everyone has their own at the moment), and building reliable ground truth is expensive (we know this firsthand). And extracting the words is only half the job. You need to know which label belongs to which field, which value sits under which table header, and how sections relate to one another. Get those relationships wrong and everything downstream is corrupted. A human still has to find and fix it. There's a consistency problem too. A general-purpose model might flatten merged cells on one pass and preserve them on the next. Or change how it interprets headers after a model update. That's a huge problem when your pipeline expects the same structure every time. Meanwhile, look at what's shipped in document AI over the last eight weeks: • A 4B parser claiming state of the art (and starting a benchmark fight) • A new multilingual OCR flagship covering 90+ languages • Single-pass parsing at one cent a page • A 2B hybrid VLM matching models 6x its size These are relatively small, purpose-built models that can be fine-tuned for specific document types and output formats, optimized for consistency of performance and economics at scale. You can control the model, measure its performance, and keep it from silently changing. None of this means frontier models aren't getting better at documents. They almost certainly are. But if your pipeline strategy is "the next frontier release will fix tables," that's not much of a plan. Screens are their job now. Enterprise files is ours.

This week Apple included risks from Agents in its 10-K and two other big vendors also showed what governance will look like in practice. Apple added a new paragraph to the risk factors in its annual 10-K. It warns that Agents, its own or anyone else’s, may exceed their permissions or resist instructions that can cause consequences which are difficult or impossible to reverse. It also says that Apple’s ability to oversee and control third-party agents “may be limited” Snowflake announced always-on Bots that inherits access rights, roles and permissions of the person it works for and comes back for human review when a decision or sign-off is needed. CrowdStrike also launched a kill switch that lets a security team stop an agent, blocking it from relaunching and cutting off its network access. Put all these together and you get the shape of agent governance that enterprises will start buying in the future: limit what an agent is allowed to do, and stop it when it does something it shouldn’t. For agents working with documents, you also need to know where the information they act on comes from because if your auditor asks where that number came from, “the agent read from the pdf” will not suffice. When your team writes the governance plan for its agents, add 3 questions about the document pipeline: 1. Can you trace every value back to the page, and the spot on the page it came from? 2. Do you know how confident the extraction was, and does a low-confidence value stop the workflow or sail through? 3. When the parser underneath it changes, do you re-test it before the agents see the outputs? When someone asks why a payment was approved, you need to be able to follow it back through the workflow to the source document and check what the agent was given. Permissions and kill switches govern the action. Provenance governs the input. Agents that touch sensitive data need both. #AgenticAI #Governance #EnterpriseAI #DocumentAI #DocumentProcessing

If you're choosing a document parser for an agent this quarter, here's how I'd test it in an afternoon. 1. Collect your 20 or 30 worst files. The skewed scans, the table that spills onto page two, the form someone filled in by hand. Your easy documents will pass with anything. 2. Write the ground truth for the fields your agent actually uses. It's tedious, and it's the part that makes the test worth trusting. 3. Score two numbers: the share of fields that are right, and the share of documents with no errors at all. The second one is what your agent experiences. 4. Look for the five patterns behind most failures: values in the wrong row or column, labels attached to the wrong value, blank fields filled in with a guess, checkboxes read wrong, and handwriting that comes back plausible but wrong. 5. Check what happens when the parser isn't sure. Does it tell you, or does it guess? A vendor's benchmark tells you very little about steps 3 and 4 on your own documents. #DocumentAI #Evaluation #AgenticAI #Unstructured

Last week, Cohere caught a Qwen model trying to upload data to an Alibaba Cloud server in Beijing during a sandboxed security test. It was rare, fewer than 1 in 100,000 calls, and the sandbox blocked it. Follow-up research suggests it was more likely unpredictable model behavior than a deliberate backdoor. But for the CISOs we work with, a model attempting an unauthorized upload to Beijing is still a serious problem. And this creates a real challenge for enterprise AI. We use Qwen models ourselves (as do most of our peers). They've been some of the best open models for document AI. Western alternatives have been harder to find, especially when you need something small enough to fine-tune and economically competitive for processing millions of pages. At the end of the day, model quality gets you through the technical evaluation. Security approval gets you into production. A customer can love your benchmarks and still tell you they can't run the underlying model. We run into this often. Fortunately, the alternatives are getting better. NVIDIA is leading the way with models we're particularly excited about, including Nemotron 3.5 Parse. Its compact architecture and ability to extract text, layout, and reading order in a single pass make it especially interesting for large-scale document processing. Two other developments we're watching: Liquid AI's LFM3 family, with compact models spanning text, vision, audio, and multimodal workloads. Google Gemma 4 family, bringing more competitive open models across a range of deployment sizes. The infrastructure around these models matters too. We fine-tune on Fireworks AI, which is expanding its support for NVIDIA's Nemotron family, as well as Modal. Reusing our existing training infrastructure makes evaluating and migrating to new base models much more practical. Chinese open-weight models have been very good for us. We're not abandoning them. But we need competitive alternatives for customers whose security teams won't approve them. That's why these releases matter. The next generation of enterprise AI will be won on model benchmarks, as well as security, deployment flexibility, and the economics of operating at scale. Our job is to evaluate these models against real-world enterprise documents and make the best ones production-ready. #EnterpriseAI #DocumentAI #AISecurity #Nemotron

Most table errors I see aren’t misread numbers. The number is read perfectly and ends up under the wrong header. Three ways it happens: 1/ A blank cell gets skipped. In the line items below, Discount is empty. A parser that reads the row as text drops the blank, so 500.00 slides left into Discount and Total comes back empty. 2/ A merged header gets flattened. “Q3 2026” sits over two columns, Budget and Actual. Flatten it and the quarter only attaches to the first column, so Actual no longer says which quarter it’s for. 3/ A row label spans several rows. “East region” covers three rows on the page, gets attached to the first one only, and the other two come out with no region at all. Every value in all three is spelled right, so the output looks fine at a glance. An agent that reads Discount = 500.00 will still act on it. What fixes it is treating the table as a grid instead of a line of text. At Unstructured we keep every cell tied to its row and column by treating tables as HTML instead of Markdown or plain text so the 500.00 stays under Total because that’s where it sits on the page. If you’re testing a parser on tables, check where each value landed, not just whether it was read. #DocumentAI #DocumentParsing #AgenticAI #Unstructured