Deep Dive into Content Faithfulness: A new metric for ensuring text accuracy

Parsebench is the first document OCR benchmark for AI agents. With its release we defined five new metrics for determining the accuracy of your document parser. In this video, we dive deep into the content faithfulness metric. The most fundamental requirement: did the parser actually capture all the text, in the right order, without making things up? We test for three failure modes: Omissions: dropped text at word, sentence, and digit levels Hallucinations: fabricated content that doesn't exist in the source Reading order violations: multi-column layouts linearized incorrectly This is eval…