The team at Ingosstrakh's AI Development Center is working on Intelligent Document Processing (IDP) solutions. They utilize multimodal LLMs to extract structured data directly from documents, bypassing traditional OCR stages.
A key challenge for the developers is ensuring data accuracy, as LLMs can produce syntactically correct JSON outputs that contain factual errors, such as incorrect passport numbers or names.
To address this, the team explored using token logprobs—the logarithms of token probabilities generated during the model's inference process—to create a risk assessment metric for automated outputs.
By analyzing these probabilities, the developers aim to identify which documents require manual verification, effectively creating a confidence score system for their multimodal LLM-based pipeline.
