AI
Datalab Introduces OmniParseBench for OCR Parser Evaluation
Datalab has launched OmniParseBench, an open-source evaluation harness featuring 16,288 tests to audit the performance of OCR document parsers.
October 15, 2026, 02:37 AM·MarkTechPost
#benchmarking
AI
Artificial Analysis Debuts AA-3D Benchmark for Interactive AI Generation
Artificial Analysis has launched AA-3D, a benchmark that uses human preference to rank AI models on their ability to generate interactive 3D scenes via three.js code.
Crypto Briefing
AI
MWS AI Evaluates AI Models on Document Fraud Detection
MWS AI has tested the ability of multimodal models to detect document fraud using a new experimental dataset of 430 images.
CNews.ru
AI
Legal AI Benchmark LAB-AA Adds Hallucination Detection
Version 1.1 of the LAB-AA legal AI benchmark now includes hallucination checks to verify that model outputs are grounded in source documentation.
Crypto Briefing
Money
Arena Valuation Hits $3.1 Billion Following New Funding Round
AI benchmarking platform Arena has reached a $3.1 billion valuation after raising $200 million in a Series B round, nearly doubling its value in 10 months.
TechCrunch
AI
Epoch AI Benchmarks Reveal Limitations of Frontier Models in Research Tasks
Epoch AI’s new benchmark reveals that top models from Anthropic and OpenAI score 65% on internal research tasks, struggling primarily with creative judgment and style.
Crypto Briefing
Money
Arena Secures $200 Million and Debuts AI Safety Benchmark
AI evaluation startup Arena has raised $200 million in a Series B round, reaching a valuation of $2.88 billion, and launched a new benchmark for AI agent behavior.
Crypto Briefing
AI
Benchmarking Specialized Decision-Making AI Models
A new benchmark has been created to evaluate the performance, speed, and cost-efficiency of specialized decision-making AI models in production.
Хабр
AI
Evaluating AI Data Masking: Datasets and Benchmarks
Researchers analyze various datasets for testing AI data masking gateways, highlighting the need to combine external benchmarks with specialized tools for verifying agent performance.
Хабр
AI
Stanford Study Finds Single AI Agents Outperform Multi-Agent Teams
A new Stanford study reveals that single AI coordinators significantly outperform multi-agent teams when managing shared resources, often due to coordination failures.
Crypto Briefing
AI
MWS AI Releases Open-Source Tools for Speech Recognition Evaluation
MWS AI has released the open-source RESON library for ASR model evaluation alongside a new benchmark featuring 38 hours of diverse Russian-language audio data.
CNews.ru
AI
New CAIS Benchmark Tracks AI Cheating Behaviors
The Center for AI Safety has released CheatBench, a new tool that tracks how often AI agents exploit shortcuts to inflate their performance scores.
Crypto Briefing
AI
Grok 4.7 Leads New AI Cyber Defense Benchmark
SpaceXAI's Grok 4.7 model has tied for the top spot on the new Artificial Analysis Cyber Index with a score of 56, though it carries a premium cost of $11.67 per task.
Crypto Briefing
AI
Evaluating LLMs: Building a Custom Benchmark for Data Tasks
Researchers created a custom benchmark to evaluate LLMs for database querying, finding that performance varies significantly across models and costs range from $0.12 to $4.94.
Хабр
AI
Datalab Launches OmniExtractBench for Standardized Data Extraction
Datalab has launched OmniExtractBench, an open-source benchmark designed to standardize the evaluation of structured data extraction systems and improve transparency in vendor comparisons.
MarkTechPost
AI
Nvidia Research Highlights Reliability Gaps in Long-Context AI Models
A new Nvidia study reveals that AI models experience a 62.8% average drop in accuracy when processing long-horizon tasks, highlighting critical reliability limits in current LLMs.
Crypto Briefing
AI
WebPageBench: A New Standard for Evaluating AI Web Agents
WebPageBench provides a standardized environment to evaluate AI agents on Russian-language simulated websites using event-based logging.
Хабр
AI
Anthropic Debuts Claude Sonnet 5.5 With Mixed Efficiency Results
Anthropic has launched the Claude Sonnet 5.5 model, which offers performance near Opus 5.5 levels but shows varying token efficiency depending on the workload and testing environment.
BeInCrypto
AI
Artificial Analysis Expands Into Cybersecurity With New Benchmarking Index
Artificial Analysis has launched the Cyber Index (v1) to evaluate how effectively AI models can identify and patch enterprise security vulnerabilities.
Crypto Briefing
AI
Google Research Launches MSEB Benchmark for Audio Models
Google Research has introduced the MSEB benchmark, a structured framework for evaluating audio models through standardized data types, encoder interfaces, and task-specific evaluators.
MarkTechPost
