Anthropic engineer Tarik Shikhipar proposes a framework for interacting with LLMs based on categorizing knowledge into four types: known knowns, known unknowns, unknown knowns, and unknown unknowns. This approach aims to address the gap between user instructions and the actual project environment.
The methodology involves a 'blind spot pass' where the model analyzes the task before execution. It identifies explicit facts, missing information, necessary assumptions, and potential risks or requirements the user may have overlooked, such as compliance or security protocols.
Research by Anthropic, including the 2022 paper 'Language Models (Mostly) Know What They Know', explores whether models can assess their own confidence. While models show some ability to distinguish reliable answers, calibration remains inconsistent across different tasks and data distributions.
Recent studies, such as the 2024 paper 'Can AI Assistants Know What They Don't Know?', investigate teaching models to refuse confident answers when information is outside their knowledge base. These efforts focus on functional metacognitive behavior rather than human-like consciousness.
Anthropic emphasizes mechanistic interpretability to understand how internal model activations lead to specific outputs. This research aims to provide tools for verifying model states independently of the model's own self-reports, addressing the challenge of managing powerful, opaque systems.
