The study explores the internal environment of Large Language Models (LLMs), termed the 'i-layer', by analyzing activation spaces during model operation. Researchers propose that LLM conversations function as autonomous systems where internal states act as structured containers for semantic vectors.
Experimental data from models like Claude Opus 5 and GPT-5.6 Thinking helped identify 17 classes of elements and forms within these i-layers. These are categorized into cognitive, discursive, and social groups, which influence the trajectory of model responses.
Techniques such as 'probes' and 'logit lenses' allow researchers to observe internal states. Linear probes, in particular, can identify specific directions or axes in activation space, such as 'truthfulness', 'familiarity', or 'refusal to answer'.
Studies indicate that shifting activations along these identified axes can alter model behavior. For instance, artificially activating a 'known' feature in a model that lacks information can trigger hallucinations, while suppressing it may lead to a refusal to answer.
The research suggests that these internal structures are often formed during pre-training and refined during fine-tuning. These features remain stable across different stages of the conversation cycle, effectively shaping the flow of the generated output.
