The project began after an error message indicated that a request exceeded the limit of 922,000 tokens, reaching 1,674,813 tokens. The team uses a self-hosted OpenWebUI instance with an internal assistant for document and engineering tasks.
The author initially implemented a filter in the OpenWebUI middleware to clean up tool execution results. The filter removes old tool messages and clears tool calls from previous assistant responses, while preserving the text content to manage context size.
A second phase introduced summarization. When conversation history exceeds defined thresholds—such as 20 recent turns or a 30,000-token budget—the system compresses older turns into a summary using a dedicated model, specifically Qwen2.5-14b-instruct-1m.
In July 2026, the architecture shifted to an external Agent Runtime. OpenWebUI now serves as the interface, while the Pipe function acts as an adapter, sending tasks to an external service built on OpenAI Agents SDK and Codex SDK.
The transition to the Responses API in October 2026 required manual handling of SSE events to reconstruct tool outputs, as the SDK initially returned empty arrays for final responses. The system now maintains state in the external Runtime.
