The SilentJSON library for Go introduces a streaming approach to data processing, moving away from the traditional model of parsing entire buffers into memory. By utilizing structural markers inherent in JSON, the system can identify data boundaries and process information as it arrives, rather than waiting for complete messages.
The architecture separates structural scanning from data mapping. A registry handles field matching and type conversion, while the parser focuses on identifying logical blocks. This allows for raw block processing, which can reach speeds exceeding 4 GB/s, significantly outperforming standard decoding methods that require object instantiation.
Performance benchmarks highlight the efficiency of this model. In parallel mode, SilentJSON achieved approximately 24 670 MB/s when processing 100 000 objects, compared to 644 MB/s for Sonic and 110 MB/s for encoding/json. The StreamDecoder.Next function reached 614 MB/s with 41 MB of allocations, while NextRawBlock hit 4009 MB/s with 0.9 MB of allocations.
This streaming model enables early event detection, allowing systems to react to specific data points before a full pipeline is initialized. It is particularly useful for logging, monitoring, and financial transactions where latency is critical. The approach also optimizes memory usage by allowing developers to choose between zero-copy references and explicit data cloning.
