Nokia has released AnyJev, an open-source layer designed to transform open large language models into calibrated decision-making tools. The software is available via PyPI under the Apache-2.0 license and supports transformers and vLLM backends.
The tool utilizes an interface derived from the Jev model by TypeSafe AI. It processes typed questions and returns decisions along with probability scores derived from the model's next-token distribution. The system requires no additional training, parsing, or text generation.
The research team addressed two primary issues found in existing methods: sensitivity to the order of options and lack of probability calibration. AnyJev implements two levels of processing: L0 for standard scoring and L1, which applies temperature scaling for 100 to 500 labels.
Performance metrics for L0 indicate a processing time of approximately 0.25 seconds per decision when using a batch size of 32 on a single H100 GPU with K equal to 20. The L1 level stores calibrated values as a small JSON artifact to adjust confidence without altering answer rankings.
