Xiaomi has introduced TabLDM, an AI model designed for tabular data that functions without the need for task-specific fine-tuning. The company released two versions: a classifier with 70 million parameters and a regressor with 71 million parameters. By default, the model generates predictions by aggregating results from eight passes with varied feature shuffling and preprocessing.
The architecture consists of three stages, utilizing a Set Transformer to encode column groups before generating row vectors. The final eight blocks incorporate a mixture-of-experts approach, selecting between two routed experts and one shared expert per token. To optimize performance, the system includes a KV-cache, and large datasets can be stored in RAM or on disk.
In the TabArena regression benchmark, TabLDM achieved an Elo rating of 1900, ranking behind TabFM at 2019. Across a broader set of 51 datasets, it secured fourth place with 1659 points, trailing AutoGluon 1.5 at 1662 and TabPFN-3 at 1650. Developers report that the model's fit stage is 82% faster and inference is 68% faster than TabFM.
The model is compatible with tables ranging from hundreds to tens of thousands of rows and from a few to over one hundred columns. However, inference time scales quadratically with both row and column counts, increasing computational costs for larger datasets. The model is available for use and modification under the Apache 2.0 license.
