The experiment involved running eight language models on an Inspur YPCB-00338 FPGA board equipped with a Kintex-7 XC7K480T-2 chip. The hardware features 2 GB of DDR3-1066 memory per channel and a PCIe Gen2 x8 interface. The setup utilized an open-source TPU implementation, with the core operating at 133.33 MHz.
Performance testing showed that the matrix of 512 multipliers achieved 121 GOPS out of a theoretical 136.5 GOPS. The bottleneck was identified as the DDR3 memory, which operated at 94% of its capacity during text generation. This limitation meant that increasing the PCIe bandwidth had minimal impact on overall performance.
The researcher successfully executed 165 LLM runs. Inference speeds reached 12.5 tokens per second for Qwen3.5-2B, 35 tokens per second for Qwen3-0.6B, and 95 tokens per second for LFM2.5-230M using 4-bit weights. Switching from 8-bit to 4-bit weights provided a speed increase of 1.4 to 1.8 times.
The total cost for the hardware setup is approximately $100, including the board ($50), shipping ($10), and JTAG adapter ($40). Using an external USB4 adapter setup increases the total cost by an additional $225. The project requires Vivado ML Enterprise edition for firmware compilation.
