The Apple A20 Pro chip features a dual 16-core Neural Engine, marking a first for Apple Silicon. This architecture is designed specifically for AI workloads and outperforms previous SoC generations. Tests on the iPhone 18 Pro demonstrate that it runs a 27-billion parameter model twice as fast as the iPhone 17 Pro.
The iPhone 18 Pro and 18 Pro Max are equipped with 12GB of 96-bit LPDDR5X memory, offering higher speeds than previous models. The device achieves a memory bandwidth of 115.2GB/s. In specialized neural tasks, the dual 16-core Neural Engine provides peak throughput that exceeds the performance of the chip's 7-core GPU.
User Adrien Grondin demonstrated the performance gains by running the Bonsai-27B model on an iPhone 18 Pro, showing high token generation speeds. However, the device faces limitations with larger models; for instance, the 2-bit quantized version of Bonsai 2 is too large to fit entirely into the local memory, which can lead to performance degradation.
The original Bonsai model uses 1-bit quantization and runs on devices with 4GB of RAM, while 8GB or 12GB devices handle it efficiently. Future hardware iterations with increased memory capacity could potentially support even more complex and larger AI models on-device.
