Apple shows how much faster the M5 runs local LLMs on MLX

A new post on Apple’s Machine Learning Research blog shows how much the M5 Apple silicon improved over the M4 when it comes to running a local LLM. Here are the details.

A bit of context

A couple of years ago, Apple released MLX, which the company describes as “an array framework for efficient and flexible machine learning on Apple silicon”.

In practice, MLX is an open-source framework that helps developers build and run machine learning models natively on their Apple…