With the recent new wave of operating system versions (BigSur, iOS 14 etc.), announced at recent WWDC, Apple quite silently introduced a new ML framework to accelerate training of neural networks across the CPU or one or more available GPUs.

ML Compute is not properly a new ML framework but new API’s that utilizes the high performance BNNS primitives made available by the Accelerate framework for the CPU and Metal Performance Shaders for the GPU.

After looking at the documentation and starting to use it on an iOS/macOS application, I understood that this is not really a simple, high level framework but something probably targeting the acceleration of existing third-party ML library like the ONNX Runtime or the TensorFlow Lite framework on the Apple platform.

Even if Apple documentation is pretty good, I would say these APIs are not really developer friendly and swifty for doing generic ML on iOS/macOS. The tensor API, for example, is really rough and it requires dealing with unmanaged pointers in Swift. Basically, you are responsible for managing by yourself the ownership and lifetime of memory allocation of objects like tensors, nodes and graphs that you pass to these API.

More generally, ML Compute does not provide by definition ML APIs like Keras, PyTorch or Swift for TensorFlow to simplify building and training ML models but low level API to build computing graph and manage low level training loop.

For general ML coding on iOS/macOS, I would suggest continuing to use Core ML with tools like CoreMLTools to import model from other framework (TensorFlow, PyTorch etc.) or eventually give a try to the SwiftCoreMLTools library I developed if you want to completely build and/or train model locally on devices avoiding any Python code.

Anyway my personal opinion, after playing with that, is that ML Compute could become potentially really powerful even for regular Swift ML developer adding for example on top of that a Swift Function Builder (DSL) high level API, like the one I developed for SwiftCoreMLTools, and a high level Tensor Swift API, hopefully, integrated with Swift Numerics multi dimensional array.

To quickly test the capability of these APIs, I decided to develop and illustrate a PoC App to train and inference with ML Compute on both iOS and macOS a simple shallow model for the MNIST dataset.

#swift #mlcompute #machine-learning #coreml #gpu #cpu