Yesterday we got our answer, with the announcement from Apple that Metal would be coming to OS X “10.11” El Capitan, and that it would be part of a larger investment into Metal for the company. Which is why for the last year we have been wondering if Metal would stay on iOS, or make the jump to OS X. What this means is that out of all of the vendors, it’s arguably Apple who has the lead time advantage in low-level API development. Microsoft’s DirectX 12 is arguably not there yet (Windows 10 is still in testing), and Khronos’s Vulkan was still in its primordial Mantle form at this point in 2014. Metal, despite being the 3 rd such low-level API to be introduced, was the first to reach production status. And while we typically don’t think of mobile devices as being GPU-bound, in reality Apple has been packing some relatively powerful GPUs like GXA6850 with what are relatively speaking still fairly weak CPUs, which means Apple has ended up in a situation where they can be CPU-bottlenecked on draw calls. A low-level graphics API, Metal was originally designed to bring the benefits of low-level graphics programming to Apple’s mobile operating system. I'd love to see a proper Mac GPU training with Metal on either Intel or M1 but am unsure how to make it happen.At last year’s WWDC, Apple introduced their Metal API for iOS 8. Fun, but too much effort to make a PR for PyTorch at the time. Years ago, I wrote an unpublished proof-of-concept Metal/MPS replacement for a "look alike" Linear layer to measure speed improvements in inference for Accelerate vs Metal on macOS.
My impression (which might be a few months outdated, sorry) is that PyTorch training< on the Mac never uses the GPU via Metal/MPS and even used to ignore Accelerate in favor of Intel libraries (was it 1.2 or 1.3 ? ) even though Apple can optimize more for CPU performance (GEMM) on M1 and Intel. Works like Add Metal/MPSCNN support on iOS #46112 have already gone down this approach but the drawback is that we can only make use of the GPU but not the neural engine. Write Metal compute shaders to accelerate PyTorch tensor operators.
I also love to have a proper macOS PyTorch using Metal/BNNS/Accelerate but specifically Metal to have TFLOPS training and inference performance on my iMac 64GB with Radeon Pro 580 8 GB. From my perspective there are three possible ways:
The very first problem I can think of is how do we want to accelerate PyTorch on macOS computers.
0 kommentar(er)
