Independent developer Ash Hart (@ashxhart) shared a personal method today for optimizing costs and efficiency in AI server configurations—MCDMA (Metal CUDA Direct Memory Access). This might currently be the most cost-effective way for personal setups to run DeepSeek V4 Flash. It enables direct memory access, similar to RDMA, between the unified memory of Apple Silicon Macs and the VRAM of NVIDIA CUDA devices using just a single USB-C cable. Two NVIDIA DGX Spark units (each with 128GB unified memory) are interconnected via ConnectX-7 and handle Prompt Processing (prefill). Each Spark is then connected to a Mac Studio via an independent USB-C link to handle Decode (generation). Both links work concurrently. The Mac can directly write data into the CUDA memory of the Spark, and the Spark can directly write back to the Mac. It’s a peer-to-peer setup with no master-slave relationship. A single Spark officially supports around 200 billion parameters (post-quantization), and dual Spark interconnection can reach approximately 405 billion parameters. Adding the unified memory of the Mac Studio, the author claims the memory can theoretically pool up to 512GB, enabling larger single-model runs. Currently, he’s primarily running DeepSeek V4 Flash and believes the community can test even more extreme sharding setups in the future. Interconnection bandwidth has been tested: - Single link: ~939 MB/s - Mac → Two Sparks concurrently: ~1.80 GB/s - Two Sparks → Mac concurrently: ~1.25 GB/s - Round-trip latency: only 24 µs The author promises to open-source and publish the full tokens/s benchmark this week. For now, based on single-device performance: - A single Spark generates roughly 14–20 tok/s - Mac Studio / M-series devices, after optimization (including speculative decoding), typically achieve 25–45+ tok/s The goal of the hybrid setup is to let Spark accelerate prefill while the Mac leverages its high bandwidth for decode, theoretically balancing scale and speed. Current USB controller limitations still exist, but the author is pushing NVIDIA to unlock higher bandwidth, leaving room for further improvement. The biggest value of this setup is that it connects the Apple MLX ecosystem and NVIDIA CUDA ecosystem using the simplest USB-C connection, enabling true cross-platform distributed local inference. It solves the issue of insufficient memory on a single machine while avoiding complex network configurations as much as possible. Looks like Macs are about to get even more expensive...