Blog / · GH-Proxy Team

Laya-MLX: Sub-15ms Local Decision Inference on Apple Silicon without Cloud Latency

#Open Source #LayaMLX #AppleSilicon #MLX #GitHub Proxy #Local AI
Table of Contents4sections

Laya-MLX: Sub-15ms Local Decision Inference on Apple Silicon without Cloud Latency

On Apple Silicon hardware, unified memory architecture provides exceptional bandwidth for local AI. Yet, even small 4-bit quantized generative LLMs suffer from time-to-first-token (TTFT) delays of hundreds of milliseconds, ruling out hyper-reactive loops like real-time robotic navigation or continuous arcade games.

Laya-MLX changes this paradigm by implementing native MLX kernels for Laya’s non-autoregressive decision checkpoints. Clocking over 6,400+ Stars on GitHub with 2,700+ Stars gained this past week, it is establishing a new benchmark for local inference speed on Mac.

🚀 Accelerated Access via GH-Proxy

Download Laya-MLX source code and Hugging Face weights at peak velocity via GH-Proxy:

👉 Direct GH-Proxy Acceleration for Laya-MLX
(Alternative URL: https://gh-proxy.com/https://github.com/mizorewww/laya-mlx)

Installation:

git clone https://gh-proxy.com/https://github.com/mizorewww/laya-mlx.git
cd laya-mlx
pip install -r requirements.txt

⚡ 7.4 ms: Real-Time Non-Autoregressive Execution

In a live benchmark navigating an autonomous Snake game run at original native framerates, Laya-MLX computes directional choices with astonishing efficiency:

  • 13.4 ms median end-to-end for English decision checkpoints.
  • 7.4 ms for the multilingual compact model.
  • 0 Output Tokens: Bypasses token decoding completely by evaluating typed output heads directly in a single forward pass.
  • Zero PyTorch Dependency: Native Metal and C++ operations run straight against Apple’s unified memory.

🛠 Quick MLX Inference Example

from laya_mlx import LayaMLXEngine

# Load checkpoint straight into Apple Silicon Unified Memory
engine = LayaMLXEngine.from_pretrained("aac6fef/laya-mlx")

# High-frequency decision routing
result = engine.decide(
    context="Wall ahead at 1 unit distance; clear open lane to the left; tail block to the right",
    choices=["UP", "DOWN", "LEFT", "RIGHT"]
)

print(f"Decided action: {result.choice} in {result.latency_ms:.2f}ms")

🎯 Impact & Use Cases

  • Robotic & Cyber-Physical Controllers: Provide sub-15ms reflex actions on autonomous rovers driven by Mac mini hardware.
  • Zero-Latency Desktop Copilots: Dispatch keystrokes, shortcuts, and application launches locally without cloud lag.
  • Air-Gapped Privacy Pipelines: Run sensitive classification pipelines without telemetry or network egress.

💬 Comments & Discussion