Dev Tools|Index 04
TurboFieldfare: Running 14GB LLMs on M-series Macs with 2GB RAM
A new inference engine streams large language models from SSD, enabling powerful local AI on consumer Apple Silicon.
- Via
- AITECH TOKYO Editors
- Dateline
- TOKYO
- Date
- July 29, 2026
- Time
- 5 min read
Source
Hacker News Top
Tagline
Run large LLMs on M-series Macs with minimal RAM.
Who & Why
For a Tokyo-based AI developer or researcher who needs to run larger LLMs locally on their M-series Mac for privacy, cost savings, or offline access, enabling faster iteration on experimental features.
vs. Existing
This competes with traditional local LLM inference tools like LM Studio or Ollama, offering a distinct advantage by running much larger models on lower-spec M-series Macs due to its SSD streaming architecture.
Tokyo Take
While the core innovation is technical, its impact for Tokyo professionals is enabling more powerful local AI development. Japanese language fine-tuning or specific data handling for local regulations would still be necessary, but the underlying capability for on-device processing is a significant step.
TurboFieldfare is a specialized inference engine designed to run large language models on Apple M-series Macs with significantly reduced RAM usage. Developed in Swift and Metal, it enables models typically too large for consumer hardware to operate efficiently.
The engine specifically targets the 4-bit quantized Gemma 4 26B-A4B-IT model, whose weights alone occupy approximately 14 GB. Conventional methods struggle to run such models on Macs with 8 GB or 16 GB of RAM, especially when accounting for the operating system and other applications.
Its core innovation lies in keeping only the shared parts of the model and the KV cache in RAM, while streaming the necessary "expert" weights from the SSD as each token is generated. This technique mitigates the SSD's slower access speeds through a small expert cache and bounded parallel read operations.
The developer reports impressive performance metrics, achieving 5–6 tokens per second on an 8 GB M2 MacBook Air and a much faster 31–35 tokens per second on an M5 MacBook Pro. This demonstrates the viability of the approach across different M-series chip generations.
Beyond core inference, TurboFieldfare includes an experimental OpenAI-compatible local server. This feature supports streaming responses and tool calls, allowing developers to integrate the local model into existing workflows designed for cloud-based OpenAI APIs.
It feels like magic that you can run a powerful NN on your Mac or iPhone.
For a Tokyo-based professional, particularly a developer or researcher, this tool offers a pathway to experiment with larger, more capable models locally without incurring cloud API costs or privacy concerns. It could accelerate local development cycles for AI-powered applications, especially those requiring data to remain on-device.
Adjacent Tools
Dev Tools
Armature Launches Analytics for AI Agent Tool Calls
Armature introduces a new analytics platform designed to provide observability into how AI agents use external tools, reconstructing user intent and agent reasoning to diagnose issues in complex AI applications.
Dev Tools
AI-First Code Editor Cursor Discontinues Operations
The dedicated AI coding environment struggled to compete with established IDEs rapidly integrating similar features.
Dev Tools
Bor: Real-time Linux Desktop Management for IT Teams
An open-source system for centralized Linux workstation management, Bor offers real-time policy enforcement and software deployment, streamlining IT operations without direct AI integration.