Local development of large language models (LLMs) has become more practical with recent advancements. Models such as Qwen 3.8 and Gemma 4 can now be run efficiently on Apple Silicon Macs, particularly on the MacBook Pro M5 with 48GB of RAM. This setup supports 30 billion parameter models with substantial context windows, making it suitable for building AI-powered web applications.
Ollama is favored for this workflow due to its ease of use, speed on Apple Silicon, and stable performance. It offers a solid directory of models and supports mlx acceleration. Opencode serves as a reliable harness for running these models locally, providing a foundation for experimentation and development.
To maintain system stability and security, Docker sandboxes (sbx) are used to isolate the model execution environment. This approach helps prevent potential issues such as system crashes or unwanted model behaviors.
Installation involves setting up Docker Sandbox via Homebrew, installing Opencode, and downloading the Ollama app from its official website. Two main models are recommended: Qwen 3.8 27B mxfp8, which is efficient for extended tasks, and Gemma 4 31B mxfp8, which is suited for more demanding workloads.
Configuration requires creating a sandbox kit with specific files to customize the environment, including network permissions and model settings. The setup limits the Qwen model's context window to 64K tokens to avoid memory exhaustion, reserving additional memory for the sandbox.
Once configured, Opencode can be launched within the sandbox environment, with the ability to adjust model reasoning effort levels to balance performance and resource use. Users must log in to Docker to run the sandbox, a requirement that has drawn some criticism but remains necessary.
This setup demonstrates a practical path for developers to leverage advanced LLMs locally on Macs, facilitating AI development without relying solely on cloud services.