Local development of large language models (LLMs) has reached a practical stage with the availability of efficient models like Qwen 3.8 and Gemma 4. These models can now be used to build web applications directly on personal machines. A MacBook Pro with Apple’s M5 chip and 48GB of RAM is considered an optimal setup, capable of running 30 billion parameter models with a sizable context window.
Ollama is favored for this setup due to its ease of use, speed on Apple silicon, a solid model directory, and stability. Opencode serves as a reliable harness for running these models locally. Additionally, Docker sandboxes (sbx) are employed to isolate the environment, preventing potential issues caused by model hallucinations or other unexpected behavior.
To get started, users install Docker Sandbox and Opencode via Homebrew, and download Ollama from its official website. Once Ollama is running, the Qwen 3.8 27B mxfp8 model and Gemma 4 31B mxfp8 model can be pulled for use. For systems with less than 48GB RAM, lighter versions of these models are available.
The setup requires creating a sandbox kit to configure the environment, including permissions and environment variables to allow Opencode to communicate with Ollama running on the host Mac. The configuration limits the Qwen model’s context to 64K tokens to prevent memory overuse.
Running Opencode is done through the sbx command, specifying the sandbox kit. Users should adjust model settings to lower reasoning effort levels to optimize performance. Note that Docker login is required to use sbx, a step that some developers find inconvenient but is currently necessary.
This approach enables developers to experiment with large language models locally, leveraging Apple silicon’s capabilities while maintaining system stability and security through containerization.