Run OpenAI's GPT-OSS Models on Your Laptop or Phone: What You Need and How

Overview of OpenAI's New AI Models

OpenAI has recently unveiled two new AI models, gpt-oss-20b and gpt-oss-120b. These models mark a significant development for the company as they are the first open-weight models since GPT-2. The gpt-oss-20b is more compact, while the gpt-oss-120b is significantly larger. Both models can be run locally on a desktop PC or laptop, without the need to connect to the internet or rely on cloud computing, provided the hardware is powerful enough.

Users can download either the 20b version or the 120b version and experiment with them on their computers. This allows users to explore how the model functions in a text-to-text format and understand its reasoning process step by step. Additionally, these models can be modified and built upon, although safety measures and censorship protocols will remain in place.

Hardware Requirements for Running the Models

To run these AI models, specific hardware requirements must be met. For the gpt-oss-20b model, which contains 21 billion parameters, a minimum of 16GB of RAM is required. This means that any laptop or PC with at least 16GB of system memory (or video RAM, or a combination of both) can run this model. However, having more memory than the minimum is recommended for smoother performance.

For the CPU, AMD recommends using a Ryzen AI 300 series processor paired with 32GB of memory, with half of it set to Variable Graphics Memory. For the GPU, AMD suggests any RX 7000 or 9000 model with 16GB of memory. While these are recommendations, they are not strict requirements. The key factor is ensuring there is enough memory available, ideally all on the GPU, to avoid performance bottlenecks.

Anecdotal reports suggest that the gpt-oss-20b can run smoothly on a MacBook Pro M3 with 18GB of RAM.

For the larger gpt-oss-120b model, which has 117 billion parameters, the requirements are much more demanding. A minimum of 80GB of RAM is needed, though it doesn't have to be entirely on the graphics card. This model is primarily designed for data center use, but it can also run on a high-end PC. Some users have reported running it on a system with 64GB of system memory and a 24GB graphics card, totaling 88GB of RAM.

AMD recommends a top-of-the-range Ryzen AI Max+ 395 processor with 128GB of system RAM, where 96GB is allocated as Variable Graphics Memory. This setup would typically require a high-end workstation laptop or desktop, possibly with multiple GPUs.

How to Run These Models on Your PC

Assuming your system meets the hardware requirements, you can run these models using Ollama, which is OpenAI’s preferred platform. You can download Ollama for Windows, Mac, or Linux from the official website. After downloading, simply double-click the executable file and follow the installation instructions.

Once installed, you can use the following commands in Ollama to pull and run the model:

ollama pull gpt-oss:20b
ollama run gpt-oss:20b

If you prefer an alternative to Ollama, you can use LM Studio with the command:

lms get openai/gpt-oss-20b

For Windows 11 or 10 users, another option is the Windows AI Foundry. However, this feature is still in preview, and it currently requires an Nvidia graphics card with 16GB of VRAM. Support for other GPUs like AMD Radeon is expected in the future.

macOS support for this feature is coming soon.

Considerations for Smartphones

Sam Altman mentioned that the smaller gpt-oss-20b model could run on smartphones, but this claim is somewhat ambitious. Qualcomm issued a press release about the model running on devices with Snapdragon chips, but this refers more to laptops, specifically Copilot+ PCs with Snapdragon X silicon, rather than smartphones.

While it might technically be possible to run gpt-oss-20b on a smartphone with 16GB+ RAM, the results may not be impressive. However, the trend toward running such models on mobile devices is likely to continue in the near future.