Short answer: GLM-5.2 does not fit on any consumer GPU. Z.ai’s 744B-parameter MoE model needs about 245 GB of combined memory even at 2-bit quantization. Your realistic options are a 256 GB unified memory Mac Studio, a 24 GB GPU plus 256 GB of system RAM with MoE offloading, a multi-GPU workstation north of $40,000, or renting cloud GPUs by the hour. This guide covers the exact numbers for each path.
Quick Navigation:
GLM-5.2 Architecture
GLM-5.2 landed in mid-June 2026 with full open weights under an MIT license, and it immediately became the model everyone wants to self-host: frontier-level coding and reasoning performance with no usage restrictions. The catch is its size.
GLM-5.2
The MoE design means only 40B of the 744B parameters fire per token, so inference speed is closer to a 40B dense model. But memory does not work that way: all 744B parameters must be loaded, because different tokens route to different experts. That single fact shapes every hardware decision below. If you want the deeper background on why, our LLM quantization guide covers how model size maps to memory.
Memory Requirements by Quantization
These figures are for Unsloth’s dynamic GGUF quantizations, currently the standard way to run GLM-5.2 on non-datacenter hardware. “Total memory” means VRAM plus system RAM combined, since llama.cpp can split MoE layers between GPU and CPU.
| Quantization | Total Memory Needed | Quality | Realistic Hardware |
|---|---|---|---|
| 1-bit (UD-IQ1_S) | 223 GB | About 76% of full accuracy | 256 GB RAM workstation |
| 2-bit (UD-IQ2_M) | 245 GB | About 82% of full accuracy | 256 GB Mac Studio or GPU + 256 GB RAM |
| 3-bit | 290-360 GB | Good | 384 GB+ multi-GPU or RAM |
| 4-bit (UD-Q4_K_XL) | 372-475 GB | Near lossless | 4x 96 GB GPUs or 4x H200 |
| 8-bit (UD-Q8_K_XL) | 810 GB | Reference quality | 8x H200 node |
Key insight: the dynamic 2-bit quant is the local sweet spot. It is 84% smaller than the full model while keeping roughly 82% of its accuracy, because Unsloth keeps the important layers at higher precision. Below 2-bit, quality falls off fast for coding work.
Four Ways to Run GLM-5.2
Path 1: Mac Studio with 256 GB Unified Memory
The simplest turnkey option. An M4 Ultra Mac Studio with 256 GB unified memory runs the 2-bit GGUF through llama.cpp’s Metal backend at roughly 3 to 6 tokens per second. Slow for agentic coding loops, workable for chat and single-shot tasks. Around $6,000 to $7,500 depending on configuration.
Path 2: One 24 GB GPU + 256 GB System RAM
llama.cpp can keep the attention layers and active experts on a single RTX 4090 or 5090 and stream MoE expert weights from system RAM. You need a platform that actually takes 256 GB, which usually means workstation boards or high-end desktop with 64 GB DIMMs. Expect single-digit tokens per second, bottlenecked by RAM bandwidth. Our RAM for local LLMs guide covers exactly this setup.
Path 3: Multi-GPU Workstation (384 GB VRAM)
The setup that made the rounds on Hacker News: four RTX PRO 6000 Blackwell cards at 96 GB each, serving a mixed-precision GLM-5.2 build through vLLM at around 80 tokens per second with 400k+ context. All-in cost lands around $50,000, with the GPUs alone at $46,000. This is genuine frontier-model performance at home, priced like a car.
Path 4: Rent Cloud GPUs (Most People Should Start Here)
A 4x H200 node (564 GB VRAM) runs the near-lossless 4-bit quant at interactive speeds, and H200s rent for a few dollars per GPU-hour. A full weekend of experimentation costs less than 1% of the workstation above. Vast.ai usually has the lowest hourly prices on multi-GPU configurations, and RunPod gives new users a $5 credit to test with. See our cloud GPU comparison for current pricing.
The AMD Option
One of the more interesting GLM-5.2 stories is happening on the datacenter side. Wafer published production benchmarks of GLM-5.2 on AMD’s MI355X running SGLang with ROCm and MXFP4 quantization: 2,626 tokens per second per node on a realistic serving workload, at more than 2x lower cost than an equivalent NVIDIA Blackwell deployment, since the AMD cards run about 2.75x cheaper per GPU.
For a home rig this changes nothing yet, the software stack is still datacenter-grade tuning. But it matters for cloud renters: as providers add MI300-series and MI355X capacity, price per token for big MoE models like GLM-5.2 should keep falling. If your workload runs on vLLM or SGLang, it is worth checking AMD instance pricing before defaulting to H100s or H200s.
Pro tip: whichever path you pick, quantize the KV cache. Running llama.cpp with —cache-type-k q4_1 —cache-type-v q4_1 stretches your usable context window roughly 3x at the same memory budget, which matters a lot when the base model already ate 245 GB.
FAQ
Can you run GLM-5.2 on an RTX 4090 or 5090?
Not on the GPU alone. GLM-5.2 needs around 245 GB of combined memory even at 2-bit quantization. A single 24 GB or 32 GB card works only as part of a MoE offloading setup with 256 GB of system RAM, and you should expect single-digit tokens per second.
How much VRAM does GLM-5.2 need?
The full FP16 model is about 1.5 TB. Practical quantized versions range from 223 GB at 1-bit to roughly 475 GB at 4-bit. The sweet spot for local use is Unsloth’s dynamic 2-bit GGUF at around 245 GB of combined VRAM and RAM, which retains about 82 percent of full accuracy.
What is the cheapest way to run GLM-5.2 locally?
A single 24 GB GPU paired with 256 GB of system RAM using llama.cpp MoE offloading is the cheapest entry, at roughly $2,500 to $3,500 in upgrades if you already own the GPU. A 256 GB unified memory Mac Studio is the simplest turnkey option. Both deliver about 3 to 6 tokens per second.
Is it better to run GLM-5.2 in the cloud?
For most people, yes. A rented multi-GPU node runs a 4-bit quant at interactive speeds for a few dollars per GPU-hour, with no upfront cost. Local only wins if you need it running around the clock, or your data cannot leave your machine.
Related Reading
Building for Big Local Models?
Need a workstation that can hold 256 GB of RAM? Check our Tailored Builds page.