Hardware Guide for Local LLMs: Bypass VRAM Limits with GPU, CPU, Memory

Run Gemini 2.5 Flash at near‑native speed on a local RTX 5090 (32 GB) by loading the 4‑bit‑quantized 32‑B model with vLLM in WSL2 Ubuntu 24.04 using CUDA 13.1, proper drivers, and adequate cooling. Install CUDA 13.1, PyTorch, and bitsandbytes, watch nvidia‑smi for OOM, and keep the context length modest.