About the NVIDIA RTX 4090
The NVIDIA RTX 4090 is the most popular GPU for hobbyist and indie AI work — Stable Diffusion image generation, small-model fine-tuning, local LLM inference. It's a consumer card (not officially supported by AWS/GCP/Azure) but widely available via community-tier providers like Vast.ai, RunPod, and Salad at extremely low prices. 24GB VRAM fits most consumer AI workflows.
Specs
Memory
24 GB GDDR6X
Bandwidth
1.0 TB/s
CUDA cores
16,384
FP16 (peak)
330 TFLOPS
Architecture
Ada Lovelace
Released
Q4 2022
What's it good for?
- Stable Diffusion / SDXL — image generation at fast iteration speeds.
- Llama 3 8B fine-tuning — QLoRA / LoRA fits comfortably in 24GB.
- Local LLM inference — run 7B-13B models at full precision, or 30B+ quantized.
- Hobbyist / learning — by far the most cost-effective way to experiment with modern AI.
When to use RTX 4090 vs alternatives
- RTX 4090 vs RTX 3090: 4090 is ~70% faster, same 24GB VRAM. The 3090 is cheaper if you find it; 4090 wins on speed-per-dollar at typical rental rates.
- RTX 4090 vs A10: 4090 is dramatically faster, similar VRAM. Use A10 only for cheap inference; 4090 for anything that needs training speed.
- RTX 4090 vs A100 80GB: A100 has 3× the VRAM and ECC memory. Use A100 for models that exceed 24GB or for production reliability; 4090 for experimentation.