Where does the work go?
Processing power does the work. Memory gives that work somewhere to sit. Think of a team of helpers and the table holding their supplies.
A cupboard is not a worktable
- Storage · the cupboard
Keeps files when the computer is off
- Working memory · the table
Holds data the program is using
- GPU calculation · the helpers
Uses that accessible data
The table and the cupboard
Memory holds information while the program uses it. Storage, such as an SSD, keeps files after the computer turns off.
A bigger cupboard does not make the table bigger. Likewise, free SSD space does not fix every GPU “out of memory” error.
When the work does not fit
Some programs split work into smaller pieces or move data between memory locations. That moving takes time. Other programs simply stop when they cannot allocate enough memory.
Separate GPU cards often have dedicated memory, called VRAM. Apple Silicon instead uses a memory system shared with the CPU. The arrangement differs, but both need working space.
Try it yourself
Your model runs out of GPU memory, but the SSD has a terabyte free. Is that contradictory?
Check your answer
No. Storage space and working memory are different resources. You need to understand the program’s memory demand, not just the size of its files.