post
Quantization and the number of LLM parameters in practice – testing local models for writing prose (Q8 or 30B?) – part I
How much can you really squeeze out of local LLMs when you want to keep everything private: no cloud, no powerful GPUs, and no “magic” context that remembers everything? Instead of theorizing, I ran a series of tests on a regular laptop, treating quantization, parameter count, and RAG ac…
