llama-quantize command. --run is required to
execute it, and the output directory is created automatically. Install a
llama.cpp build exposing llama-quantize before running conversions.Models & inference
Local model quantization
Create smaller GGUF models with a safe preview-first workflow.
Related topics
Local model optimization and setupIntegrationsSecurity model and prompt-injection defensesArka — AI terminal agent documentationAI agent guide — use Arka over MCPWas this page helpful?