Fill your skill gaps in AI and Data Science

External Tag: gguf

Harnessing Power at the Edge: An Introduction to Local Large Language Models

External Tags AI Deployment, AI Technology, ggml, gguf, gptq, large-language-models, local deployment, local large language models, machine-learning, model formats, nvidia chatrtx, Ollama, on-premise ai, privacy, pytorch, safetensors, technology-trends

Democratizing LLMs: 4-bit Quantization for Optimal LLM Inference

External Tags deep-dives, gguf, hugging face, llamaindex, model-quantization