LlamaEdge compatible quants for Llama 3.2 3B and 1B Instruct models.
Second State
company
AI & ML interests
Run open source LLMs across CPU and GPU without changing the binary in Rust and Wasm locally!
Organization Card
Run Open source LLMs and create OpenAI-compatible API services for the Llama2 series of LLMs locally With LlamaEdge!
Give it a try
Run a single command in your command line terminal.
bash <(curl -sSfL 'https://raw.githubusercontent.com/LlamaEdge/LlamaEdge/main/run-llm.sh') --interactive
Follow the on-screen instructions to install the WasmEdge Runtime and download your favorite open-source LLM. Then, choose whether you want to chat with the model via the CLI or via a web UI.
See it in action | GitHub | Docs
Why?
LlamaEdge, powered by Rust and WasmEdge, provides a strong alternative to Python in AI inference.
- Lightweight. The total runtime size is 30MB.
- Fast. Full native speed on GPUs.
- Portable. Single cross-platform binary on different CPUs, GPUs, and OSes.
- Secure. Sandboxed and isolated execution on untrusted devices.
- Container-ready. Supported in Docker, containerd, Podman, and Kubernetes.
Learn more
Please visit the LlamaEdge project to learn more.
models
175
second-state/functionary-medium-v3.1-GGUF
Updated
•
373
second-state/functionary-small-v3.1-GGUF
Updated
•
169
second-state/functionary-small-v3.2-GGUF
Updated
•
451
second-state/FinGPT-MT-Llama-3-8B-LoRA-GGUF
Text Generation
•
Updated
•
232
second-state/EXAONE-3.0-7.8B-Instruct-GGUF
Text Generation
•
Updated
•
106
•
1
second-state/FLUX.1-dev-GGUF
Text-to-Image
•
Updated
•
867
•
1
second-state/Llama-3.2-3B-Instruct-GGUF
Text Generation
•
Updated
•
944
•
1
second-state/Qwen2.5-3B-Instruct-GGUF
Text Generation
•
Updated
•
319
second-state/Llama-3.2-1B-Instruct-GGUF
Text Generation
•
Updated
•
752
second-state/FLUX.1-schnell-GGUF
Text-to-Image
•
Updated
•
2.2k
•
5
datasets
None public yet