🦙 Local RAG Agent with Llama 3.2
This open source application implements a fully local Retrieval-Augmented Generation (RAG) system built on Llama 3.2 served through Ollama, paired with Qdrant as the vector database. It is designed to run without external API dependencies, keeping document retrieval and answer generation on the user's own machine.
Why it caught our eye
Because retrieval and generation stay entirely local, the setup points toward a practical pattern for querying private document collections without sending content to third-party services. A user could, for example, load their own reference material and search it through the bundled AgentOS interface rather than an online service. This is an interpretation of the stated architecture, not a tested outcome.
What it does
The application implements a Retrieval-Augmented Generation pipeline: documents are managed and retrieved, Qdrant performs vector search, and Llama 3.2 via Ollama generates responses. An Agno v2.0 Knowledge class handles document management, and an interactive AgentOS web interface provides the user-facing way to work with the agent.
Who it is for
Given its requirements, it appears suited to users who want a self-hosted RAG setup and are comfortable installing and running local services. This user profile is an inference from the access requirements rather than an explicit statement in the source.
Access and setup
Running it requires Qdrant installed locally, Ollama with the llama3.2 and openhermes models pulled (OpenHermes serves as the embedder), and Python dependencies installed from requirements.txt.
Known limitations
On first run the knowledge base loads a sample Thai Recipes PDF; the loading call can be commented out afterward to avoid reloading it each time. This is the only limitation the provided facts describe.
Sources & freshness
- raw.githubusercontent.com Checked 2026-10-22
AI-assisted introduction based on public sources. We have not conducted a hands-on review. Check the official site for current access and pricing.