rag-client

A lightweight, portable RAG (Retrieval-Augmented Generation) node that answers questions over your documents — and instruments every stage of the pipeline so runs can be benchmarked across hardware and models.

Part of a two-repo system:

Repo	Role
rag-client (this repo)	Portable RAG service — runs on any machine (laptop, GPU server, cloud) and reports metrics home
rag-demo	Central hub — aggregates per-stage metrics from every node into Postgres and serves a live stats dashboard

How it works

PDF/TXT docs ─▶ chunking ─▶ sentence-transformers ─▶ Qdrant (vector DB)
                                                          │
question ─▶ embed ─▶ vector search ─▶ top-k chunks ─▶ OpenAI ─▶ answer
                                                          │
        per-stage metrics (embed / search / LLM latency, similarity, cost)
                                                          ▼
                                            rag-demo logging server

Every query and document upload is timed stage-by-stage — text extraction, chunking, embedding, vector search, and LLM generation — along with retrieval similarity scores and estimated cost. Tag each node with a RAG_HARDWARE_ID / LLM_HARDWARE_ID and the central dashboard can compare, e.g., a local llama3:8b on a GPU box against gpt-4o-mini over the API, on identical documents and questions.

Stack: FastAPI · sentence-transformers · Qdrant · OpenAI · Docker Compose

Endpoints

Method	Path	Purpose
POST	/upload	Upload a single PDF/TXT and index it
POST	/load-documents	Bulk-index everything in ./data
POST	/query	Ask a question; returns the answer + retrieval metadata
GET	/documents	List indexed documents
GET	/health	Liveness check

Setup

1. Clone the repo:

git clone https://github.com/al-gent/rag-client.git
cd rag-client

2. Create .env file:

cp .env.example .env
# Edit .env with your values

Required variables:

• RAG_HARDWARE_ID - Identifier for your hardware (e.g., laptop-mac, server-gpu)
• LLM_HARDWARE_ID - Where LLM runs (e.g., openai-api, local-gpu)
• MODEL_NAME - Which model to use (e.g., gpt-4o-mini)
• LOG_SERVER_URL - Remote logging server (a deployed rag-demo instance); omit to run standalone
• OPENAI_API_KEY - Your OpenAI API key

3. Start the system:

docker-compose up -d

4. Load documents:

# Put PDF or TXT files in ./data directory
curl -X POST http://localhost:8000/load-documents

5. Query:

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "Your question here"}'