How to Build a RAG-Powered Document Chat in Rails

What is RAG?

Retrieval-Augmented Generation (RAG) lets your AI answer questions using your own documents as context. Instead of relying on the model's training data, it searches your uploaded files and generates answers with citations.

How it works in the Rails AI Kit

1. Upload — User uploads a PDF, DOCX, or text file
2. Process — The document is chunked into paragraphs and each chunk is embedded using pgvector
3. Query — When a user asks a question, we find the most relevant chunks via vector similarity search
4. Generate — The relevant chunks are passed as context to the LLM, which generates an answer with citations

The tech stack

We use pgvector for vector storage (no external vector DB needed), ruby_llm for embeddings and chat, and Sidekiq for background document processing. The entire pipeline runs in your existing PostgreSQL database.

Try it yourself

Upload any document to the Documents section in your dashboard. Ask a question in the chat — the AI will search your docs and respond with sourced answers.