A RAG pipeline you can explain
A RAG pipeline you can explain. In this video you'll build an assistant that answers from a company handbook, and shows its sources. Retrieval-augmented generation, or RAG, works in two steps. First, the handbook is cut into chunks, and each chunk becomes a vector in Chroma. Then a question finds its closest chunks, and the model answers from them, citing each one. In this lesson you will index the handbook, ask it questions, and use its web page. Before you start, you need Ollama with two models, and the starter repository's virtual environment. make check tests both. Ollama answers on this Mac, and both models are there. gemma4, size e4b, is a small chat model from Google. embeddinggemma turns a piece of text into a vector, a list of numbers that captures its meaning. The knowledge is eight Markdown pages of a company handbook. Each page starts with a title and a team, and is split into sections with their own heading. index.py reads every page, cuts each section into chunks of up to six hundred characters, and puts the page title and the section heading in front of each chunk. Then it asks embeddinggemma for one vector per chunk, and stores text, vector and metadata in Chroma, a vector database that keeps everything in a folder of the repository. Twenty-seven chunks, all new. Now a question. The show flag also prints the chunks that came back. ask.py turns the question into a vector, and Chroma returns the closest chunks with a score: the cosine similarity, one for the same meaning, zero for unrelated text. The model receives only these chunks, numbered, and has to cite the ones it uses. That is the whole instruction: answer only from the numbered sources, cite them, and if they do not contain the answer, say so. Here is a question the handbook cannot answer. I don't know, and no source. No chunk reached the minimum score, so the app did not even call the model: nothing to read means nothing to make up. The same app also has a web page. make app starts it in the background. This is the web page of the same app, app.py in the repository. It is built with Gradio, a Python library for small web pages, and it listens on one two seven dot zero dot zero dot one only. The answer comes from the on-call page of the handbook, and cites it as source one. Under it, the sources table: the file, the section, and the similarity score of each chunk. Anyone who doubts the answer can open the page and check it. The team filter and the two check boxes switch on the same options as the command line. Documents, chunks, vectors, the closest chunks, and an answer that names them: every step is a file you can open, and a command you can run again. On OpenShift AI, the models are served by vLLM on KServe, and Llama Stack offers the same retrieval steps, with a vector database such as Milvus. In this lesson you learned to index documents into a local vector store, to answer from the closest chunks with their sources, and to say I don't know when nothing matches.