A Retrieval-Augmented Generation (RAG) pipeline is typically divided into two distinct phases: an offline data preparation/indexing phase and an online inference/querying phase. To ensure a near real-time user experience, operations that are computationally intensive and independent of the user's immediate query are performed offline in batch.

The generation of embeddings for the entire corpus of published content (A) and the subsequent creation or updating of the search index (C) are part of this offline phase. Since new content is added daily, these steps can be scheduled as a recurring batch job to process the new documents and update the vector store without impacting the performance of the live user-facing application. The remaining steps are part of the online inference pipeline that must execute quickly upon receiving a user query.

1. Lewis

P.

et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS) 33.

Section 3

"METHODS": The paper describes the non-parametric memory

which is "a dense vector index of Wikipedia

built using the Dense Passage Retriever (DPR)". This "building" of the index from the content corpus is an offline

prerequisite step before any queries are processed.

2. Stanford University

CS224N: NLP with Deep Learning | Winter 2023.

Lecture 16: "Retrieval and Question Answering": The lecture on Dense Passage Retrieval (DPR)

a common retriever in RAG

explicitly distinguishes between the offline and online processes. It details the "offline pre-computation" step where all passages (content) are encoded into embeddings and indexed. The "at test time" (online) process involves encoding the question and performing a fast nearest neighbor search.

3. AWS Documentation

"Retrieval Augmented Generation (RAG) on AWS".

Architectural Overview: Official AWS architectural patterns for RAG consistently show a "Data Ingestion and Processing" pipeline that is separate from the "Inference" pipeline. The ingestion pipeline

which runs on a schedule (e.g.

daily)

is responsible for chunking documents

generating embeddings

and loading them into a vector store like Amazon OpenSearch Service or Amazon Kendra. This is explicitly described as a batch process.