The most advisable and direct approach among the options is to use the system prompt to set explicit rules for the model's behavior. By instructing the application not to answer questions unrelated to SnoPen AI, you are implementing a "guardrail." This constrains the language model's generation step, ensuring that even if the retriever mistakenly fetches irrelevant context from the noisy source documents, the final output remains focused on the intended topic. This is a standard and effective prompt engineering technique for controlling the scope and relevance of a RAG application's responses.

A. Keeping all articles pollutes the vector index with irrelevant data, which degrades retrieval quality, increases the chance of hallucination, and raises operational costs.

B. Forcing the model to assume all retrieved context is about SnoPen AI is dangerous; it will directly lead to factual inaccuracies and hallucinations when irrelevant content is retrieved.

D. Consolidating all documents into a single chunk is technically incorrect for a RAG system. It makes targeted retrieval impossible and would likely exceed the model's context window.

1. Databricks Official Documentation, "Query foundation models": The documentation for the Foundation Model APIs explains the systemprompt parameter. It states, "A system prompt can be used to specify the persona and rules for the model." This directly supports using the system prompt to enforce a rule like not answering off-topic questions.

Source: Databricks AI and Machine Learning Documentation > Foundation models > Query foundation models.

2. Databricks Blog, "Building High-Quality RAG Applications": This official blog post discusses techniques for improving RAG quality. It emphasizes the importance of the prompt template in guiding the model, stating, "The prompt template is critical for instructing the LLM on how to use the provided context to answer the user’s question... It can also include instructions on what to do if the answer is not in the context." This principle directly applies to instructing the model not to answer questions outside its designated scope (SnoPen AI).

Source: Databricks Blog, "Building High-Quality RAG Applications", section on "Prompt Engineering".

3. Gao, Y., et al. (2023). "Retrieval-Augmented Generation for Large Language Models: A Survey." This academic survey on RAG discusses the generator component. It highlights that the generator's role is to synthesize an answer based on the prompt and retrieved documents. Controlling the generator through carefully crafted prompts (prompt engineering) is a key technique to ensure the final output is relevant and faithful to the provided context.

Source: Gao, Y., et al. (2023). "Retrieval-Augmented Generation for Large Language Models: A Survey." arXiv preprint arXiv:2312.10997, Section 4.2 "Generator". DOI: https://doi.org/10.48550/arXiv.2312.10997