Topic 1: RAG (Retrieval-Augmented Generation) Quick overview
RAG (Retrieval-Augmented Generation) Quick overview
Large Language Models are powerful, but they do not automatically know everything about your organisation, your products or the latest information you are working with.
This is where RAG, or Retrieval-Augmented Generation, becomes useful.
RAG helps an LLM find and use relevant information from trusted sources before generating its response. Instead of relying only on what it already knows, you give it access to information relevant to the question.
Think of RAG like giving AI reference material
Imagine asking an AI assistant a question about your organisation’s product roadmap.
Without access to your actual roadmap, the AI can only give you a general response based on what it already knows.
With RAG, the system first finds relevant information from your product documents and then gives it to the LLM.
The LLM can now answer using your information as additional context.
In simple terms:
Question → Find relevant information → Give it to the LLM → Generate the answer
Where does the information come from?
The information RAG retrieves can come from different trusted sources.
For Product Managers and Business Analysts, these might include:
- product documentation
- business requirements
- customer research
- policies and procedures
- knowledge bases
- product roadmaps
- user stories and backlogs
- internal reports
The important point is that the LLM is not expected to know this information already. The RAG system retrieves relevant information when it is needed.
A simple PM example
Imagine you ask an internal AI assistant:
“What are the main customer problems we are trying to solve in the onboarding journey?”
A normal LLM may not know anything about your organisation’s onboarding research.
A RAG-enabled assistant could search your approved customer research, product documentation and requirements, retrieve the most relevant information, and provide it to the LLM.
The LLM can then generate an answer based on that information.
A simple BA example
A Business Analyst might ask:
“What are the current requirements for customer identity verification?”
Instead of searching manually across several documents, a RAG-enabled assistant could retrieve relevant sections from approved requirements, policies and project documents and use them to help answer the question.
This is one reason RAG is becoming useful in enterprise AI applications where organisations have large amounts of internal knowledge.
Why RAG matters
RAG can help make AI responses more relevant and grounded in the information that matters to your organisation.
It is particularly useful when the information is:
- private to your organisation
- spread across multiple documents
- specific to a product or project
- regularly updated
- not part of the LLM’s built-in knowledge
But RAG does not automatically make every answer correct. The quality of the response still depends on whether the system finds the right information and whether that information is accurate and up to date.
DSM Tip
Think of RAG as giving the LLM the right reference material before it answers.
You do not need to understand the technical architecture to understand its value. For a PM or BA, the important question is often: What trusted information should the AI use to answer this question?
