Topic 2: Where RAG helps and what to watch
Where RAG helps and what to watch
RAG becomes especially useful when AI needs to work with specific, trusted information rather than relying only on an LLM’s general knowledge.
For Product Managers and Business Analysts, this can mean helping people find information across product documents, requirements, research, policies and internal knowledge.
Where RAG helps
RAG can support several practical business scenarios.
1. Enterprise AI assistants
An organisation can build an AI assistant that answers questions using approved company knowledge. This makes the assistant more useful for organisation-specific questions.
2. Knowledge search
Instead of manually searching through multiple documents, users can ask a question and the system can retrieve relevant information from available sources.
3. Document Q&A
Teams can ask questions about policies, research, requirements or other documents. This can be particularly useful when working with large amounts of information.
4. Customer support
RAG can help customer support assistants answer questions using approved product information, support documentation and knowledge articles.
5. Internal copilots
Employees can use AI copilots that retrieve information from organisational knowledge to support everyday tasks and decisions.
What to watch
RAG can improve the quality and relevance of AI responses, but RAG does not guarantee accuracy.
Three important things to consider.
1. Source quality
The retrieved information needs to be reliable. If your source documents contain incorrect, outdated or incomplete information, the AI response may reflect those problems.
2. Retrieval quality
Having good information is not enough. The system also needs to find the right information for the question.
If it retrieves the wrong document or misses an important piece of information, the final answer can still be weak.
3. LLM output
Even when it retrieves the right information, the LLM still needs to interpret it and generate the response. It can misunderstand information, miss important details or produce an incorrect conclusion.
This is why human review still matters, particularly when the output affects important product, business or customer decisions.
What this means for PMs and BAs
You may not be responsible for building the technical RAG system, but you can play an important role in defining:
- what information the AI should have access to
- which sources should be trusted
- what users should be able to ask
- what a useful response should contain
- which outputs require human review
These are product and business decisions, not just technical ones.
DSM Tip
A useful way to think about RAG is:
Better Sources + Better Retrieval + Human Judgement = More Reliable Use of RAG
The technology matters, but so does the quality of the information and the decisions we make about how AI should use it.
