When working with large language models, achieving accurate, relevant results depends on several factors. 

Two of them are Retrieval-Augmented Generation (RAG) and prompt engineering. 

RAG helps generate better answers by allowing the generative AI model to retrieve information from external knowledge sources, keeping responses fact-based and up to date. 

Prompt engineering, on the other hand, focuses on how you communicate with the AI to produce clear and context-aware answers. 

* RAG adds new knowledge without retraining, whereas prompt engineering improves the model’s use of its existing knowledge.
* RAG works well for research, legal, healthcare, and customer support; prompt engineering is well-suited for summarization, creative writing, Q&A, and data formatting.
* RAG is complex and can be slower, while prompt engineering is cheaper but requires skill.
* RAG requires more setup and maintenance, but prompt engineering is faster and cost-effective.

Choosing the right approach depends on what you want: accuracy, freshness, or speed. Often, combining both gives the best results. Let’s dig deeper into this. 

## What is prompt engineering?

Prompt engineering is the art of crafting clear instructions for large language models (LLMs) to achieve the best possible results. 

Think of it like learning to ask the right questions. A vague prompt might produce a generic answer, while a carefully worded one can give you precisely what you need. 

Prompt engineering is about understanding how the gen AI model interprets instructions and using that insight to guide it toward your desired output. 

## What is RAG (retrieval-augmented generation)?

[RAG](https://www.meilisearch.com/blog/what-is-rag) is a technique that improves LLM outputs by providing them with access to external resources. This could include databases, documents, and APIs. Instead of relying solely on what they were trained on, RAG allows AI models to access additional information when generating answers. 

For instance, we can give an AI model both memory and internet search skills, so that it knows today’s date and searches the internet for the most recent data.

## How does prompt engineering work?

Prompt engineering is about learning how to interact with an artificial intelligence model in a way that generates the best possible response.

As a rule of thumb, it is good practice to provide the LLM with enough context to answer your questions. LLMs are trained to follow user instructions, and they do so within the context they have. 

For example, if you ask an LLM to act like a hip-hop artist, its response will sound like one. Ask it to write like a Nobel Prize–winning author, and it will adapt its responses accordingly.

Finding the right prompt involves experimentation. If the answer from an initial prompt is not accurate, you can refine your prompt to provide more context and make it clearer. You may also check for noise or contradictory statements and eliminate them. 

For example, if you do not have a background in science, instead of asking the model to “Explain photosynthesis,” you could say, “You have over a decade of experience teaching biology. Explain photosynthesis like you are teaching someone in eleventh grade.” The goal is to give the AI enough context and direction to produce relevant responses.

## How does RAG work?

RAG works by combining retrieval and generation so that an LLM can answer using external facts. 

In the retrieval phase, RAG searches external sources to find the most relevant information, and in the generation phase, the model uses that retrieved data to craft an accurate response. 

![How RAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_e26007ec0c.png)

Here is how the process goes, step-by-step:

1. A user types in a search query. 
2. A retriever searches a document store or database for information that matches the query. 
3. The information is ranked, and the best is selected to form a context. 
4. The original question and the context are sent to the LLM. 
5. The model generates a response grounded in the retrieved information. 
6. Finally, a verifier or re-ranker checks the model's outputs before they are returned. This is optional.

## What are the main differences between RAG and prompt engineering?

The main difference between RAG and prompt engineering is how they help AI models produce better answers. 

Different [RAG types](https://www.meilisearch.com/blog/rag-types) improve accuracy by allowing the model to retrieve real-time data before generating a response. 

Prompt engineering focuses on phrasing and optimizing questions to get the best result from the model’s existing knowledge. 

To sum it up, RAG gets its information externally, while prompt engineering uses information already within the AI model.

![RAG vs. Prompt Engineering.png](https://unable-actionable-car.media.strapiapp.com/RAG_vs_Prompt_Engineering_a00885536d.png)

Now, let’s discuss the benefits and challenges of using RAG and prompt engineering.

## What are the advantages of prompt engineering?

Prompt engineering can help users get the most out of AI tools without requiring additional data. 

Here are some of the advantages:

* **Better control over responses:** You can guide the AI model to produce more relevant outputs by learning how to phrase questions effectively. For example, a marketer can ask ChatGPT to ‘act like an expert marketer in a Fortune 500 company and write an email to promote a new product,’ and receive a good draft email immediately.
* **Works across different tasks:** Prompt engineering can be applied across various sectors. It is helpful for specific tasks such as coding, summarizing reports, writing essays, and even brainstorming ideas. 
* **Saves time and effort:** Making your input prompts clearer reduces the need for multiple iterations. Instead of constantly reworking the answer, a well-crafted prompt helps the AI deliver the desired result in a single go.
* **Encourages creativity and personalization:** Prompt engineering gives users the freedom to set their own tone or style. This makes it an excellent tool for writers, designers, or educators who desire unique results.


## What are the advantages of RAG?

RAG has several benefits that make AI responses more helpful in real-life situations.

* **Improves accuracy:** RAG reduces the risk of errors by pulling information only from verified external sources.
* **Access to up-to-date information:** RAG can retrieve current information from live databases or documents. This makes it perfect for fast-changing topics, such as the news.
* **Better context understanding:** RAG provides the AI model with additional background information before generating an answer, helping it better understand the question and respond more accurately.
* **Flexibility across domains:** Because it can connect to various data sources, RAG can easily adapt to resource-intensive, specialized fields like healthcare, agriculture, or finance while still delivering insights tailored to each domain.
* **Reduced AI model training costs:** Since RAG relies on retrieval rather than constant retraining, organizations can update knowledge from data sources rather than rebuilding the entire model for each process.

## What are common challenges with prompt engineering?

Prompt engineering is a powerful skill, but it is not always straightforward. Generating the perfect response from AI often takes creativity and practice. 

Here are some common challenges:

* **Inconsistency in results:** Just a slight change in wording can completely alter the AI’s answer. Also, because AI is non-deterministic, what works once may not work the same way next time, making it hard to achieve consistent responses.
* **A need for expertise:** Crafting a strong prompt requires some level of domain knowledge and an understanding of how the AI model interprets instructions. Newbies may give generic prompts, leading to generic AI responses.
* **Can be time-consuming:** Refining your prompts through trial and error can take a long time, especially when dealing with complex tasks. If not properly managed, this can slow down your workflow or even reduce your productivity.
* **Model limitations:** AI systems only know the training data on which they were built. Therefore, even the best prompt cannot fix a model whose knowledge base is lacking. Such situations lead to hallucinations or outdated responses.

## What are the common challenges with RAG?

Some challenges can make RAG implementation difficult, even though it greatly improves accuracy and relevance. 

Here are some of them:

* **System complexity:** RAG can be complex to set up and maintain, especially for non-technical teams. It requires [connecting a retriever and a database to the LLM](https://www.meilisearch.com/blog/how-do-you-search-in-a-database-with-llm).
* **Integration overhead:** Integrating a RAG system into an existing application or workflow often requires extra resources. You have to set up the RAG infrastructure while maintaining the external data sources at the same time.
* **Latency and speed issues:** Balancing accuracy with performance can be challenging with RAG. Since it searches and retrieves data before generating a response, it can be slower than the traditional models, especially when handling large datasets. 
* **Information retrieval error:** Sometimes a RAG system retrieves the wrong documents, leading to answers that sound confident but are based on inaccurate data. This makes proper data indexing and filtering critical when setting up the RAG system.

## What are examples of prompt engineering use cases?

Prompt engineering can be applied in many real-world situations to make AI tools more valuable and efficient: 

* **Question answering:** Businesses use prompt engineering to make chatbots and help desks smarter. With the right prompt, an AI agent can provide clear and accurate responses to customer or employee questions.
* **Text summarization:** It is excellent for turning long text into short but meaningful takeaways. For instance, you can say, ‘Summarize this report in three simple points,’ and save hours of reading time.
* **Data conversion:** Prompt engineering can help you quickly clean or restructure information. For example, changing your raw notes into a professional report.
* **Creative writing:** Writers use prompts to get innovative ideas, explore storylines, or overcome creative block.

## What are examples of RAG applications?

[RAG tools](https://www.meilisearch.com/blog/rag-tools) are powering many of the apps we use today. Here are a few ways RAG is being used in the real world:

* **Customer support systems:** Many customer service chatbots now use RAG to fetch real answers from company manuals. So when you ask your bank’s chatbot about your account details, it gives you the exact information instead of a generic reply.
* **Legal research:** Law firms use RAG to review thousands of case files and contracts in seconds. A lawyer can ask about precedents for a case and instantly get relevant examples instead of manually searching the archives.
* **Healthcare and medical support:** Doctors and researchers use RAG to find the latest studies, treatment guidelines, and patient data. This enables them to make more informed medical decisions.
* **E-commerce product comparison:** RAG is used by e-commerce companies to compare products. It retrieves product specs, reviews, or prices from the database and generates personalized recommendations.

## Can RAG and prompt engineering be combined?

Yes, RAG and prompt engineering work better together. 

For example, when someone asks, ‘Summarize this customer’s issue in three bullet points,’ a RAG setup might pull a customer’s record from the database, and a good prompt allows the LLM to summarize only the issues returned. 

In legal work, RAG can find relevant cases, which are then combined with a prompt that says, ‘Compare these rulings and highlight the key differences.’ This provides a concise, actionable comparison rather than a long list that might be difficult to interpret. 

In healthcare, RAG retrieves patient history and recent medical studies, while prompts ensure the results follow clinical reporting standards.

RAG handles the ‘what’ (or the facts), and prompt engineering handles the ‘how’ (or the presentation). When used together, they make answers both relevant and accurate.

## What are the costs of RAG vs. prompt engineering?

The costs of setting up RAG and prompt engineering are different. 

RAG can be expensive to implement because it requires additional infrastructure (such as a database, a retrieval system, and integration tools). There is also a maintenance cost for updating data sources, handling storage, and ensuring the retrieval system stays accurate and fast. 

Prompt engineering, on the other hand, is far cheaper because it mainly involves using your time and skill. You have to learn how to design effective prompts and also test them for consistent results. There is no need for new servers, APIs, or databases, making it suitable for quick, low-cost projects.

RAG requires more upfront investment but delivers higher accuracy, while prompt engineering is cost-effective and easily scalable for smaller applications.

Want to decide whether to use RAG or prompt engineering? Let’s see the factors you should consider when choosing.

## How do you choose between RAG and prompt engineering?

Choosing between RAG and prompt engineering depends on what you need from your system and the kind of resources you have. 

Here is a simple decision guide to use:

* **Check if you need fresh data:** RAG is the way to go if your answers must include the latest information. But if you are working with general knowledge that doesn't change often, prompt engineering should be sufficient.
* **Weigh your accuracy and reliability needs:** For fields like healthcare, law, or finance, where mistakes can be costly, RAG is safer because it cites real sources. Prompts alone cannot always guarantee accuracy. 
* **Consider your time, budget, and skills:** RAG takes more setup and maintenance, while prompt engineering requires creative thinking and practice. If you have limited resources, then go with prompt engineering.
* **Balancing performance and workflow:** RAG systems can be slightly slower since they search external data before responding, but the answers are usually more accurate. Prompt engineering is quicker and is perfect for lightweight applications where speed matters more than depth.

## How does RAG compare to fine-tuning?

The main difference between [RAG and fine-tuning](https://www.meilisearch.com/blog/fine-tuning-vs-rag) is that RAG pulls in fresh information from external sources when needed, while fine-tuning actually trains the model to learn that knowledge internally.

With fine-tuning, you take a pre-trained model and feed it domain-specific data (such as legal documents or customer records) so that it learns the patterns and language of that field. This makes it faster and consistent. 

The limitations are that it is expensive to train, requires lots of clean data, and must be redone whenever new information comes in.

RAG, on the other hand, does not retrain the model. Instead, it connects to a database or document store and retrieves the most relevant information whenever a question is asked. This ensures the answers are up to date and adaptable to new topics.

## How does prompt engineering compare to fine-tuning?

The main difference between prompt engineering and fine-tuning is that prompt engineering focuses on how you ask the model your question to get the best response, while fine-tuning changes what* *the model actually knows.

With prompt engineering, you are not retraining anything. You are instead crafting the proper instructions to guide the model’s behavior. It is fast, affordable, and flexible, making it perfect for tailoring outputs to specific needs. 

The catch is that results can vary greatly depending on how well the prompts were written.

Fine-tuning, on the other hand, takes things deeper. You train the model on new data so it permanently learns that knowledge. This gives you consistency and expertise but comes with higher costs, more setup time, and the need to continuously retrain to stay up to date.

## RAG vs. prompt engineering: and the winner is...

The reality is that both techniques shine in their own way. 

RAG is your go-to tool when accuracy and up-to-date information matter most, since it can retrieve from live data sources. Prompt engineering is the fastest, most flexible way to generate outputs without retraining any model.

If you are building something that needs freshness, go with RAG. For quick, creative control, prompt engineering takes the lead. 

RAG vs. prompt engineering: choosing the right approach for you

GraphRAG vs. Vector RAG: Side-by-side comparison guide

We're kicking off [Launch Week](https://www.meilisearch.com/launch-week-q3-2025) with something we've been excited to share: Meilisearch Chat, a new way to make your data conversational.

For the first time, you can turn your existing Meilisearch index into an AI-powered chat experience – without complex infrastructure, separate vector databases, or months of custom development.

## Search is becoming a conversation

Your users and buyers don't want to search anymore – they want to ask. They expect apps, documentation, and internal tools to understand natural language and give them direct answers, not endless lists of results.

Until now, that's been really hard to deliver. Traditional conversational AI required juggling multiple LLM calls, vector databases, reranking pipelines, and endless tuning. The complexity alone made it expensive and slow, often taking months just to get a working prototype.

Meilisearch Chat changes all of that.

## One API call, complete conversational AI

The new `/chat` brings AI-native search to your Meilisearch data through a single API endpoint. It handles the entire Retrieval-Augmented Generation (RAG) workflow, from understanding the query to generating the response, automatically.

Here's what you get:

**Conversational AI for search** – Users ask natural-language questions and get precise, contextual answers from your Meilisearch index.

**Built-in RAG capabilities** – Retrieves relevant information, contextualizes it in your factual data, and generates accurate responses. No extra vector database, reranker, or separate AI service required.

**Native integration** – Runs directly on your Meilisearch instance with a single, OpenAI-compatible endpoint.

Less infrastructure. Faster implementation. No compromise on quality or accuracy.

## Why this matters for your business

**Get to market faster**

Meilisearch Chat turns months of AI integration work into days. You can prototype conversational experiences immediately, test them with real users, and iterate quickly without needing a dedicated AI team.

**Lower costs, less complexity**

We've streamlined the entire RAG pipeline, so you don't need separate LLM orchestration, reranking logic, or vector storage. Fewer moving parts means lower maintenance costs and less things that break.

**Better user experiences**

Whether it's customer support or internal knowledge bases, your users get direct, reliable answers in context – not just a list of links. That means faster resolution times, higher engagement, and happier users.

## Built for real-world use cases

**Retail & e-commerce** – Help shoppers find exactly what they need through natural conversation.

**Customer support** – Let users ask questions in plain language and get answers directly from your documentation or help center.

**Internal knowledge bases** – Turn your company documentation into a chat assistant that answers questions instantly.

**Developer docs** – Add conversational search to your API or SDK documentation so developers can find what they need faster.

## Simple to implement, easy to scale

If you're already using Meilisearch, adding conversational capabilities is straightforward. The `/chat` endpoint is [OpenAI-compatible](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat#configure-a-chat-completions-workspace), which means it works with your existing LLM setup and fits right into your current workflows.

You can start with your existing indexes – no need to rebuild or re-embed your data.

## A new approach to AI-powered search

With Meilisearch Chat, we're making it simpler to bring conversational AI to your data. Instead of managing multiple services and complex pipelines, you get a complete conversational search experience through one endpoint.

This isn't just another feature – it's a new foundation for more natural, human-like interactions with your data.

**Ready to make your search conversational? Explore the [docs](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat) for how to get started and [try /chat today](https://www.meilisearch.com/cloud)**

Introducing Meilisearch Chat: conversational search for your data

When building AI systems, one of the most significant decisions to make is how your model accesses and uses knowledge. Do you need real-time data every time, or can you rely on pre-stored, cached information? 

This is where the difference between retrieval-augmented generation (RAG) and cache-augmented generation (CAG) lies: 

* RAG retrieves live and updated data, making it ideal for situations where changes occur quickly and accuracy is crucial.
* CAG relies on cached information, delivering answers faster and more cheaply. It works well for repeated or predictable questions.
* RAG provides fresh, reliable responses and can handle various queries. However, it can be slow, expensive, and requires a solid infrastructure to run smoothly.
* CAG is quick, efficient, and easy to scale, but it can get outdated. It is also limited to what is already stored and requires regular updates to stay useful.

Choosing between the two comes down to considering your users: 

* Are the questions predictable or wide-ranging? 
* How up-to-date does the information need to be?
* What is your budget and infrastructure like?

You do not always have to pick just one. Hybrid approaches can combine both RAG and CAG. They allow the speed of cached answers while still retrieving live data when needed.

By the end of this article, you will understand how RAG and CAG work and which one better suits your projects.

## What is RAG (Retrieval-Augmented Generation)?

[RAG is a technique](https://www.meilisearch.com/blog/what-is-rag) that enhances the capabilities of generative AI by using external knowledge to provide more accurate responses to user queries. 

Rather than solely relying on the datasets the AI model was trained on, RAG searches a vector database or docs, retrieves the most relevant information, and then uses it to generate a response. This makes the answers more accurate and up-to-date. 

## What is CAG (Cache-Augmented Generation)?

CAG builds responses around the specific context window you provide. 

Unlike RAG, which searches external databases for information, CAG uses past interactions to shape its answers. This makes its output feel more relevant and personalized. 

In short, CAG makes AI less generic and more tuned to your needs by getting the proper context before generating a response.

## How does RAG work in AI models?

RAG combines two steps: retrieving information and generating a response. 

When you ask a question, the model first searches through an external knowledge base to find the most relevant documents. This retrieved information acts like notes or references that the large language model (LLM) keeps on hand for response generation. 

![A graphic depicting how RAG works.](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_8e6de82c99.png)

When generating its response, the LLM uses both internal knowledge obtained during training and the newly retrieved relevant context to return a response. It makes the response sound natural and grounded in real data. 

RAG co/mbines the best of both search engines and language models, ensuring the AI system is less prone to hallucinations.

Now, let’s take a closer look at CAG and how it works.

## How does CAG function in AI systems?

CAG works by providing AI with the necessary background information to generate an accurate response. When you interact with a CAG-enabled system, it does not rely on general knowledge alone. 

Instead, it takes note of the context you provide, such as your past conversations or personal preferences. This information is cached temporarily using a key-value (KV) cache and can be referenced while generating the responses.

![How CAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_CAG_Works_8f5381b703.png)

During the generation process, the LLM combines this cached context with its pre-trained knowledge to produce personalized and relevant answers. 

Fundamentally, this limits the possibility of generating generic responses. CAG also continuously updates its context and determines the most relevant piece, ensuring the conversation stays aligned with your needs. 

## What is the main difference between RAG and CAG?

The main difference between these two [systems](https://www.meilisearch.com/blog/rag-types) is how they handle information when generating responses. 

RAG obtains external resources for knowledge augmentation, whereas CAG relies on previously stored data. 

Below is a quick preview of their key differences: 

![RAG vs CAG.png](https://unable-actionable-car.media.strapiapp.com/RAG_vs_CAG_cbdc135b87.png)

* **Data source:** RAG retrieves fresh information from external databases or search engines whenever a query is made. CAG works with pre-cached data that is already stored and ready for reuse.
* **Latency & performance:** RAG tends to have higher latency, as it requires a retrieval step every time, whereas CAG achieves low latency through preloading and reuse.
* **Cost efficiency:** With RAG, costs are typically higher due to the constant need to retrieve and process data. CAG, on the other hand, is more cost-effective since it eliminates repeated queries by reusing stored data.
* **Use case fit:** RAG is best suited for scenarios where real-time accuracy is essential. CAG is more useful for repetitive tasks where the same knowledge is applied often.
* **Scalability & maintenance:** RAG can be harder to maintain because it relies heavily on live updates and connections to external sources. CAG is easier to scale since cached data can be refreshed periodically and managed with less effort. 

Now, let’s look at the pros and cons of both RAG and CAG.

## What are the pros and cons of RAG?

RAG is beneficial for situations where accuracy is crucial, but it also has some limitations. Here are some upsides and downsides of RAG:

**Advantages**

* **Provides fresh information through real-time retrieval:** It can pull in the most up-to-date information, which is particularly beneficial when details change a lot.
* **Has a broad knowledge base:** Because it connects to external sources, it has access to a broader pool of knowledge than an AI model on its own.
* **Reliable for fact-checking:** It is beneficial for fact-checking or knowledge tasks where precision is key.

**Disadvantages**

* **Slower performance:** Since RAG must fetch new data, it has a higher inference time and responds more slowly.
* **Higher costs:** The constant retrieval of information makes it more expensive to operate.
* **Maintenance challenges:** RAG relies on external systems, so the results can be inaccurate if a source goes down or changes its data.
* **Complex setup:** Setting RAG up and keeping it running smoothly can be technically demanding.


## What are the pros and cons of CAG?

CAG may be faster and cheaper, but it does not come without downsides. Here are some advantages and disadvantages of CAG:

**Advantages**

* **Faster responses:** Since the data is pre-cached, there is no need to fetch information repeatedly, resulting in faster reactions.
* **Cost-effective:** CAG is more cost-effective and efficient because cached material reduces the need for repeated processing.
* **Efficient for repeat tasks:** CAG is ideal for repetitive queries, where the same information is frequently required.
* **Easier to scale:** It is easier to scale and maintain, as updating the cache can be done periodically with minimal hassle.

**Disadvantages**

* **Needs refresh cycles:** To remain useful, the caches used in CAG require periodic reprocessing and refreshment.
* **Risk of outdated information:** Cached data can go stale, meaning it might miss recent changes or updates.
* **Limited extended context:** The CAG is limited in terms of what is already stored, which means the scope of answers is narrower.
* **Potential bias:** If the cached content is biased or incomplete, the responses will reflect those flaws.

Wondering when it is best to use RAG over CAG? Let’s discuss the best-fit scenarios for both.

## When should you use RAG instead of CAG?

RAG is the better choice when your [retriever](https://www.meilisearch.com/blog/rag-tools) must fetch new information from an external source for every user’s query. 

For instance, in financial market updates, things move fast. Caching may not be helpful, as you do not want your AI system to return stale information. In such a case, RAG is an excellent approach. 

It can also be used in fact-checking scenarios or high-stakes research projects where credibility depends on having the most current and comprehensive information available. 

Another good use case of RAG over CAG is in scenarios where you do not expect repeated questions from users. It could also be used when you cannot precisely predict what users will ask. 

In these cases, caching will not be particularly helpful because the LLM must retrieve new information from the knowledge sources each time. 

In short, use RAG when your goal is to get current and accurate results, even if it takes a little longer or costs more.

## When is CAG preferable to RAG?

Use CAG when optimization for speed and low latency is key. By reusing embeddings and cached context, CAG reduces costs and improves response times for predictable scenarios.

This is particularly useful in situations such as customer support chatbots, where people frequently ask the same questions repeatedly. Answers to questions such as ‘How do I change my password?’ or ‘What is the refund policy?’ rarely change, so caching saves time and cost without sacrificing accuracy.

CAG also works well for internal knowledge bases where employees frequently need the same set of guidelines or instructions. 

Another great fit for CAG is large-scale deployments where keeping retrieval costs low is essential for scalability. 

While CAG may not always reflect the latest updates, its strengths lie in consistency and speed. So, if the information you are dealing with does not change often, then CAG is a practical choice.

## How do you choose between RAG and CAG for your project?

Choosing between RAG and CAG really depends on what your project needs most in terms of freshness, speed, or efficiency. 

Here are some key criteria to consider:

* **Nature of queries:** If your system faces unpredictable queries, RAG is stronger since it can reach out for fresh information. If the questions are repetitive (for example, common FAQs) then CAG will serve you better.
* **Budget considerations:** RAG is more costly because every query must undergo the retrieval phase. On the other hand, CAG is cheaper because the cached data is reused.
* **Performance requirements:** RAG can be slower, but it gives you depth and updated context. CAG is faster because it does not need to fetch new data each time it is executed.
* **Freshness of information required:** When working on tasks that require real-time information updates, such as financial data, it is recommended to use RAG. For stable knowledge, cached responses are more than enough.
* **Growth potential:** Scaling RAG can become complicated due to constant retrievals. Scaling CAG is easier, since updating caches is simpler to manage.
* **Handling varying outputs:** CAG keeps things quick and efficient when you do not need much flexibility, but when accuracy is non-negotiable, using RAG is the better option.
* **Technical setup:** RAG needs a stronger infrastructure to maintain live connections. CAG is easier to deploy due to fewer moving parts.

## Can RAG and CAG be combined?

Yes, you can combine RAG and CAG. In fact, combining RAG and CAG often gives you the best of both worlds. These are called [hybrid approaches](https://www.meilisearch.com/blog/rag-techniques). 

In such RAG pipelines, KV cache data is served first, and if it is missing or inadequate, the system triggers the retrieval step.

RAG and CAG can be combined in different ways, depending on your project’s goals. For example, you can build a:

* **Fallback hybrid:** The system defaults to cached responses for speed, but if the cache does not have a good answer, it switches to RAG for live retrieval.
* **Selective hybrid:** Certain types of queries are always routed to RAG (e.g., time-sensitive information) while routine or repetitive queries go straight to CAG.
* **Blended hybrid:** Both RAG and CAG are used together. Cached data provides a quick first draft, and RAG fills the gaps with fresh context if needed.
* **Tiered hybrid:** Responses start with cache for instant speed, then RAG silently updates or corrects the answer in the background so the answer to the query is fresher.

The benefits of a hybrid system are clear: faster responses, reduced costs, and still having the option to retrieve real-time data when needed. 

However, combining both also brings challenges. You need to decide on the best way to determine when to use the cache and when to fetch new information. Managing two approaches together can also add complexity to the system’s infrastructure and maintenance.

## RAG vs CAG: Choosing the right path for smarter, faster AI

When it comes to building more intelligent AI systems, RAG gives you freshness and accuracy by retrieving live data, while CAG provides you with speed and efficiency with its cached responses. 

The real power lies in knowing when to use each one or even combine them in a hybrid system. 

By balancing your API design, workflows, and knowledge integration needs, you can create LLMs that deliver optimized, real-world performance.

RAG vs. CAG: The Smarter Choice for Your AI Stack

Retrieval-augmented generation (RAG) has rapidly become one of the most effective methods for building AI systems. While most apps lean on Python, Ruby offers a lightweight path to implementing RAG pipelines without the need for large frameworks.

This guide will walk you through the essentials, from understanding what RAG means to building a working Ruby app.

You’ll discover: 

* What RAG is and how it fits into Ruby projects without heavy frameworks.
* Why developers pick Ruby for flexible, lightweight RAG setups.
* Libraries and services that make retrieval and generation work smoothly.
* Clear steps to build a working RAG app using plain Ruby.
* The difference between Ruby’s ecosystem and performance with Python for RAG development.
* How tools like Meilisearch help you build fast and reliable retrieval layers.

By the end, you’ll have everything you need to develop a functional RAG pipeline in Ruby from scratch.

## What is a RAG application in Ruby?

A RAG application in Ruby combines retrieval, augmentation, and generation to create smart, context-aware LLM responses. For details on how it works, you can take a peek at[ our guide on what RAG is](https://www.meilisearch.com/blog/what-is-rag). 

In essence, RAG retrieves relevant information from a vector knowledge base, augments a prompt with that data, and then sends it to a large language model (LLM) to generate accurate answers.

![How RAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_7352c5302f.png)

A frequent question is whether you can implement RAG without relying on heavy frameworks. Yes, you can. 

Ruby developers can rely on embeddings, search engines, and API calls to connect with LLM services for seamless integration. As a result, the AI-powered tools built through Ruby are incredibly lightweight. 

The following section explains why Ruby is a suitable option for using RAG.

## Why use RAG with Ruby?

In a RAG system, Ruby’s clean syntax and expressive language makes retrieval and embedding management relatively easy. Plus, you can connect to LLM APIs without any heavy frameworks. 

While the focus of this guide isn’t on Rails, generally, many teams already run production systems on Ruby or Rails. Adding RAG capabilities on top of that lets them integrate vector search and generation directly into existing apps without migrating their stack. 

Ruby’s ecosystem, although not as mature as Python's, simplifies API calls and background job handling, allowing developers to build lightweight scripts that optimize retrieval, augmentation, and generation.

Next, let’s look at the essential tools you’ll need to build a RAG application in Ruby.

## What tools are needed for RAG in Ruby?

To build a RAG application in Ruby, you need a combination of libraries and APIs that cover retrieval, embedding generation, and LLM integration. 

While the Ruby ecosystem is still growing, several reliable options are available to set up a complete RAG pipeline, many of which overlap with those discussed in [our RAG tools comparison](https://www.meilisearch.com/blog/rag-tools). 

Let’s look at the best ones below:

* **Langchainrb:** A Ruby library that aids in building retrieval and generation workflows through building blocks. It excels at supporting document loading, embedding generation, and basic orchestration.
* **Qdrant-ruby:** A popular open-source vector database as well as a Ruby client for Qdrant. It allows you to store and search embeddings efficiently from Ruby code.
* **Hugging Face or Transformer APIs:** These APIs can be called from Ruby to generate embeddings or run models remotely. They’re useful when you want to avoid maintaining your own ML infrastructure.
* **Other vector store gems:** Some developers use custom wrappers for databases, like PostgreSQL with pgvector, or they integrate with Meilisearch for hybrid keyword and [vector search](https://www.meilisearch.com/blog/what-is-vector-search).
* **LLM API gems:** These are gems for OpenAI, Anthropic, or other model providers that allow you to generate responses directly from Ruby.

There you have it: a set of tools associated with Ruby that provide the foundation for handling data retrieval and connecting to powerful language models.


## Step-by-step guide to building a RAG app in Ruby

This section is your walkthrough of a five-step process that covers data preparation, embedding generation, retrieval setup with Meilisearch, connecting to an LLM, and building a minimal interface to tie it all together.


### 1. Load and prepare your data

The first step is to bring text into your Ruby environment in a structured way. This could be from local files, APIs, or databases. 

You will need to clean the text, split it into logical chunks, and attach metadata (such as titles or URLs) so that retrieval later on is meaningful.


```ruby
# Example: Loading and cleaning a text file
file_path = "docs/guide.txt"
content = File.read(file_path).strip
chunks = content.split(/\n\n+/) # split into paragraphs
docs = chunks.map.with_index do |chunk, i|
  { id: i, text: chunk, metadata: { source: file_path } }
end
```


This preparation is the foundation that ensures your retriever can quickly find relevant information without having to parse messy inputs later.


### 2. Generate embeddings and store them

Once the documents are ready, you’re going to convert each chunk into a numerical embedding. You can use an API like OpenAI or Hugging Face for this purpose. 

Next, you will store the resulting vectors alongside the original text in a lightweight database or pair them with Meilisearch for hybrid retrieval.


```ruby
require 'net/http'
require 'json'

def embed(text)
  uri = URI("https://api.openai.com/v1/embeddings")
  headers = { "Content-Type" => "application/json", "Authorization" => "Bearer #{ENV['OPENAI_API_KEY']}" }
  body = { input: text, model: "text-embedding-3-small" }.to_json
  JSON.parse(Net::HTTP.post(uri, body, headers).body)["data"].first["embedding"]
end

embeddings = docs.map { |d| d.merge(embedding: embed(d[:text])) }
```


This gives your pipeline a semantic backbone for accurate retrieval.


### 3. Implement a retriever with Meilisearch

Now, it’s time to index your documents in Meilisearch. Why? To enable fast keyword and hybrid search. Feel free to store metadata alongside text for filtering as well. This gives you typo tolerance, relevance ranking, and blazing-fast queries.


```ruby
require 'meilisearch'

client = MeiliSearch::Client.new('http://127.0.0.1:7700', 'masterKey')
index = client.create_index('documents', { primaryKey: 'id' })
index.add_documents(docs)

# Query example
results = index.search('embedding generation')["hits"]
```


Developers often underestimate how significantly Meilisearch outperforms manual filtering or SQL full-text search in text retrieval. A clean index makes every later step more reliable.


### 4. Pass retrieved context to an LLM

Now that you have the retrieved context, let’s move it on to the LLM so the answers are actually grounded in data. You’ll build a prompt that includes the essentials, such as user query, brief instructions, and a trimmed context window. 

You can avoid the bloat by focusing on only the most relevant snippets. If you really want to show where the facts came from, just use citation markers. 


```ruby
require "openai"

class AnswerGenerator
  def initialize(model: ENV.fetch("OPENAI_MODEL", "gpt-4o-mini"))
    @client = OpenAI::Client.new(access_token: ENV.fetch("OPENAI_API_KEY"))
    @model  = model
  end

  def answer(query:, hits:)
    context = build_context(hits, max_chars: 6_000) # keep under model limits
    messages = [
      { role: "system", content: "You are a helpful assistant. Answer using only the provided context. Cite sources like [doc:index]. If the answer is not in the context, say you do not know." },
      { role: "user",   content: "Query: #{query}\n\nContext:\n#{context}\n\nAnswer:" }
    ]

    @client.chat(parameters: { model: @model, messages: messages, temperature: 0.2 })
           .dig("choices", 0, "message", "content")
  end

  private

  def build_context(hits, max_chars:)
    buf = +""
    hits.each_with_index do |hit, i|
      snippet = hit.dig("_formatted", "content") || hit["content"] || hit["snippet"] || ""
      block = "[doc:#{i+1}] #{snippet}\n"
      break if buf.size + block.size > max_chars
      buf << block
    end
    buf
  end
end
```


Several factors enhance the retrieval-to-generation step. 

Keep the temperature low so the model sticks to facts. If the context doesn’t support an answer, it should say so clearly. Trim or rerank longer contexts to stay within token limits. 

And don’t forget to log document IDs for easy traceability and later evaluation.


### 5. Build a CLI or service interface

Here’s a tidbit you’ll enjoy: wrap retrieval and generation in a small interface so you and your teammates can try the flow quickly. 

A CLI keeps things dependency-light and works well for scripts, cron jobs, and demos. Add retries and basic timeouts to handle flaky networks or rate limits.


```ruby
require "json"
require "timeout"
require_relative "indexer"          # your Meili client and search method
require_relative "answer_generator" # the class above

class RagCli
  def initialize
    @indexer  = Indexer.new(index: ENV.fetch("MEILI_INDEX", "docs"))
    @llm      = AnswerGenerator.new
  end

  def run
    puts "RAG on Ruby. Type a question, or 'exit'."
    loop do
      print "> "
      q = STDIN.gets&.strip
      break if q.nil? || q.downcase == "exit"
      next if q.empty?

      hits = safe { @indexer.search(q, limit: 5) } || []
      if hits.empty?
        puts "No context found."
        next
      end

      response = safe { @llm.answer(query: q, hits: hits) }
      puts "\n#{response}\n\n"
      puts "Sources:"
      hits.each_with_index do |h, i|
        id = h["id"] || h.dig("_formatted", "id") || "unknown"
        puts "  [doc:#{i+1}] #{id}"
      end
      puts
    end
  end

  private

  def safe(max_retries: 3)
    tries = 0
    begin
      Timeout.timeout(30) { return yield }
    rescue => e
      tries += 1
      sleep(0.5 * tries)
      retry if tries < max_retries
      warn "Error: #{e.class} #{e.message}"
      nil
    end
  end
end

RagCli.new.run if $PROGRAM_NAME == __FILE__
```


A few production checks go a long way. Ensure API keys are validated upfront and with clear errors when any information is missing. If your provider allows it, stream tokens so users see answers as they’re generated. 

Log the essentials, such as queries, selected document IDs, answers, and citations, for easy debugging and evaluation later. Keep prompts within safe length limits and sanitize inputs to block prompt injection attempts. 

And if you’re exposing the app over HTTP, run it through a small Rack or Sinatra service with sensible timeouts and rate limits to maintain stability.

This completes the core loop: retrieve with Meilisearch, ground the answer with your context, and ship an interface that is easy to run and extend.

## How is RAG in Ruby different from Python?

While Python dominates the RAG ecosystem, Ruby has its own strengths for developers who prefer a leaner, language-level approach. 

The key differences lie in the ecosystem, available tools, and performance characteristics.

![Ruby vs. Python for RAG.png](https://unable-actionable-car.media.strapiapp.com/Ruby_vs_Python_for_RAG_4790deaa9a.png)

Ruby focuses on simplicity and direct integrations rather than building entire RAG stacks internally. 

Python offers more built-in support for machine learning and orchestration, but can feel heavier for smaller apps. 

The right choice depends on your goals, scale, and existing stack.

## Unlocking the power of RAG with Ruby

RAG unlocks new possibilities for Ruby developers who want intelligent, context-aware applications without shifting their entire stack. 

For those working with Rails, our [guide to building RAG apps on Rails](https://www.meilisearch.com/blog/build-rag-apps-on-rails) shows how to extend these ideas further. 

By combining Ruby’s flexibility with the right tools, you can build retrieval and generation pipelines that are both lightweight and powerful.

How to Build RAG Applications Using Ruby: A Developer’s Guide

How to Build RAG Applications on Rails: Step-by-Step Guide

RAG (retrieval-augmented generation) evaluation is the process of measuring how well a system actually retrieves relevant information to generate accurate answers for the users. It helps improve retrieval logic as well as the response quality.

RAG evaluation is crucial for identifying gaps in the system’s performance and reliability across various use cases.

The most common methods associated with RAG evaluation are human evaluation, reference answer scoring, and retrieval relevance checks.

In addition to this, the most widely used retrieval metrics are Exact Match, F1 score, Recall, Precision, BLEU, and ROUGE. The tools you’ll see used most often in this domain include Meilisearch, LangChain, RAGatouille, and Haystack. 

Now that the stage is set, let’s have a deeper look at what RAG evaluation is and the methods involved.

## What is RAG evaluation?

RAG evaluation is the process that determines how effectively a [RAG system](https://www.meilisearch.com/blog/what-is-rag) retrieves information and then uses that information to generate accurate answers for users. 

You might have noticed that RAG evaluation covers both retrieval and generation. This is to provide a clear view of the system's overall correctness. 

Once you have that, it becomes easier to identify weaknesses, test improvements, and benchmark different models or approaches. 

## Why is RAG evaluation important?

RAG evaluation is important because it helps determine whether your system is actually delivering responses that are relevant and helpful to users. 

Without this kind of evaluation, it's difficult to know if your pipeline is improving or drifting off course.

RAG evaluation can help teams identify and address issues early. This can include spotting hallucinations, irrelevant retrieval, and complex, inaccurate answers: 

* **Reduces hallucination risk:** Evaluation highlights the generation based on weak retrieval, allowing you to correct it immediately.
* **Improves user trust:** This evaluation helps make your RAG system fact-based and transparent, improving user confidence.
* **Enables faster iteration:** With regular evaluation, you can measure impact across changes and optimize performance quickly.

Next, let’s look at the methods used in RAG evaluation. 

## What methods are used in RAG evaluation?

The different methods used in RAG evaluation are shown in the image below:

![RAG Evaluation Methodologies.png](https://unable-actionable-car.media.strapiapp.com/RAG_Evaluation_Methodologies_28942e88c7.png)

These approaches evaluate both the retriever and generator, ensuring no compromise on high-quality outputs at any step.


### End-to-end evaluation

First up is the end-to-end evaluation. It focuses on the overall output of the RAG system. 

The factors under consideration are the usefulness, accuracy, and fluency of the generated response without breaking down the pipeline components. 

While it provides a quick way to compare systems, it doesn’t reveal which part of the pipeline – if any – actually failed.

### Retriever evaluation

Retrieval evaluation, as its name suggests, focuses on whether the retrieval system brings back relevant documents for the query. 

The evaluators can be human or automated. Either way, their function is to judge the quality of the retrieved context using a relevance score. 

You can pinpoint the weakness that led to poor answers downstream. 

### Grounding evaluation

Grounding eval checks if the generated response remains faithful to the retrieved content. 

How do we find that out? The output is compared with the supporting context to check factual accuracy. Naturally, this helps you spot and fix potential hallucinations.


## What are the key RAG evaluation metrics?

RAG evaluation metrics measure the quality of the retrieved documents and the accuracy of the generated response. Developers can use this information to identify where the pipeline is failing and make the necessary corrections.

Some of the key RAG evaluation metrics to know are the following:

* **Precision and recall:** Precision measures the percentage of retrieved docs that are actually relevant, while recall assesses the percentage of relevant documents that were retrieved. You need both to achieve a balance between accuracy and coverage in information retrieval. 

![Precision and recall.png](https://unable-actionable-car.media.strapiapp.com/Precision_and_recall_13f62b830c.png)

* **Groundedness:** This metric evaluates whether the generated output is factually consistent with the retrieved documents. A high groundedness score means the model is less likely to hallucinate.
* **Faithfulness:** Faithfulness measures how well the generated text sticks to the facts in the source material. Unlike groundedness, faithfulness seeks to identify distortions or unsupported claims.
* **Answer relevance:** Is the final answer even relevant to the user’s query? This is determined through answer relevance that relies on human ratings or automated scoring methods.
* **Fluency:** Fluency assesses how natural and readable the generated response is. After all, the target user is human. While secondary to accuracy, fluency impacts user trust and the overall experience.

Together, these metrics serve as essential benchmarks for any RAG applications, providing a complete and honest picture of the RAG system.

## What are the challenges in RAG evaluation?

RAG evaluation has a few unique challenges that we’ll list here:

1. **No universal benchmarks:** Unlike classic machine learning, there is no universal benchmark for RAG evaluation that applies to every [type of RAG system](https://www.meilisearch.com/blog/rag-types), as they all differ in architecture.
2. **Human evaluation costs too much:** Groundedness and relevance often require human judgment, which is expensive and hard to scale.
3. **Dynamic data and drift:** The frequent interaction of RAG pipelines with evolving knowledge bases means that evaluation results soon become outdated.
4. **Trade-offs between metrics:** There’s always a trade-off between the metrics. For instance, optimizing for fluency might reduce faithfulness. Similarly, improving recall could harm precision. 

## How is RAG evaluation implemented?

Different tools, frameworks, and services are responsible for implementing RAG evaluation. The evaluation process is mostly a combination of human review, automated evaluation, and continuous monitoring. 

Most teams prefer to conduct offline testing first and then move on to live monitoring once the pipeline is deployed. 

### Popular RAG evaluation frameworks

Popular open-source frameworks, such as Ragas, TruLens, and DeepEval, are a major help when it comes to automated testing. 

These frameworks use automated tests to help measure retrieval precision and factual consistency, and monitor hallucination rates. Through the LLM-as-a-judge grading they provide, you can easily identify weaknesses and correct them within the system.

For an easily digestible overview of retrieval layers and orchestration platforms, feel free to check out our guide on[ the top RAG tools](https://www.meilisearch.com/blog/rag-tools). 

### Synthetic test datasets

Another common method is creating synthetic evaluation datasets that replicate real-world query patterns. 

How does this help? It allows developers to benchmark retrieval quality and groundedness under a controlled set of conditions. You get to identify coverage gaps before the pipeline enters the production stage.

### Human-in-the-loop validation

There’s no alternative to human insight. Even with strong automation, human evaluation remains critical. 

Subject matter experts review sampled queries to verify if the retrieved content is relevant and the generated answers are accurate. It provides a nuanced view of the system that metrics alone cannot determine.

### Continuous monitoring in production

As a developer, you already know that evaluation does not stop once a system is live. Production monitoring tracks key metrics, including latency, error rates, and feedback loops. 

A reliable retrieval layer, such as Meilisearch, supports this process by consistently delivering fast, high-quality context that evaluation frameworks depend on.

Together, these approaches form a cycle of pre-deployment testing, human review, and real-time monitoring. As a result, the RAG pipelines remain accurate, efficient, and trustworthy over time.

## What are the best practices in RAG evaluation?

Best practices give you the most efficient way to ensure that pipelines are tested thoroughly and fairly. Early weakness detection, hallucination spotting, and maintaining reliability are all part of these best practices.

![Best Practices in RAG Evaluation.png](https://unable-actionable-car.media.strapiapp.com/Best_Practices_in_RAG_Evaluation_06aaf08217.png)

Some of the key best practices include:

* **Use more than one test set:** Evaluate your system with multiple data types and domains, rather than relying solely on a single benchmark. This exposes weaknesses that would otherwise go unnoticed.
* **Combine human and automated checks:** Humans provide context-sensitive judgments, while automated metrics offer speed and scale. A balance between both is your best bet.
* **Test the robustness of the system:** Introduce adversarial or noisy inputs to see how well the system maintains accuracy under less-than-ideal conditions.
* **Track longitudinal performance:** Don’t stop evaluating pipelines. Continue over a staggered time period. This helps identify model drift and measure whether improvements stick.
* **Target user expectations:** Design evaluations that mirror how end-users will interact with the system. Metrics should reflect usefulness, not just technical accuracy.

These practices make RAG evaluation a reliable part of development, ensuring systems remain stable and trustworthy at scale. To explore techniques that can further improve pipeline performance, take a look at our article on[ advanced RAG techniques](https://www.meilisearch.com/blog/rag-techniques).

## How long does it take to evaluate a RAG pipeline?

The answer to this question lies in the variables. Smaller prototypes can be evaluated within hours, while production-scale pipelines require days of rigorous evaluation. 

The duration also depends on factors such as dataset chunk size, evaluation methods, and the number of metrics being tracked. 

For example, using human annotators for quality checks is slower but provides better, more nuanced insights than automated benchmarks. 

The complexity of the pipeline also affects the timeline.

## Are there any papers on RAG evaluation?

Yes, there are several notable papers on RAG evaluation, and most of them can be found on arXiv. Here are some recent ones from 2024 and 2025:

* ‘Evaluation of Retrieval-Augmented Generation: A Survey’ (2024) by Hao Yu et al.
* ‘Evaluating Retrieval Quality in Retrieval-Augmented Generation’ (2024) by A. Salemi et al. 
* ‘VERA: Validation and Evaluation of Retrieval-Augmented Systems’ (August 2024) by Tianyu Ding et al.
* ‘Unanswerability Evaluation for Retrieval-Augmented Generation’ (December 2024) by Xiangyu Peng et al.
* ‘Retrieval Augmented Generation Evaluation in the Era of Large Language Models’ (April 2025) by Aoran Gan et al.
* ‘The Viability of Crowdsourcing for RAG Evaluation’ (April 2025) by Lukas Gienapp et al. 

These papers provide a rich foundation for anyone wanting to understand or benchmark RAG evaluation methods, metrics, and emerging best practices.

## The future belongs to teams that master RAG evaluation

RAG evaluation is not just a technical exercise – it is the feedback loop that keeps RAG systems honest, reliable, and ultimately useful. 

In practice, evaluation becomes the difference between a system that only looks impressive in demos and one that consistently delivers value in production.

As data sources shift, user expectations evolve, and generative models grow more complex, the ability to adapt evaluation strategies will separate cutting-edge teams from those left chasing errors downstream. 

RAG evaluation: Metrics, methodologies, best practices & more

Modular RAG refers to a method of building [retrieval-augmented generation (RAG) systems](https://www.meilisearch.com/blog/what-is-rag) where the entire setup is divided into smaller and independent components. This makes designing, customizing, or upgrading the pieces easier.

The key components of modular RAG are the retriever, which finds useful information, the generator, which turns the information into responses, and additional modules such as rerankers, memory, or vector databases, which improve the accuracy of the artificial intelligence system. Each module handles a specific function that contributes to the overall quality.

Modular RAG works by retrieving relevant context from external knowledge sources, passing it through a generative model, and allowing different modules to handle tasks such as filtering, summarization, or reasoning, depending on the task's requirements.

The key advantages of modular RAG are increased flexibility, easier debugging, and better scalability. The key disadvantages of modular RAG are added complexity in system design and potential challenges in integrating the different modules.

You can implement modular RAG by combining retrievers and LLMs into a RAG pipeline that fits your use case.

This article will discuss everything you should know about the modular RAG framework.

## What is modular RAG?

Modular RAG is a way of building RAG systems that treats each part of the process as separate but connected modules. Instead of having one rigid design, you get to choose the system build that works best.

In modular RAG, you can create a pipeline that adapts to your needs, be it faster retrieval or more accurate answers. Because you are not forced to use the same retrieval method, vector database, or language model every time, you get finer control over how information is retrieved, processed, and turned into answers. 

For example, you might love how one company's embedding model works but dislike its retrieval system. With modular RAG, you can retain the embeddings and change to a different retrieval process. You can even experiment with multiple language models to see the best-performing one for your use case.

Modular RAG is subdivided into three different components:

* The retriever that gathers context
* The generator that creates responses
* Additional modules, such as filters, memory, or rerankers, that refine the output to suit the user’s needs

Now, let’s see the working mechanism behind modular RAG.

## How does modular RAG work?

Modular RAG works by breaking the [RAG process](https://www.meilisearch.com/blog/how-to-build-rag) into separate components, with each component handling a specific task. 

![image5.png](https://unable-actionable-car.media.strapiapp.com/image5_6b10b78e7f.png)

These components can include:

* **Retriever:** The retriever searches a knowledge base or [vector database](https://www.meilisearch.com/blog/what-is-a-vector-database) for relevant information.
* **Reranker or filter:** This is optional but recommended. Before sending the results forward, a reranker or filter can reorder, clean, or narrow down the retrieved data so that only the most useful content is passed along to the next component.
* **Generator:** The generator (typically a large language model such as OpenAI GPT) takes the refined context and uses it to produce a response.
* **Additional modules:** Depending on the use case, additional modules such as memory modules or summarizers can be slotted in to improve accuracy or break down complex queries.
* **Output delivery:** Finally, the GenAI system returns an accurate response that is grounded in the retrieved information.

Let’s see some examples of modular RAG.

## What is an example of modular RAG?

An example of modular RAG in action can be an online shopping assistant. 

Let’s say you search for ‘running shoes under $100.’ The router first decides the best way to handle that query: either through keyword search or by semantic similarity search. 

The retriever then digs through the store’s product database. 

Next, the processing modules clean up the results, filter out any irrelevant items, and then rank the best fits. 

The generator takes that context and produces a useful reply such as, ‘Here are three running shoes under $100 that match your style.’

If the store later wants to add a recommendation setup or customer reviews as part of its process workflow, it can be done without changing the overall setup.

## What are the advantages of modular RAG?

The key advantages of using modular RAG over traditional and one-piece [RAG types](https://www.meilisearch.com/blog/rag-types) are the following:

* **Easy maintenance:** You can fix, upgrade, or replace a single part without worrying that the entire system will break.
* **Faster development:** Different teams can work on separate modules in parallel, allowing for faster deployment and optimization of new features.
* **Easy customization:** Different industries have different needs for modular RAG. A legal assistant may add citation tracking, while an e-commerce chatbot may use recommendation modules. Modular RAG makes it easy to tailor the modules to your needs.
* **Scalable performance:** Modules that handle heavier workflow can be given extra resources, while lighter ones can run with less. 
* **Reliability:** Since the AI system is divided into separate components, weak spots can be easily spotted and fixed, making it more reliable over time.

## What are the disadvantages of modular RAG?

The disadvantages of implementing modular RAG are the following:

* **Higher setup costs:** More modules mean more infrastructure and skilled people are needed to run them, which drives costs up. 
* **Complexity overload:** Managing several modules at once is not easy. There are more parts to manage individually and collectively. Troubleshooting the modules can also involve complex procedures, particularly in identifying the root cause of the issues.
* **Learning curve:** Teams using modular RAG must understand how each module works and fits together, which can slow down onboarding and often requires knowledge of data science and NLP.

## What is the architecture of modular RAG?

The architecture of modular RAG is built around breaking the whole system into smaller parts that work together. Here is how the typical architecture looks:

![image1.png](https://unable-actionable-car.media.strapiapp.com/image1_c115ed35eb.png)

* **Client query interface:** This is where the user’s query first comes in. 
* **Retrieval module:** The query from the client query interface is sent to the retrieval module. This module includes a vector search store (used for semantic search), a document store (used for traditional text search), and a cache layer (used to quickly retrieve recent or repeated queries). Together, they act like a library housing the important information.
* **Reasoning module:** Here, the context processor organizes what was retrieved, the inference engine helps the system reason through it, and the state manager keeps track of any history or ongoing context.
* **Generation module:** This is the last phase. The LLM generates a draft response, the template engine structures it neatly, and the output validator checks it for accuracy before delivering the final response to the user.

The architecture of modular RAG makes it easy to use for domain-specific use cases, such as customer support or enterprise knowledge graphs.

## How can modular RAG be implemented?

Modular RAG can be implemented by combining several independent pillars. Each pillar addresses a different aspect of the system and can work on its own or alongside the others. 

Let’s look at these pillars:  

### 1. Data pipeline

The data pipeline handles how datasets are ingested and processed. You can create multiple data pipelines based on the type of content. 

For example, an ETL pipeline that implements modular RAG for customer support documents and another that handles technical manuals. Each use case has its processing pipeline that knows how to handle that specific content. 

Techniques such as chunking and carefully selecting the chunk size can significantly improve retrieval quality.

### 2. Microservices 

The system is divided into independent services that communicate with each other via API calls. Your ‘retrieval’ service can handle document searches, while the ‘reasoning’ service processes the logic, and your ‘generation’ service crafts the final response.

[Meilisearch](https://www.meilisearch.com/docs) can be used as the retrieval service because it can quickly index and retrieve documents without heavy infrastructure overhead.

### 3. Routing layer

The routing layer directs queries based on rules or classifiers and decides which module or pipeline should handle user queries using retrieval strategies or classifiers. You can use simple keyword matching for simple queries, but machine learning classifiers are better for complex queries. 

### 4. Containers for technical implementation

Containers for technical implementation manage how services are packaged and deployed. They make it easy to scale specific modules up or down, swap one out if needed, or update a service without breaking the whole setup.

### 5. Monitoring and orchestration tools

Monitoring and orchestration tools ensure smooth coordination, error handling, and balancing of generation processes. They track how each module performs, catch errors before users find them, and coordinate complex queries that need multiple modules to work together.

## How can modular RAG be evaluated?

Evaluating modular RAG requires tracking metrics across modules:  

* Information retrieval quality via precision and recall.  
* Post-retrieval validation to ensure the correct retrieved documents.  
* End-to-end evaluation of speed and quality, compared against baseline pipelines.

The most comprehensive setup is a ‘sandboxed RAG playground,’ which includes static data, swappable modules, and an LLM to evaluate answers across different configurations.

You want to measure both individual module performance and overall system effectiveness. 

For individual modules, you can track metrics like retrieval accuracy, response time, and resource usage. Each specialized module should behave like an expert in its field, meaning your legal module should confidently handle legal queries, while the technical one should handle engineering problems perfectly.

End-to-end [RAG evaluation](https://www.meilisearch.com/blog/rag-evaluation) matters just as much as module benchmarks. Run complex queries and observe how well the modules work with each other. Check if the system returns an accurate answer. 

In addition, compare the overall quality of information retrieval against a single-pipeline baseline to determine if modular RAG is an improvement.

Another consideration is comparing latency and quality trade-offs. Modular systems often introduce increased latency due to the coordination overhead, so you will need to balance response speed with answer quality.

Real-world testing with iterative feedback loops and augmentation scenarios is great for evaluating modular RAG. It is better to use actual user queries on your system than synthetic test cases. 

Also, pay attention to edge cases where routing gets tricky or where the modules need to collaborate.

## How does modular RAG compare to naive RAG?

Modular RAG uses multiple specialized components that capture domain-specific details and deliver more accurate results, while naive RAG relies on a single retrieval system that may overlook nuances. 

Modular RAG is like a team of specialists, while naive RAG is more like relying on a generalist.

Compared to naive RAG, modular RAG is more powerful for complex queries, though it requires more effort to build and maintain. 

Naive RAG, on the other hand, is quick and straightforward. Your query goes into one retrieval step, documents come back, and an answer is generated.

## How does modular RAG compare to advanced RAG?

Modular RAG can be seen as an evolution of advanced RAG techniques, taking the same core idea but pushing it further by emphasizing modularity, scalability, and flexibility. 

Advanced RAG represents an improvement over naive RAG by incorporating smarter algorithms, enhanced reranking, or multiple retrieval steps. But it is still often built as one big pipeline. 

Modular RAG breaks this apart into smaller, specialized modules that can be swapped, upgraded, or scaled independently.

Both approaches are powerful, but modular RAG shines when you want control, scalability, and the freedom to adjust as your needs evolve.

## Are there papers on modular RAG?

Yes, there are different papers on modular RAG:

1. ‘Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks,’ published July 2024 by Yunfan Gao, Yun Xiong, Meng Wang, and Haofen Wang (arXiv). It introduces the modular RAG paradigm and lays out a flexible, multi-module architecture that evolves beyond the classic linear RAG flow. 
2. ‘Retrieval-Augmented Generation for Large Language Models: A Survey’ by Gao *et al.,* published December 2023 on arXiv. It covers the broader RAG evolution, including the various modular approaches.
3. ‘RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing’ by Hu and Lu, published April 2024 on arXiv. It examines modular components and their interactions. 
4. ‘A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models’ by Ding *et al.,* published May 2024 on arXiv. It explores modular architectures and training strategies.


## Start building intelligent RAG systems with Meilisearch

If you are looking to build smarter RAG systems, Meilisearch is your go-to tool. 

It is easy to set up, meaning you can focus more on building intelligent retrieval workflows rather than battling with designing RAG infrastructure. 

Meilisearch also shines at delivering highly relevant documents with low latency, making it ideal for feeding accurate context into your generative AI models.

Modular RAG: What it is, how it works, architecture & more

Traditional retrieval-augmented generation (RAG) is powerful, but it often treats information as isolated chunks, missing the connections between entities that give answers depth and context. 

GraphRAG changes that by combining knowledge graphs with large language models, allowing AI to reason across linked data, trace relationships, and produce richer, more accurate responses. 

Whether you’re building advanced chatbots, analyzing legal documents, or exploring healthcare research, GraphRAG offers explainable and scalable results. 

In this article, we explain how GraphRAG works, why it matters, and the tools and steps needed to implement it effectively.

## What is GraphRAG?

GraphRAG stands for Graph Retrieval-Augmented Generation. It is an advanced approach that follows the [basic principles of RAG](https://www.meilisearch.com/blog/what-is-rag) systems, but it also combines knowledge graphs with large language models to produce context-rich and accurate responses. 

While the common assumption might be that it retrieves isolated chunks, GraphRAG actually uses the structure of the graph to map the relationship between the entities or documents. This allows the model to reason and provide more depth in answers. 

Let’s look at how GraphRAG works in practice.

## How does GraphRAG work?

GraphRAG works by blending structured data from a graph with language model reasoning. 

The process starts with indexing and retrieval steps similar to those in a traditional [RAG retrieval workflow](https://www.meilisearch.com/blog/rag-techniques), but extends them with a graph layer for connected reasoning and contextual information that reflects real-world relationships. 

![image6.png](https://unable-actionable-car.media.strapiapp.com/image6_688a566c5d.png)

Let’s look at a sequential progression of a typical RAG workflow:

1. **Build the knowledge graph:** Start by collecting your source material – documents, datasets, or domain-specific content. From this, you extract entities (such as people, concepts, or places) and the relationships between them, either with natural language processing tools or manual curation. These entities and connections are then stored in a graph database or graph data structure.
2. **Retrieve relevant context:** When a user submits a query, the system first searches for relevant nodes and edges in the graph. A search engine like Meilisearch can quickly handle text or metadata retrieval before moving on to graph traversal. The result is a set of related entities and their connections, forming a structured context for the query.
3. **Generate with LLMs:** Finally, the retrieved subgraph, supporting documents, and the user’s query are passed to the LLM. The model combines the textual content with the graph’s relationships to produce a context-rich answer that responds directly to the query and provides supporting details tied to the graph’s structure.

In practice, this setup helps the LLM understand how different pieces of information fit together and provides a better overall explainability.

## What are the benefits of using GraphRAG?

The key benefits of GraphRAG include:

* **Improved accuracy:** Graph traversal only brings you the most relevant nodes and documents, so there’s a reduction of noise in the search results.
* **Stronger reasoning:** Structured connections in a graph let the LLM follow logical paths, making its answers more coherent and less prone to hallucinations than when reasoning over unstructured data.
* **Explainability:** GraphRAG makes tracing how an answer was formed easier. Developers can better view and explain the nodes and relationships supporting the output.
* **Scalability:** Graph structures make it easier to expand knowledge bases while keeping retrieval efficient, even as the volume of data grows.

Based on these benefits, it’s clear that GraphRAG can be invaluable in domains that rely on precision and trust, such as healthcare, finance, and law. 

## How is GraphRAG different from baseline RAG?

GraphRAG represents one of the different [types of RAG](https://www.meilisearch.com/blog/rag-types), and its retrieval process differs from baseline RAG. 

Baseline RAG is vector search-based, while GraphRAG uses structured relationships to get the end result. GraphRAG can still use vector and full-text search, but relationships drive what gets retrieved.

![image2.png](https://unable-actionable-car.media.strapiapp.com/image2_d3c776618c.png)

Key differences between baseline RAG and GraphRAG include:

* **Data structure:** Baseline RAG stores documents as [embeddings in a vector database](https://www.meilisearch.com/blog/what-are-vector-embeddings), while GraphRAG models entities and relationships as nodes and edges, optionally combined with embeddings.
* **Retrieval:** Baseline RAG relies on semantic similarity over vectors, whereas GraphRAG uses graph traversal with local or global search, often combined with vector or text search.
* **Query processing:** Baseline RAG sends a single query to the vector index, while GraphRAG follows a multi-step process that interprets the query, selects a subgraph, and may generate graph queries such as Cypher.
* **Context assembly:** Baseline RAG provides top-k chunks by similarity, while GraphRAG assembles a subgraph, community summaries, and linked documents aligned to the question.
* **Reasoning:** Baseline RAG is limited to reasoning within the retrieved chunk, whereas GraphRAG enables relational reasoning across connected facts and summaries.
* **Explainability:** Baseline RAG makes it difficult to trace why a chunk was retrieved, while GraphRAG offers traceable paths and nodes that support each answer.

## What is a knowledge graph in GraphRAG?

A knowledge graph is a structured representation of information in which entities are stored as nodes and the connections between them are relationships. 

The nodes don’t always represent the same thing. Each one can represent a different document, concept, or data point. The relationships tell you how these nodes are linked. All of this helps genAI use better context to generate its answers.

Next, we will look at common use cases where GraphRAG shines.

## What are common use cases for GraphRAG?

GraphRAG is ideal in domains where the answers must be based on structured relationships and context.

Let’s see some examples:

* **Advanced chatbots:** GraphRAG helps chatbots deliver more relevant responses through a better-established context from linked sources.
* **Answering complex queries:** GraphRAG contributes significantly to handling queries that span multiple topics, such as linking research findings across disciplines.
* **Financial services:** This industry has arguably the most use for GraphRAG, which can help connect company data, market trends, and regulatory filings to support investment analysis.
* **Healthcare research:** In healthcare, GraphRAG contributes to research by linking patient records, studies, and treatment outcomes for more informed recommendations.
* **Legal document analysis:** In the legal domain, GraphRAG helps map case law, statutes, and prior rulings to answer legal questions with supporting references.
* **Scientific knowledge exploration:** This is another use case involving connecting datasets, experiments, and literature to build hypotheses.

These use cases highlight the variety and potential influence of GraphRAG when applied correctly. 

Moving on, we will review tools that can help you implement it.


## What tools are available for implementing GraphRAG?

We already know that GraphRAG tools are designed to manage knowledge graphs, retrieve relevant nodes and documents, and integrate with LLMs for generation. 

But, depending on the goal, you may need a graph database, a fast search engine, or an orchestration framework.

[RAG tools](https://www.meilisearch.com/blog/rag-tools) commonly used in Graph RAG applications include:

* **Meilisearch:** Our fast, developer-friendly search engine that can serve as the initial retrieval layer before graph traversal.
* **LangChain:** A framework for chaining retrieval and generation components, with support for graph-based workflows.
* **Weaviate:** An open-source vector database with built-in support for knowledge graphs, allowing you to combine semantic search with graph-based retrieval.
* **FAISS:** A high-performance library for efficient similarity search across large-scale embeddings, often used alongside graphs to speed up relevant information retrieval with specialized algorithms.
* **Haystack:** A flexible framework for building end-to-end RAG pipelines, including integration with graph databases and LLMs.
* **Neo4j:** A widely used graph database that supports Graph RAG approach implementations.
* **GitHub repositories:** Many generative AI examples of GraphRAG can be found on GitHub, shared by both independent developers and enterprise providers like Microsoft or OpenAI.

The right combination of tools depends on the format your data sources, scale, and reasoning needs. 

Now, it’s time to go through the steps to implement GraphRAG.

## How to implement GraphRAG

GraphRAG follows a relatively simple pattern involving fast text indexing, building a graph of entities and links, and retrieving a focused subgraph before LLM generation uses that structure.

Here’s a practical path [you can build](https://www.meilisearch.com/blog/how-to-build-rag):


### 1. Prepare data and set up Meilisearch

The first step ingests documents, normalizes fields, and creates fast text and filter search.

Fast recall ensures the graph only has to operate over a smaller, high-relevance subset of documents instead of the entire dataset, which keeps queries efficient.


```python
# Install Meilisearch client
# pip install meilisearch
from meilisearch import Client
client = Client("http://127.0.0.1:7700", "MASTER_KEY")

# Create an index and add documents
idx = client.index("docs")
docs = [
  {"id":"r1","title":"AML report 2024","body":"Acme Ltd acquired Nova Bank in 2021."},
  {"id":"r2","title":"Filing 10-K","body":"Nova Bank operates in New York."}
]

# Add docs and make title filterable
idx.add_documents(docs)
idx.update_settings({"filterableAttributes":["title"]})
```


**Note:** `update_settings` is asynchronous. In production, you’d check the task status before searching. Otherwise, results may not be immediately available.

Next, use these docs to build the graph.


### 2. Build the knowledge graph

Step two involves extracting entities and relationships and storing nodes and edges in a database or in-memory graph.

Unlike flat text, a knowledge graph preserves the relationships between entities (who did what, where, when) giving the LLM structured context for reasoning.


```python
import spacy
nlp = spacy.load("en_core_web_sm")

text = "Acme Ltd acquired Nova Bank in 2021. Nova Bank operates in New York."
doc = nlp(text)

# Extract entities
entities = [{"id": ent.text, "type": ent.label_} for ent in doc.ents]

# For simplicity, define relations manually
relations = [
    ("Acme Ltd", "ACQUIRED", "Nova Bank"),
    ("Nova Bank", "OPERATES_IN", "New York")
]

# Connect to Neo4j
from neo4j import GraphDatabase
driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))

# Insert entities and relations
def add_entities_and_relations(entities, relations):
    with driver.session() as session:
        for e in entities:
            session.run(
                "MERGE (n:Entity {name: $name, type: $type})",
                name=e["id"],
                type=e["type"]
            )
        for src, rel, dst in relations:
               # Ensure relation label is safe before using it
               session.run(
                     f"""
                MATCH (a:Entity {{name: $src}}), (b:Entity {{name: $dst}})
                MERGE (a)-[r:`{rel}`]->(b)
                """,
                src=src, dst=dst
            )
add_entities_and_relations(entities, relations)
```


**Note:** In practice, you’d extract relations automatically (e.g., using dependency parsing or an IE model). Here they’re hard coded for clarity.


### 3. Connect retrieval to the graph

This step takes the user query, does fast recall in Meilisearch, maps results to entities, and expands the subgraph.

This step ensures the LLM sees only the relevant neighborhood of connected entities, reducing noise and making the answer more precise.


```python
q = "Who owns Nova Bank and where do they operate?"
hits = idx.search(q, limit=5)["hits"]

# Map results to seed entities (here done manually, but usually via NER/metadata)
seed_entities = ["Nova Bank"]

with driver.session() as session:
    graph_results = session.run("""
        MATCH (n:Entity {name: $name})-[r]-(m)
        RETURN n.name AS source, type(r) AS relation, m.name AS target
        LIMIT 50
    """, name=seed_entities[0])

edges = [dict(record) for record in graph_results]
print("Retrieved edges:", edges)
```


Next, pack the right context for generation.


### 4. Assemble context and generate with the LLM

This step summarizes nodes, edges, and linked passages. It builds a compact prompt that shows content and relationships.

The assembled context presents both the facts and their links, so the LLM can generate an answer that reflects not just content but relationships.


```python
context = {
    "nodes": entities,
    "edges": [{"src": e["source"], "rel": e["relation"], "dst": e["target"]} for e in edges],
    "supporting_text": [hit["body"] for hit in hits]
}

prompt = f"""
Question: {q}
Graph nodes: {context['nodes']}
Graph edges: {context['edges']}
Supporting text: {context['supporting_text']}
Answer with cited nodes and edges.
"""

# Example with OpenAI
from openai import OpenAI
client = OpenAI()  # expects OPENAI_API_KEY in environment

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}]
)

print(response.choices[0].message.content)
```


Moving to the final step, we will measure quality and tighten the loop.


### 5. Evaluate and improve

The final step involves checks for grounding, novelty, and path coverage. It adds caching, reranking, and guardrails.


```python
# Feedback storage
feedback_log = []
CONFIDENCE_THRESHOLD = 0.7

# Logging query feedback
def log_query_feedback(query, answer, confidence, missing_entities=None):
    feedback_log.append({
        "query": query,
        "answer": answer,
        "confidence": confidence,
        "missing_entities": missing_entities or []
    })

# Confidence / gap detection
def needs_expansion(entry):
    return (
        entry["confidence"] < CONFIDENCE_THRESHOLD
        or len(entry["missing_entities"]) > 0
    )

# Auto-expand graph
def auto_expand_graph(missing_entities):
    for entity in missing_entities:
        # Placeholder: implement recall + entity extraction
        docs = search_index(entity)
        for doc in docs:
            entities, relations = extract_entities_relations(doc["content"])
            upsert_graph(entities, relations)

# Feedback loop runner
def run_feedback_loop():
    for entry in feedback_log:
        if needs_expansion(entry):
            auto_expand_graph(entry["missing_entities"])
```


Evaluation closes the loop: it helps detect missing entities, reduce latency, and keep answers grounded and reliable.

**Note:** `search_index`, `extract_entities_relations`, and `upsert_graph` are placeholders. In a real pipeline, you’d implement these functions to query Meilisearch, run NER/RE, and insert results into Neo4j.


## Key takeaways on building with GraphRAG

GraphRAG brings structure and reasoning to retrieval-augmented generation, improving accuracy, context, and explainability. Combining a knowledge graph with a fast search layer gives LLMs the context they need to deliver better answers – a crucial step toward [mastering RAG](https://www.meilisearch.com/blog/mastering-rag).

What is GraphRAG: Complete guide [2025]

Agentic RAG (retrieval-augmented generation) gives your RAG pipelines agent-like decision-making ability. Instead of retrieving blindly, agentic RAG plans, retrieves, and validates steps to provide context-rich answers.

Why is this important? It helps developers by improving reliability, reducing fragmented responses, and enabling large language models (LLMs) to handle more complex tasks. 

The key benefits of agentic RAG are stronger reasoning, higher accuracy, and better explainability, while the key drawbacks are more computation, dependency on orchestration, and added complexity.

Agentic RAG works through a loop of retrieval, planning, and evaluation. Tools like Meilisearch, LangChain, and LlamaIndex make these pipelines possible.

Let’s break down what agentic RAG is in detail.

## What is agentic RAG?

Agentic RAG is an advanced method that equips a RAG pipeline with agent-like reasoning. 

Instead of relying on static information retrieval, it uses planning, decision-making, and iterative evaluation to determine when and how to retrieve context. Naturally, this makes the process more adaptive and precise.

Technically, agentic RAG refers to retrieval systems that embed autonomous agents into the pipeline to refine context, rerank sources, and validate outputs before passing them to the AI model. This ensures that accurate responses are generated consistently.

Curious to see an example? Let’s get right to it.

## An example of agentic RAG in action

While there are plenty of examples of agentic RAG in action, we’ll stick to two. 

The first example would be an AI customer support chatbot that actually considers a query instead of returning the first matching document it finds. 

It takes time to think whether to search product manuals, FAQs, or previous tickets before giving you an answer. 

This can be applied in finance as well. Imagine a compliance assistant that’s smart enough to retrieve regulatory filings and cross-check them against internal policies before it generates an output. 

In both these cases, agentic RAG acts as a decision-maker instead of just a retriever, leveraging tool use and automation to improve results.

## How does agentic RAG work?

Agentic RAG turns retrieval into a step-by-step process with reasoning behind it. The ‘agent’ plans, validates, and iterates instead of just generating the top result ASAP. 

![Agentic RAG.png](https://unable-actionable-car.media.strapiapp.com/Agentic_RAG_1a8a6c9e4f.png)

The workflow of agentic RAG looks like this:

1. **Query understanding:** It starts by understanding the user query. Then, if needed, it breaks it down into sub-tasks.
2. **Planning retrieval:** The agent decides which data sources to consult and in what order, be they structured databases, document collections, or APIs.
3. **Context retrieval:** The relevant documents are retrieved through vector, hybrid, or keyword search. Tools like Meilisearch make this stage efficient and customizable based on your preference.
4. **Evaluation and correction:** The retrieved data is scored for relevance, consistency, and completeness. The passages with low confidence are reranked, filtered, or replaced.
5. **Generation with oversight:** The LLM produces a draft response, but the agent may call for additional data retrieval steps or corrections before finalizing.
6. **Feedback loop:** Agent logs, confidence scores, and user feedback are used to refine retrieval and planning for future queries.

For the pipeline to work, specific key components are critical:

* Retriever for fast recall (e.g., Meilisearch)
* Planner that breaks down queries and decides retrieval order
* Evaluator that checks factual grounding and confidence
* Feedback loop to continuously improve outputs

Combining these elements allows agentic RAG to transform retrieval from a static lookup into a proper decision-making process with multi-agent systems coordinating tasks.

Now, let’s see why this approach is important for LLM applications.

## Is agentic RAG important?

Yes, agentic RAG is important because it addresses the core weaknesses of [traditional RAG](https://www.meilisearch.com/blog/what-is-rag) pipelines. 

Right off the bat, it reduces hallucination by validating context. Structured planning eliminates fragmented retrieval and automatically improves reliability for complex queries. 

For example, it helps in compliance research, where accuracy is critical, and in customer support, where context switching can cause irrelevant answers.

## Where is agentic RAG used?

Agentic RAG is applied in domains with significant reliance on accurate retrieval and adaptive reasoning. 

Let’s see some common use cases below:

* **Customer support:** Customers love accurate and context-aware responses, so agentic RAG in this field is becoming a must-have. 
* **Healthcare:** Agentic RAG can be applied in the form of clinical assistants capable of validating medical literature and patient records before producing an output.
* **Legal research:** The legal domain can benefit from agentic RAG, which maps statutes, precedents, and case law and cross-checks context before drafting summaries or legal arguments.
* **Financial services:** In finance, compliance bots retrieve regulatory filings, compare them with internal policies, and highlight discrepancies for auditors.
* **Education:** You will find intelligent tutoring systems that plan which sources to pull from and explain answers with supporting context.
* **Enterprise knowledge management:** Agents review large document repositories to rank and validate sources, which helps resolve contradictions and eliminate noise.
* **Scientific research:** RAG systems explore datasets and published papers, iteratively validating evidence to support hypothesis building.
* **Multimodal applications:** Agentic RAG integrates text, images, and structured data for real-time data analysis in media and research contexts.

By adapting retrieval to domain-specific needs, agentic RAG improves the reliability and usability of artificial intelligence systems. 

## What are the benefits of implementing agentic RAG?

![Benefits of Agentic RAG.png](https://unable-actionable-car.media.strapiapp.com/Benefits_of_Agentic_RAG_babce95706.png)

The key benefits of implementing agentic RAG are:

* **Improved accuracy:** Agentic RAG brings about a major difference in accuracy of responses by improving context and reducing hallucinations.
* **Context-aware reasoning:** Planning retrieval steps connects multiple sources into a coherent response. 
* **Explainability:** Each decision, including what was retrieved, why it was chosen, and how it influenced the output, can easily be traced. Hence, the output becomes transparent to both developers and end users.
* **Adaptability:** Agentic RAGs dynamically adjust retrieval strategies for different queries, making pipelines more flexible than static RAG implementations.
* **Continuous improvement:** With feedback loops, the system learns from past queries to refine retrieval and planning over time.
* **Smarter routing:** Agents determine which sources or external tools to consult, reducing noise and improving function selection.

Together, these benefits make agentic RAG a strong candidate for mission-critical AI systems.

## Are there drawbacks in agentic RAG implementation?

Indeed, there are tradeoffs for agentic RAG’s advanced usability and benefits. These include:

* **Higher computational cost:** With great power comes a higher recurring cost. The added steps of planning, reranking, and evaluation increase latency and resource usage, which can be a bottleneck for real-time applications.
* **Complexity of setup:** Unlike standard RAG, agentic pipelines require orchestration between retrievers, evaluators, and feedback loops. This raises the barrier to adoption.
* **Dependency on evaluators:** The accuracy of corrective steps often relies on evaluator models or heuristics. If these are weak or misaligned, the whole system can crumble.
* **Risk of overcorrection:** In some cases, agentic RAG is smarter than it needs to be. This leads to overcorrection, which might discard useful retrieved information. 

These drawbacks highlight the need for careful infrastructure planning before implementing agentic RAG. 

We also need to consider the challenges of deploying agentic RAG.

## What are the challenges of implementing agentic RAG?

Beyond drawbacks, teams face practical challenges when deploying agentic RAG. Let’s look at some below:

* **Data integration:** Many enterprises operate across siloed systems and formats. Building pipelines that unify structured, semi-structured, and unstructured data is not easy, to say the least. 
* **Scalability:** Agentic RAG requires extensive strategic planning and evaluation, which becomes more challenging during scaling.
* **Security and compliance:** Security is obligatory and keeping up with compliance adds a lot of complexity to the architecture. 
* **Developer expertise:** Implementing orchestration logic demands experience with frameworks, retrievers, and agent systems. Keeping up with the adoption could be difficult if your team is not skilled enough.

Next, let’s look at the tools commonly used in agentic RAG pipelines.

## What tools are used in agentic RAG pipelines?

[Different retrieval tools](https://www.meilisearch.com/blog/rag-tools) support agentic RAG:

* **LangChain:** LangChain is a widely used orchestration framework that connects retrievers, evaluators, and generators into an agentic workflow. It allows the LLM to decide when to retrieve, when to generate, and when to validate. It’s central to many agentic setups.
* **LlamaIndex:** A tool designed for indexing and structuring data, which helps agents plan retrieval strategies across multiple external knowledge sources. It is often used to build a hierarchical context or combine unstructured and structured data.
* **Vector databases (e.g., Weaviate, Pinecone):** These provide scalable [vector search](https://www.meilisearch.com/blog/what-is-vector-search) that supports semantic similarity through embedding-based methods. Agents use them for dense retrieval before applying correction or validation steps.
* **Meilisearch:** A fast, developer-friendly search engine that fits as the initial retrieval layer in agentic RAG. It stands out thanks to its hybrid search, filters, and typo tolerance, which help agents recall precise candidates before running corrective or validation loops. For AI agent deployment, it provides speed and fine-grained control, which keeps pipelines responsive while maintaining relevance. It’s an all-in-one package for developers.

These tools create the infrastructure agents need to plan, retrieve, evaluate, and refine responses. 

Now, let’s walk through how to implement an agentic RAG pipeline.


## How can you implement agentic RAG?

There’s a methodical implementation for agentic RAG. It requires a step-by-step process that includes layering planning, validation, and feedback on top of a standard retrieval–generation pipeline. 

The following steps show how to set it up using Meilisearch for retrieval and an LLM for reasoning:


### 1. Ingest and index documents with Meilisearch

The first step is preparing your training data. The documents are indexed in Meilisearch so the agent can query them quickly. This provides a fast recall layer before validation and reasoning.

```python
from meilisearch import Client
client = Client("http://127.0.0.1:7700", "MASTER_KEY")
index = client.index("docs")

docs = [
  {"id": "1", "title": "Onboarding Guide", "body": "Steps to onboard a new user securely."},
  {"id": "2", "title": "Security Policy", "body": "MFA is mandatory for admins."}
]
index.add_documents(docs)
```


This gives the agent a searchable knowledge base to work with.


### 2. Retrieve candidates and build context

Next, the agent recalls relevant information and organizes it into a usable context for reasoning.


```python
query = "How do I onboard a new user with MFA?"
hits = index.search(query, {"limit": 5})["hits"]

context = "\n".join([f"{h['title']}: {h['body']}" for h in hits])
```


By structuring evidence, the system avoids sending raw, noisy text directly to the model.


### 3. Validate and refine context

Before passing context to the generator, the agent checks for gaps or contradictions. If key terms are missing, it expands the query with synonyms or related phrases.


```python
def needs_more_context(hits):
    return not any("MFA" in h["body"] for h in hits)

if needs_more_context(hits):
    more = index.search("multi-factor authentication", {"limit": 5})["hits"]
    hits += more
```


This step ensures the generator sees comprehensive and accurate information.


### 4. Generate the final answer and log feedback

Finally, the agent generates a response with citations and logs feedback to improve future runs.


```python
from openai import OpenAI
client = OpenAI(api_key="YOUR_API_KEY")

prompt = f"Q: {query}\nContext:\n{context}\nAnswer with citations."
resp = client.chat.completions.create(model="gpt-4o", messages=[{"role":"user","content":prompt}])

answer = resp.choices[0].message.content
print(answer)
```


The feedback on confidence and missing facts can be stored to refine retrieval strategies over time. 

In the following section, let’s tackle some of the most frequently asked questions about the performance of agentic RAG.


## How does agentic RAG perform?

A recent[ survey on agentic RAG](https://arxiv.org/abs/2501.09136) highlights that it enables flexibility, scalability, and context awareness across complex domains like healthcare, finance, and education.

Another[ study on Search-R1](https://arxiv.org/html/2507.10411v1) found that effective retrievers shorten reasoning steps and improve answer quality, with query performance prediction strongly correlating with final accuracy.

Finally, in recommendation systems, an[ agentic RAG framework](https://arxiv.org/abs/2506.21931) achieved a 42.1% improvement in NDCG@5 and 35% in Hit@5 compared to standard RAG approaches. 


### How does agentic RAG differ from traditional RAG?

We already know that traditional RAG systems use one-shot retrieval followed by generation. Agentic RAG is more capable and comprehensive. It plans, iterates, and retrieves adaptively through context validation. 


### How does agentic RAG differ from naive RAG?

Naive RAG uses static retrieval rules with no feedback, while agentic RAG defers retrieval decisions to the agent within a reasoning and feedback loop. This enables retrieval to evolve based on the context.


### How does agentic RAG differ from graph RAG?

[Graph RAG](https://www.meilisearch.com/blog/graph-rag) uses structured relationships for retrieval, but is still a single-pass retrieval system. Agentic RAG layers planning and feedback on top of graph or vector retrieval, enabling reasoning over evolving multi-hop paths.


### How does agentic RAG differ from adaptive RAG?

[Adaptive RAG](https://www.meilisearch.com/blog/adaptive-rag) adjusts retrieval parameters based on query difficulty. Agentic RAG, on the other hand, goes further. It actively generates new retrieval queries mid-reasoning to fill gaps or refine context.


### How does agentic RAG differ from an MCP?

Agentic RAG is a retrieval-planning paradigm. In contrast, an MCP (multi-choice prompt) is a prompting technique focused on structured output design. It’s not associated with retrieval optimization or adaptive querying.


## Start building intelligent RAG systems

By treating retrieval as a process of reasoning rather than a static lookup, agentic RAG forces us to design RAG agents that think before they answer. 

That added deliberation may cost more computing and complexity. Still, it sets the foundation for agentic AI that is not only more accurate but also more trustworthy, explainable, and adaptable to real-world challenges.

Meilisearch latest news and company updates