(2025-03-05) Lab Notebook: Update: How to run RAG in OpenWebUI

Warning

This is as a Lab Notebook which describes how to solve a specific problem at a specific time. Please keep this in mind as you read and use the content. Please pay close attention to the date, version information and other details.

Retrieval-Augmented Generation (RAG) is a technique that grants generative artificial intelligence models information retrieval capabilities. It modifies interactions with a large language model so that the model responds to user queries with reference to a specified set of documents. This allows the model to generate more accurate and relevant responses.

OpenWebUI has changed their layout in version 0.5.X and changed the steps to setup and use RAG.

  1. Create a Knowledge Base

    • Go to Workspace on the left sidebar and click Knowledge on the top bar.
    • Click + on the right side to create a Knowledge Base.
    • Fill in the following details:
    • Name: your-knowledge-base-name
    • Purpose: your-purpose
    • Visibility: Public or Private
    • Click Create Knowledge.

    Note: No matter what the visibility is set to, only you can see the knowledge base. All files are stored locally on the HPCC in your home directory.

  2. Upload the Files

    Files must be on the HPCC; if they are not already, please view our File Transfer documentation.

    After creating the knowledge base, you will see a search bar on the right side of the screen. Clicking the plus icon next to it will allow you to upload files to the knowledge base. It gives four options:

    • Upload File (Upload a single file)
    • Upload Directory (Upload a directory of files and all subdirectories)
    • Sync Directory (Delete all files in the knowledge base and allow you to upload a new directory)
    • Add Text Content (Opens a simple text editor where you can directly type in the text and save it as a file in the knowledge base)

    Now that files have been uploaded, you can access them in any chat, like before, using the # symbol, this will bring up a menu to either append single files or the entire knowledge base.

  3. Manage Files

    Click Workspace on the left sidebar and then click Knowledge on the top bar. You will see a list of all your knowledge bases.

    To delete a knowledge base, click on the three dots on the knowledge base you want to delete and click Delete.

    To manage files in a knowledge base, click on the knowledge base you want to manage.

    Once files are uploaded you will be able to see them under the search bar on the right, clicking on a file will open a preview of the file, clicking the "X" icon on the left will delete the file. To delete all files quickly you can click the "Sync Directory" option inside the upload menu(+).

  4. Create a RAG Model

    Open WebUI now has a new feature where you can pair a collection of files with a model for easy access. This is NOT a fine-tuned model, it is just a shortcut to access a specific LLM with a specific knowledge base.

    A general rule of thumb is that large models are better for RAG. Around 70B parameters is a good starting point.

    • Go to Workspace on the left sidebar and click Models on the top bar.
    • Click + on the right side to create a Model.
    • Fill in the details (the most important being):
    • Name: your-model-name
    • Base Model(From): your-base-model
    • Knowledge: your-knowledge-base-name
    • Click Save & Create

    Note: You must have already created a knowledge base (using the steps above). And you can only select from already installed models, check out how to install models for Ollama(OpenWebUI uses Ollama as a backend) in our docs: Download Models.

    To edit or manage these models, go to Workspace > Models and click on the three dots on the model you want to manage. Here you can clone, export or delete the model. Click the pen icon to edit the model.

See Open WebUI's documentation for more details: