Rig Agents: High-Level LLM Orchestration

Agents in Rig provide a high-level abstraction for working with LLMs, combining models with context, tools, and configuration. They serve as the primary interface for building complex AI applications, from simple chatbots to sophisticated RAG systems.

Core Concepts

Agent Structure

An Agent consists of:

Base Components
- Completion Model (e.g., GPT-4, Claude)
- System Prompt (preamble)
- Configuration (temperature, max tokens)
Context Management
- Static Context: Always available documents
- Dynamic Context: RAG-based contextual documents
- Vector Store Integration
Tool Integration
- Static Tools: Always available capabilities
- Dynamic Tools: Context-dependent capabilities
- Tool Management via ToolSet

Usage Patterns

Basic Agent Creation

use rig::{providers::openai, Agent};
 
let openai = openai::Client::from_env();
 
// Create simple agent
let agent = openai.agent("gpt-4")
    .preamble("You are a helpful assistant.")
    .temperature(0.7)
    .build();
 
// Use the agent
let response = agent.prompt("Hello!").await?;

RAG-Enabled Agent

use rig::{Agent, vector_store::InMemoryVectorStore};
 
// Create vector store and index
let store = InMemoryVectorStore::new();
let index = store.index(embedding_model);
 
// Create RAG agent
let agent = openai.agent("gpt-4")
    .preamble("You are a knowledge assistant.")
    .dynamic_context(3, index)  // Retrieve 3 relevant documents
    .build();

Tool-Augmented Agent

use rig::{Agent, Tool};
 
// Create agent with tools
let agent = openai.agent("gpt-4")
    .preamble("You are a capable assistant with tools.")
    .tool(calculator)
    .tool(web_search)
    .dynamic_tools(2, tool_index, toolset)
    .build();

Key Features

Automatic Context Fetching

Agents can use multiple sources of context:

Static context documents (provided from AgentBuilder::context()). These will be appended to every single request.
Dynamic context fetching (provided from AgentBuilder::dynamic_context()). A max number of results will be fetched from a given vector store and returned to the agent.

Documents will be automatically appended to a given provider request before it is sent.

Tool Management

Agents can:

Maintain static and dynamic tool sets
Resolve tool calls automatically
Handle tool execution and error states

Below is an explanation of the types of tools (and tool related fields) that can be found:

toolset: The actual tool repository (where tools live). Contains a map of tool names as well as their respective boxed function pointers.
static_tools: Tools that will always be presented and available to use in tool definition lists when interacting with model providers.
dynamic_tools: When given dynamic toolsets (via AgentBuilder::dynamic_tools), agents are able to dynamically retrieve tools from a given vector store (assuming that the tools already exist in the vector store). Based on fetched results, tool definitions will be slotted into the model provider request to tell it which tools it can call.

Additional Parameters

Sometimes the default Agent configuration is not enough and the model provider you want to use might need extra parameters (for example, reasoning or some other provider-specific parameters). In this case, you can use AgentBuilder::additional_params() to be able to provide these parameters which will then be passed into the completion request.

// replace `create_openai_client() with whatever way you're using
// to instantiate your model provider client
let openai_client = create_openai_client();
 
let agent = openai_client.agent("gpt-5")
    .preamble("You are a helpful agent")
    .additional_params(serde_json::json!({
        "foo": "bar"
    }))
    .build();

Prompt Hooks

Prompt hooks are a way to be able to add customisable behaviour to the Agent prompt loop. The technical implementation provides a default method implementation for every method, so you only need to implement whatever methods you’d like to add.

Features:

Access completion request prompts, completion responses, tool calls and tool responses
You can cancel the agentic prompting loop from the agent hook (by using CancelSignal::cancel()). This returns a PromptError::PromptCancelled error, as well as the internal chat history.

Use cases:

Add extra observability to your programs
Add any extra custom behaviour you want to the agent loop!

Bear in mind that while prompt hooks are not blocking, it’s generally advisable to make not your behavior too computationally intensive.

Best Practices

Context Management
- Keep static context minimal and focused
- Use dynamic context for large knowledge bases
- Consider context window limitations
Tool Integration
- Prefer static tools for core functionality
- Use dynamic tools for context-specific operations
- Implement proper error handling in tools
Performance Optimization
- Configure appropriate sampling sizes for dynamic content
- Use temperature settings based on task requirements
- Monitor and optimize token usage

Common Patterns

Conversational Agents

let chat_agent = openai.agent("gpt-4")
    .preamble("You are a conversational assistant.")
    .temperature(0.9)
    .build();
 
let response = chat_agent
    .chat("Hello!", previous_messages)
    .await?;

RAG Knowledge Base

let kb_agent = openai.agent("gpt-4")
    .preamble("You are a knowledge base assistant.")
    .dynamic_context(5, document_store)
    .temperature(0.3)
    .build();

Agents with Tools

When using AgentBuilder, you can add all kinds of tools to your Agent:

let tool_agent = openai.agent("gpt-4")
    .preamble("You are a tool-using assistant.")
    .tool(calculator) // calculator tool
    .tool(web_search) // web search tool - for example, Bing API
    .dynamic_tools(2, tool_store, toolset)
    .temperature(0.5)
    .build();

Multi-turn

Assuming your agent only calls 1 tool at a time, the default configuration should typically be enough. However, as seen in the above code snippet, we’ve just created an agent with two potential tools using .dynamic_tools() - meaning that there may be an instance of it calling two tools to be used, in which case you will need to increase the number of maximum turns to avoid receiving a MaxDepthError (produced when your agent has used up too many turns to avoid issues with infinite tool calling).

To do this in Rig, once you’ve added your prompt you can simply use the .multi_turn() function to add extra maximum turns:

let res = tool_agent
    .prompt("Please calculate 2+5")
    .multi_turn(1)
    .send()
    .await?;
 
println!("{res}");

The above example will have a maximum of 1 turn due to passing a value of 1 into .multi_turn() - so it can use 1 round of tool calls.

There is no arbitrary maximum on the amount of turns possible, although you may want to use reasonable turn limits to avoid errors.