Rig Agents: High-Level LLM Orchestration

Agents in Rig provide a high-level abstraction for working with LLMs, combining models with context, tools, and configuration. They serve as the primary interface for building complex AI applications, from simple chatbots to sophisticated RAG systems.

Core Concepts

Agent Structure

An Agent consists of:

Base Components
- Completion Model (e.g., GPT-4, Claude)
- System Prompt (preamble)
- Configuration (temperature, max tokens)
Context Management
- Static Context: Always available documents
- Dynamic Context: RAG-based contextual documents
- Vector Store Integration
Tool Integration
- Static Tools: Always available capabilities
- Dynamic Tools: Context-dependent capabilities
- Tool Management via ToolSet

Usage Patterns

Basic Agent Creation

use rig::{providers::openai, Agent};
 
let openai = openai::Client::from_env();
 
// Create simple agent
let agent = openai.agent("gpt-4")
    .preamble("You are a helpful assistant.")
    .temperature(0.7)
    .build();
 
// Use the agent
let response = agent.prompt("Hello!").await?;

RAG-Enabled Agent

use rig::{Agent, vector_store::InMemoryVectorStore};
 
// Create vector store and index
let store = InMemoryVectorStore::new();
let index = store.index(embedding_model);
 
// Create RAG agent
let agent = openai.agent("gpt-4")
    .preamble("You are a knowledge assistant.")
    .dynamic_context(3, index)  // Retrieve 3 relevant documents
    .build();

Tool-Augmented Agent

use rig::{Agent, Tool};
 
// Create agent with tools
let agent = openai.agent("gpt-4")
    .preamble("You are a capable assistant with tools.")
    .tool(calculator)
    .tool(web_search)
    .dynamic_tools(2, tool_index, toolset)
    .build();

Key Features

Dynamic Context Resolution

The agent automatically:

Processes incoming prompts
Queries vector stores for relevant context
Integrates retrieved information
Maintains conversation coherence

Reference:

rig-core/src/agent.rs [171:197]

        let dynamic_context = stream::iter(self.dynamic_context.iter())
            .then(|(num_sample, index)| async {
                Ok::<_, VectorStoreError>(
                    index
                        .top_n(prompt, *num_sample)
                        .await?
                        .into_iter()
                        .map(|(_, id, doc)| {
                            // Pretty print the document if possible for better readability
                            let text = serde_json::to_string_pretty(&doc)
                                .unwrap_or_else(|_| doc.to_string());
 
                            Document {
                                id,
                                text,
                                additional_props: HashMap::new(),
                            }
                        })
                        .collect::<Vec<_>>(),
                )
            })
            .try_fold(vec![], |mut acc, docs| async {
                acc.extend(docs);
                Ok(acc)
            })
            .await
            .map_err(|e| CompletionError::RequestError(Box::new(e)))?;

Tool Management

Agents can:

Maintain static and dynamic tool sets
Resolve tool calls automatically
Handle tool execution and error states

Reference:

rig-core/src/agent.rs [199:221]

        let dynamic_tools = stream::iter(self.dynamic_tools.iter())
            .then(|(num_sample, index)| async {
                Ok::<_, VectorStoreError>(
                    index
                        .top_n_ids(prompt, *num_sample)
                        .await?
                        .into_iter()
                        .map(|(_, id)| id)
                        .collect::<Vec<_>>(),
                )
            })
            .try_fold(vec![], |mut acc, docs| async {
                for doc in docs {
                    if let Some(tool) = self.tools.get(&doc) {
                        acc.push(tool.definition(prompt.into()).await)
                    } else {
                        tracing::warn!("Tool implementation not found in toolset: {}", doc);
                    }
                }
                Ok(acc)
            })
            .await
            .map_err(|e| CompletionError::RequestError(Box::new(e)))?;

Additional Parameters

Sometimes the default Agent configuration is not enough and the model provider you want to use might need extra parameters (for example, reasoning or some other provider-specific parameters). In this case, you can use AgentBuilder::additional_params() to be able to provide these parameters which will then be passed into the completion request.

// replace `create_openai_client() with whatever way you're using
// to instantiate your model provider client
let openai_client = create_openai_client();
 
let agent = openai_client.agent("gpt-5")
    .preamble("You are a helpful agent")
    .additional_params(serde_json::json!({
        "foo": "bar"
    }))
    .build();

Best Practices

Context Management
- Keep static context minimal and focused
- Use dynamic context for large knowledge bases
- Consider context window limitations
Tool Integration
- Prefer static tools for core functionality
- Use dynamic tools for context-specific operations
- Implement proper error handling in tools
Performance Optimization
- Configure appropriate sampling sizes for dynamic content
- Use temperature settings based on task requirements
- Monitor and optimize token usage

Common Patterns

Conversational Agents

let chat_agent = openai.agent("gpt-4")
    .preamble("You are a conversational assistant.")
    .temperature(0.9)
    .build();
 
let response = chat_agent
    .chat("Hello!", previous_messages)
    .await?;

RAG Knowledge Base

let kb_agent = openai.agent("gpt-4")
    .preamble("You are a knowledge base assistant.")
    .dynamic_context(5, document_store)
    .temperature(0.3)
    .build();

Agents with Tools

When using AgentBuilder, you can add all kinds of tools to your Agent:

let tool_agent = openai.agent("gpt-4")
    .preamble("You are a tool-using assistant.")
    .tool(calculator) // calculator tool
    .tool(web_search) // web search tool - for example, Bing API
    .dynamic_tools(2, tool_store, toolset)
    .temperature(0.5)
    .build();

Multi-turn

Assuming your agent only calls 1 tool at a time, the default configuration should typically be enough. However, as seen in the above code snippet, we’ve just created an agent with two potential tools using .dynamic_tools() - meaning that there may be an instance of it calling two tools to be used, in which case you will need to increase the number of maximum turns to avoid receiving a MaxDepthError (produced when your agent has used up too many turns to avoid issues with infinite tool calling).

To do this in Rig, once you’ve added your prompt you can simply use the .multi_turn() function to add extra maximum turns:

let res = tool_agent
    .prompt("Please calculate 2+5")
    .multi_turn(1)
    .send()
    .await?;
 
println!("{res}");

The above example will have a maximum of 1 turn due to passing a value of 1 into .multi_turn() - so it can use 1 round of tool calls.

There is no arbitrary maximum on the amount of turns possible, although you may want to use reasonable turn limits to avoid errors.