Creating Agents

Design and deploy reusable agents with pre-configured properties for consistent voice experiences.

Understanding Agents

Agents provide a way to define voice assistants that can be reused across multiple calls, ensuring consistent behavior and capabilities. This enables you to maintain a cohesive user experience with minimal configuration overhead at call creation time. Each agent includes a call template that defines system prompts, voice settings, available tools, and more.

Key benefits of using Agents:

Reusable Configuration → Create a single agent definition and use it for multiple calls without repeating configuration settings.

Consistent Experience → Ensure your voice experience maintains the same personality, capabilities, and behavior across all interactions.

Version Control → Update an agent’s configuration in one place and have changes apply to all future calls.

Simplified Deployment → Reduce the complexity of starting calls by referencing an existing agent instead of providing all configuration details.

Time-Saving Feature

Agents are ideal for production applications where you want consistent behavior across multiple user interactions.

Creating and Managing Agents

To implement Agents in your Ultravox application, follow these steps:

1. Plan Your Agent

   Define the agent’s purpose, personality, and capabilities. Consider what system prompt, voice, and tools it should have.

2. Create the Agent

   Use the Create Agent API to create a new agent with a name and call template that specifies:

   • System prompt that defines the agent’s personality and knowledge
   • Voice settings (voice ID, language, etc.)
   • Available tools the agent can use
   • Other call settings like timeout values, inactivity messages, etc.
3. Use the Agent for Calls

   When starting a new call, use the Create Agent Call API and reference the agent ID instead of providing all call settings individually.

   You can optionally override specific settings for individual calls while keeping the agent’s base configuration.

4. Update as Needed

   Modify your agent’s configuration over time using the Update Agent API as requirements change, ensuring all future calls use the updated settings.

Active Calls not Impacted

Agent changes don’t affect active calls, only new calls started after the change.

Example Agent Creation

Here’s a basic example of how to create an agent using the Ultravox API:

Example: Creating a New Customer Support Agent

// Note: we are using a template variable for customerName
const createAgent = async () => {
  const response = await fetch('https://api.ultravox.ai/api/agents', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-API-Key': 'your-api-key'
    },
    body: JSON.stringify({
      name: 'Customer Support Agent',
      callTemplate: {
        systemPrompt: "You are a friendly customer support agent for Acme Inc. Your name is Alex. You are talking to {{customerName}}. You should help them with their questions about our products and services. If you can't answer a question, offer to connect them with a human support agent.",
        voice: "Jessica",
        languageHint: "en",
        temperature: 0.4,
        recordingEnabled: true,
        firstSpeakerSettings: {
          agent: {
            text: "Hello! This is Alex from Acme customer support. How can I help you today?"
          }
        },
        selectedTools: [
          { toolName: 'knowledgebaseLookup' },
          { toolName: 'transferToHuman' }
        ]
      }
    })
  });
  
  return await response.json();
};

Mono Prompts and Call Templates

When you start building your agent, we recommend you start with a simple mono prompt and then add complexity if and when needed. If mono prompting isn’t working for your use case (i.e. you are building something complicated), you may need to use what we call “inline instructions” which allow you to provide reminders to the model and help steer the agent in the right direction. Please see Guiding Agents.

If you are building an application with completely distinct, unrelated stages where there is no overlap, please see the Call Stages guide.

Starting a Call with an Agent

Once you’ve created an agent, you can start a call using the agent ID:

Example: Starting a Call with an Agent

// Start a call using an existing agent, pass in customer name
const startAgentCall = async (agentId) => {
  const response = await fetch(`https://api.ultravox.ai/api/agents/${agentId}/calls`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-API-Key': 'your-api-key'
    },
    body: JSON.stringify({
      templateContext: {
        customerName: "Jane Smith"
      }
    })
  });
  
  return await response.json();
};

Context Variables

If your agent’s call template uses template variables (such as in the system prompt), you can provide context values when starting a call:

body: JSON.stringify({
  templateContext: {
    customerName: "Jane Smith",
    accountType: "Premium",
    lastInteraction: "2023-10-15"
  }
})

Overriding Agent Call Parameters

When starting a call with an agent, you can override specific settings from the agent’s call template. Here are the parameters you can include in the request body:

Parameter	Description	Type
templateContext	Context for filling any mustache templates for the call	Object
initialMessages	The conversation history to start from for this call	Array
metadata	Optional metadata key-value pairs to associate with the call	Object
medium	The overridden medium used for this call (WebRTC, Twilio, etc.)	Object
joinTimeout	The overridden timeout for joining this call	String (e.g., ”60s”)
maxDuration	The overridden maximum duration of this call	String (e.g., “1800s”)
recordingEnabled	The overridden setting for whether the call should be recorded	Boolean
initialOutputMedium	The overridden medium initially used by the agent (voice/text)	String
firstSpeakerSettings	The overridden settings for the initial conversation message	Object
experimentalSettings	Experimental settings for the call	Object

Example of overriding agent settings when creating a call:

// Start a call with an agent, overriding certain settings
const response = await fetch(`https://api.ultravox.ai/api/agents/${agentId}/calls`, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'your-api-key'
  },
  body: JSON.stringify({
    // Fill template variables in the system prompt
    templateContext: {
      customerName: "Alex",
      accountType: "Premium"
    },
    // Override the maximum call duration
    maxDuration: "900s", // 15 minutes instead of default
    // Add call-specific metadata
    metadata: {
      source: "mobile_app",
      sessionId: "sess_789012",
      userTimezone: "America/New_York"
    },
    // Override the first speaker settings
    firstSpeakerSettings: {
      agent: {
        text: "Hi Alex! Welcome back to premium support. How can I help you today?"
      }
    }
  })
});

Agent API Reference

The API provides multiple endpoints for creating, listing, updating, and deleting agents. See the Agents API Reference for more information.

Conclusion

Agents in the Ultravox API provide a powerful way to create consistent, reusable voice experiences. By defining call templates with the desired configuration, you can maintain standardized voice interactions while still allowing for customization when needed.