EVI simplifies the integration of external APIs through function calling. Developers can integrate custom functions that are invoked dynamically based on the user’s input, enabling more useful conversations. There are two key concepts for using function calling with EVI: **Tools** and **Configurations** (Configs): - **Tools** are resources that EVI uses to do things, like search the web or call external APIs. For example, tools can check the weather, update databases, schedule appointments, or take actions based on what occurs in the conversation. While the tools can be user-defined, Hume also offers natively implemented tools, like web search, which are labeled as “built-in” tools. - **Configurations** enable developers to customize an EVI’s behavior and incorporate these custom tools. Setting up an EVI configuration allows developers to seamlessly integrate their tools into the voice interface. A configuration includes prompts, user-defined tools, and other settings. Tool use is only supported when specifying certain [supplemental LLMs](https://dev.hume.ai/docs/speech-to-speech-evi/configuration/language-model) within your configuration. Currently, tool use is supported by [Claude](https://docs.anthropic.com/en/docs/tool-use), [GPT](https://platform.openai.com/docs/guides/function-calling), [Gemini](https://ai.google.dev/gemini-api/docs/function-calling), and [Moonshot AI](https://platform.moonshot.ai/docs/guide/use-kimi-api-to-complete-tool-calls) models. Function calling is also available if you are using your own custom language model using the [OpenAI function calling specification](https://platform.openai.com/docs/guides/function-calling). For best results, we suggest choosing a fast and intelligent LLM that performs well on function calling benchmarks. The focus of this guide is on creating a Tool and a Configuration that allows EVI to use the Tool. Additionally, this guide details the message flow of function calls within a session, and outlines the expected responses when function calls fail. Refer to our [Configuration Guide](https://dev.hume.ai/docs/speech-to-speech-evi/configuration) for detailed, step-by-step instructions on how to create and use an EVI Configuration. Explore these sample projects to see how Tool use can be implemented in [TypeScript](https://github.com/HumeAI/hume-api-examples/tree/main/evi/evi-typescript-function-calling), [Next.js](https://github.com/HumeAI/hume-api-examples/tree/main/evi/evi-next-js-function-calling), and [Python](https://github.com/HumeAI/hume-api-examples/tree/main/evi/evi-python-function-calling). ## Setup For EVI to leverage tools or call functions, a configuration must be created with the tool’s definition. Our step-by-step guide below walks you through creating a tool and adding it to a configuration, using either a no-code approach through our [Portal](https://app.hume.ai/) or a full-code approach through our API. ###### No code ###### Full code [1](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#create-a-tool) ### Create a Tool We will first create a Tool with a specified function. In this example, we will create a tool for getting the weather. In the [Portal](https://app.hume.ai/), navigate to the [EVI Tools page](https://app.hume.ai/evi/tools). Click the **Create tool** button to begin. [2](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#fill-in-tool-details) ### Fill in Tool details Next, we will fill in the details for a weather tool named `get_current_weather`. This tool fetches the current weather conditions in a specified location and reports the temperature in either Celsius or Fahrenheit. We can establish the tool’s behavior by completing the following fields: - **Name**: Specify the name of the function that the language model will invoke. Ensure it begins with a lowercase letter and only contains letters, numbers, or underscores. - **Description**: Provide a brief description of what the function does. - **Parameters**: Define the function’s input parameters using a JSON schema. The JSON schema defines the expected structure of a function’s input parameters. Here’s an example JSON schema we can use for the [parameters](https://dev.hume.ai/reference/speech-to-speech-evi/tools/create-tool#request.body.parameters) field of a weather function: ```json { "type": "object", "required": ["location", "format"], "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "format": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "The temperature unit to use. Infer this from the user's location." } } } ``` [3](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#create-a-configuration) ### Create a Configuration Next, we will create an EVI Configuration called Weather Assistant Config. This configuration will utilize the `get_current_weather` Tool created in the previous step. See our [Configuration guide](https://dev.hume.ai/docs/speech-to-speech-evi/configuration) for step-by-step instructions on how to create a configuration. During the **Set up LLM** step, remember to select a model that supports tool use. [4](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#add-tool-to-configuration) ### Add Tool to Configuration Finally, we will specify the `get_current_weather` Tool in the Weather Assistant Config. Navigate to the **Tools** section of the EVI Config details page. Click the **Add** button to add a function to your configuration. Since we have already created a `get_current_weather` Tool in previous steps, we can simply select **Add existing** **tool…** from the dropdown to specify it. Select the tool to add `get_current_weather` to your configuration, then complete the remaining steps to create the configuration. ## Function calling In this section, we will go over the end-to-end flow of a function call within a chat session. This flow will be predicated on having specified the **Weather Assistant Config** when establishing a connection with EVI. See our [Configuration Guide](https://dev.hume.ai/docs/speech-to-speech-evi/configuration/build-a-configuration#apply-the-configuration) for details on how to apply your configuration when connecting. Check out the [TypeScript](https://github.com/HumeAI/hume-api-examples/blob/main/evi/evi-typescript-function-calling/src/handleToolCall.ts) and [Python](https://github.com/HumeAI/hume-api-examples/blob/main/evi/evi-python-function-calling/main.py) example projects for complete implementations of the weather Tool you’ll build in this tutorial. [1](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#define-a-function) ### Define a function We must first define a function for your Tool. This function will take the same [parameters](https://dev.hume.ai/reference/speech-to-speech-evi/tools/create-tool#request.body.parameters) as those specified during your Tool’s creation. For this tutorial, we will define a function that calls a weather API (e.g., the [Geocoding API](https://geocode.maps.co/)) to retrieve the weather for a designated city in a specified format. This weather function will accept `location` and `format` as its parameters. See the code below for a sample implementation: ### TypeScript ```typescript async function fetchWeather(location: string, format: string): Promise { // Fetch the location's geographic coordinates using Geocoding API const locationApiURL = `https://geocode.maps.co/search?q=${location}&api_key=${YOUR_WEATHER_API_KEY}`; const locationResponse = await fetch(locationApiURL); const locationData = await locationResponse.json(); // Extract latitude and longitude from fetched location data const { lat, lon } = locationData[0]; // Fetch point metadata using the extracted location coordinates const pointMetadataEndpoint = `https://api.weather.gov/points/${parseFloat(lat).toFixed(3)},${parseFloat(lon).toFixed(3)}`; const pointMetadataResponse = await fetch(pointMetadataEndpoint); const pointMetadata = await pointMetadataResponse.json(); // Extract weather forecast URL from point metadata const forecastUrl = pointMetadata.properties.forecast; // Fetch the weather forecast using the forecast URL const forecastResponse = await fetch(forecastUrl); const forecastData = await forecastResponse.json(); const forecast = JSON.stringify(forecastData.properties.periods); // Return the temperature in the specified format return `${forecast} in ${format}`; } ``` ### Instead of calling a weather API, you can hardcode a return value like `75F` as a means to quickly test for the sake of this tutorial. [2](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#evi-signals-function-call) ### EVI signals function call Once EVI is configured with your Tool, it will automatically infer when to signal a function call within a chat session. With EVI configured to use the `get_current_weather` Tool, we can now ask it: “what is the weather in New York?” Let’s try it out in the [EVI Playground](https://app.hume.ai/evi/playground). We can expect EVI to respond with a [User Message](https://dev.hume.ai/reference/speech-to-speech-evi/chat#receive.UserMessage) and a [Tool Call](https://dev.hume.ai/reference/speech-to-speech-evi/chat#receive.ToolCallMessage.name) message: Sample User Message ```json { "type": "user_message", "message": { "role": "user", "content": "What's the weather in New York?" } } ``` Sample Tool Call message ```json { "type": "tool_call", "tool_type": "function", "response_required": true, "tool_call_id": "call_m7PTzGxrD0i9oCHiquKIaibo", "name": "get_current_weather", "parameters": "{\"location\":\"New York\",\"format\":\"fahrenheit\"}" } ``` Currently, EVI does not support parallel function calling. Only one function call can be processed at a time. [3](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#extract-arguments-from-tool-call-message) ### Extract arguments from Tool Call message Upon receiving a [Tool Call](https://dev.hume.ai/reference/speech-to-speech-evi/chat#receive.ToolCallMessage.name) message from EVI, we will parse the [parameters](https://dev.hume.ai/reference/speech-to-speech-evi/chat#receive.ToolCallMessage.parameters) and extract the arguments. The code below demonstrates how to extract the `location` and `format` arguments, which the user-defined fetch weather function is expecting, from a received **Tool Call** message. ### TypeScript ```typescript import { Hume } from 'hume'; async function handleToolCallMessage( toolCallMessage: Hume.empathicVoice.ToolCallMessage, socket: Hume.empathicVoice.chat.ChatSocket): Promise { if (toolCallMessage.name === "get_current_weather") { // 1. Parse the parameters from the Tool Call message const args = JSON.parse(toolCallMessage.parameters) as { location: string; format: string; }; // 2. Extract the individual arguments const { location, format } = args; // ...etc. } } ``` [4](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#invoke-function-call) ### Invoke function call Next, we will pass the extracted arguments into the previously defined fetch weather function. We will capture the return value to send back to EVI: ### TypeScript ```typescript import { Hume } from 'hume'; async function handleToolCallMessage( toolCallMessage: Hume.empathicVoice.ToolCallMessage, socket: Hume.empathicVoice.chat.ChatSocket): Promise { if (toolCallMessage.name === "get_current_weather") { // 1. Parse the parameters from the Tool Call message const args = JSON.parse(toolCallMessage.parameters) as { location: string; format: string; }; // 2. Extract the individual arguments const { location, format } = args; // 3. Call fetch weather function with extracted arguments const weather = await fetchWeather(location, format); // ...etc. } } ``` [5](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#send-function-call-result) ### Send function call result Upon receiving the return value of your function, we will send a [Tool Response](https://dev.hume.ai/reference/speech-to-speech-evi/chat#send.ToolResponseMessage.content) message containing the result. The specified `tool_call_id` must match the one received in the [Tool Call](https://dev.hume.ai/reference/speech-to-speech-evi/chat#receive.ToolCallMessage.tool_call_id) message from EVI: ### TypeScript ```typescript import { Hume } from 'hume'; Let’s try it in the [EVI Playground](https://app.hume.ai/evi/playground). Enter the return value of your function in the input field below the **Tool Call** message, and click **Send Response**. In practice, you will use the actual return value from your function call. However, for demonstration purposes, we will assume a return value of “75F”. To summarize, **Tool Call** serves as a programmatic tool for intelligently signaling when you should invoke your function. EVI does not invoke the function for you. You will need to define a function, invoke the function, and pass the return value of your function to EVI via a [Tool Response](https://dev.hume.ai/reference/speech-to-speech-evi/chat#send.ToolResponseMessage.content) message. EVI will generate a response based on the content of your message. ## Using built-in tools User-defined tools allow EVI to identify when a function should be invoked, but you will need to invoke the function itself. On the other hand, Hume also provides built-in tools that are natively integrated. This means that you don’t need to define the function; EVI handles both determining when the function needs to be called and invoking it. Hume supports the following built-in tools: - **web_search:** Enables EVI to search the web for real-time information when needed. - **hang_up:** Closes the WebSocket connection with status code `1000` when appropriate (e.g., after detecting a farewell, signaling the end of the conversation). This section explains how to specify built-in tools in your configurations and details the message flow you can expect when EVI uses a built-in tool during a chat session. [1](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#specify-built-in-tool-in-evi-configuration) ### Specify built-in tool in EVI configuration Let’s begin by creating a configuration which includes the built-in web search tool. To specify the web search tool in your EVI configuration, during the **Add tools** step, ensure **Web search** is enabled. Refer to our [Configuration Guide](https://dev.hume.ai/docs/speech-to-speech-evi/configuration/build-a-configuration#create-a-configuration) for more details on creating a configuration. Alternatively, you can specify the built-in tool by making a POST request to [/configs](https://dev.hume.ai/reference/speech-to-speech-evi/configs/create-config) with the following request body: ```json { "name": "Web Search Config", "language_model": { "model_provider": "OPEN_AI", "model_resource": "gpt-5-mini" }, "builtin_tools": [ { "name": "web_search", "fallback_content": "Optional fallback content to inform EVI’s spoken response if web search is not successful." } ] } ``` Upon success, expect EVI to return a response similar to this example: ```json { "id": "3a60e85c-d04f-4eb5-8076-fb4bd344d5d0", "version": 0, "version_description": null, "name": "Web Search Config", "created_on": 1714421925626, "modified_on": 1714421925626, "prompt": null, "voice": null, "language_model": { "model_provider": "OPEN_AI", "model_resource": "gpt-5-mini", "temperature": null }, "tools": [], "builtin_tools": [ { "tool_type": "BUILTIN", "name": "web_search", "fallback_content": "Optional fallback content to inform EVI’s spoken response if web search is not successful." } ] } ``` [2](https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use#evi-uses-built-in-tool) ### EVI uses built-in tool Now that we’ve created an EVI configuration which includes the built-in web search tool, let’s test it out in the [EVI Playground](https://app.hume.ai/evi/playground). Try asking EVI a question that requires web search, like “what is the latest news with AI research?” EVI will send a response generated from the web search results: Let’s review the message flow for when web search is invoked. ```json // 1. User asks EVI for the latest news in AI research { "type": "user_message", "message": { "role": "user", "content": "What is the latest news with AI research?" } } // 2. EVI infers it needs to use web search, generates a search query, and invokes Hume's native web search function { "name": "web_search", "parameters": "{\"query\":\"latest news AI research\"}", "tool_call_id": "call_zt1NYGpPkhR7v4kb4RPxTkLn", "type": "tool_call", "tool_type": "builtin", "response_required": false } // 3. EVI sends back the web search results { "type": "tool_response", "tool_call_id": "call_zt1NYGpPkhR7v4kb4RPxTkLn", "content": "{ \"summary\":null, \"references\": [{\"content\":\"Researchers have demonstrated a new method...etc.\", \"url\":\"https://www.sciencedaily.com/news/computers_math/artificial_intelligence/\", \"name\":\"Artificial Intelligence News -- ScienceDaily\"}] }", "tool_name": "web_search", "tool_type": "builtin" } // 4. EVI sends a response generated from the web search results { "type": "assistant_message", "message": { "role": "assistant", "content": "Oh, there's some interesting stuff happening in AI research right now." } } { "type": "assistant_message", "message": { "role": "assistant", "content": "Just a few hours ago, researchers demonstrated a new method using AI and computer simulations to train robotic exoskeletons." } } ``` ## Interruptibility Function calls can be interrupted to cancel them or to resend them with updated parameters. ### Canceling a function call Just as EVI is able to infer when to make a function call, it can also infer from the user’s input when to cancel one. Here is an overview of what the message flow would look like: Cancel function call message flow ```json // 1. User asks what the weather is in New York { "type": "user_message", "message": { "role": "user", "content": "What's the weather in New York?" } } // 2. EVI infers it is time to make a function call { "type": "tool_call", "tool_type": "function", "response_required": true, "tool_call_id": "call_m7PTzGxrD0i9oCHiquKIaibo", "name": "get_current_weather", "parameters": "{\"location\":\"New York\",\"format\":\"fahrenheit\"}" } // 3. User communicates sudden disinterest in the weather { "type": "user_message", "message": { "role": "user", "content": "Actually, never mind." } } // 4. EVI infers the function call should be canceled { "type": "assistant_message", "message": { "role": "assistant", "content": "If you change your mind or need any weather information in the future, feel free to let me know." } } ``` ### Updating a function call Sometimes we don’t necessarily want to cancel the function call, and instead want to update the parameters. EVI can infer the difference. Below is a sample flow of interrupting the interface to update the parameters of the function call: Update function call message flow ```json // 1. User asks EVI what the weather is in New York { "type": "user_message", "message": { "role": "user", "content": "What's the weather in New York?" } } // 2. EVI infers it is time to make a function call { "type": "tool_call", "tool_type": "function", "response_required": true, "tool_call_id": "call_m7PTzGxrD0i9oCHiquKIaibo", "name": "get_current_weather", "parameters": "{\"location\":\"New York\",\"format\":\"fahrenheit\"}" } // 3. User communicates to EVI they want the weather in Los Angeles instead { "type": "user_message", "message": { "role": "user", "content": "Actually, Los Angeles." } } // 4. EVI infers the parameters to function call should be updated { "type": "tool_call", "response_required": true, "tool_call_id": "call_5RWLt3IMQyayzGdvMQVn5AOQ", "name": "get_current_weather", "parameters": "{\"location\":\"Los Angeles\",\"format\":\"celsius\"}" } // 5. User sends results of function call to EVI { "type": "tool_response", "tool_call_id":"call_5RWLt3IMQyayzGdvMQVn5AOQ", "content":"72F" } // 6. EVI sends response container function call result { "type": "assistant_message", "message": { "role": "assistant", "content": "The current weather in Los Angeles is 72F." } } ``` ## Handling errors It’s possible for tool use to fail. For example, it can fail if the [Tool Response](https://dev.hume.ai/reference/speech-to-speech-evi/chat#send.ToolResponseMessage.content) message content was not in UTF-8 format or if the function call response timed out. This section outlines how to specify fallback content to be used by EVI to communicate a failure, as well as the message flow for when a function call failure occurs. ### Specifying fallback content When defining your Tool, you can specify fallback content within the Tool’s `fallback_content` field. When the Tool fails to generate content, the text in this field will be sent to the LLM in place of a result. To accomplish this, let’s update the Tool we created during setup to include fallback content. We can accomplish this by publishing a new version of the Tool via a POST request to [/tools/{id}](https://dev.hume.ai/reference/speech-to-speech-evi/tools/create-tool-version): ```json { "version_description": "Adds fallback content", "description": "This tool is for getting the current weather.", "parameters": "{ \"type\": \"object\", \"properties\": { \"location\": { \"type\": \"string\", \"description\": \"The city and state, e.g. San Francisco, CA\" }, \"format\": { \"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"], \"description\": \"The temperature unit to use. Infer this from the users location.\" } }, \"required\": [\"location\", \"format\"] }",