Tool & Function Calling
Use tools in your prompts
Tool calls (also known as function calls) give an LLM access to external tools. The LLM does not call the tools directly. Instead, it suggests the tool to call. The user then calls the tool separately and provides the results back to the LLM. Finally, the LLM formats the response into an answer to the user’s original question.
OpenRouter standardizes the tool calling interface across models and providers.
For a primer on how tool calling works in the OpenAI SDK, please see this article, or if you prefer to learn from a full end-to-end example, keep reading.
Tool Calling Example
Here is Python code that gives LLMs the ability to call an external API — in this case Project Gutenberg, to search for books.
First, let’s do some basic setup:
Define the Tool
Next, we define the tool that we want to call. Remember, the tool is going to get requested by the LLM, but the code we are writing here is ultimately responsible for executing the call and returning the results to the LLM.
Note that the “tool” is just a normal Python function. We then write a JSON ‘spec’ compatible with the OpenAI function calling spec. We’ll pass that spec to the LLM so that it knows this tool is available and how to use it. It will request the tool when needed, along with any arguments. We’ll then marshal the tool call locally, make the function call, and return the results to the LLM.
Tool use and tool results
Let’s make the first OpenRouter API call to the model:
The LLM responds with a finish reason of tool_calls, and a tool_calls array. In a generic LLM response-handler, you would want to check the finish reason before processing tool calls, but here we will assume it’s the case. Let’s keep going, by processing the tool call:
The messages array now has:
- Our original request
- The LLM’s response (containing a tool call request)
- The result of the tool call (a json object returned from the Project Gutenberg API)
Now, we can make a second OpenRouter API call, and hopefully get our result!
The output will be something like:
We did it! We’ve successfully used a tool in a prompt.
A Simple Agentic Loop
In the example above, the calls are made explicitly and sequentially. To handle a wide variety of user inputs and tool calls, you can use an agentic loop.
Here’s an example of a simple agentic loop (using the same tools
and initial messages
as above):