LlamaIndex Agent Step-Wise Execution Framework With Agent Runners & Agent Workers
The LlamaIndex lower-level agent API provides an array of functionalities extending beyond the mere execution of a user query from start to finish.
Introduction
LlamaIndex provides a comprehensive agent API with advanced functionalities that goes beyond just executing user queries.
The step-wise approach allows for precise control and navigation of the agent. Hence empowering users to create dependable software systems tailored to their data.
LlamaIndex’s design draws inspiration from various sources including the Agent Protocol, the OpenAI Assistants API, and numerous agent research papers.
Step-Wise Agent Architecture
The step-wise agent is constituted by two objects, the AgentRunner and the AgentWorker.
The AgentRunner object interact with the AgentWorkers.
AgentRunner
AgentRunners are orchestrators that holds:
State
Conversational Memory
Create & Maintain Tasks
Run steps through each task
User Interface
AgentWorker
The AgentWorker controls the step-wise execution of tasks.
AgentWorkers generates what the next step is, based in the input.
AgentWorkers are initialised with parameters and act base on the state received from the task object.
The overarching AgentRunner is responsible for calling an AgentWorker and collecting the results.
A few classes as defined by LlamaIndex:
Task: A high-level task which takes in a user query and passes along other information, like for instance memory.
TaskStep: A TaskSep represents a single step which feeds in as an input to the AgentWorker, and subsequently getting back a TaskStepOutput. Completing a single Task can involve multiple TaskSteps.
TaskStepOutput: This class holds output from a given step execution. The TAskStepOutput outputs whether or not a task is completed.
Working Code Example
Below is a simple working example of the step-wise agent…you can copy and paste this code into a notebook and run it. Just be sure to add your OpenAI API key.
%pip install llama-index-agent-openai
%pip install llama-index-llms-openai
!pip install llama-index
import json
from typing import Sequence, List
from llama_index.llms.openai import OpenAI
from llama_index.core.llms import ChatMessage
from llama_index.core.tools import BaseTool, FunctionTool
import nest_asyncio
###########################################
nest_asyncio.apply()
def multiply(a: int, b: int) -> int:
"""Multiple two integers and returns the result integer"""
return a * b
multiply_tool = FunctionTool.from_defaults(fn=multiply)
def add(a: int, b: int) -> int:
"""Add two integers and returns the result integer"""
return a + b
add_tool = FunctionTool.from_defaults(fn=add)
tools = [multiply_tool, add_tool]
###########################################
import os
import openai
os.environ['OPENAI_API_KEY'] = str("<Your API Key Goes Here>")
###########################################
llm = OpenAI(model="gpt-3.5-turbo")
###########################################
from llama_index.core.agent import AgentRunner
from llama_index.agent.openai import OpenAIAgentWorker, OpenAIAgent
agent = OpenAIAgent.from_tools(tools, llm=llm, verbose=True)
###########################################
response = agent.chat("What is (121 * 3) + 42?")
And the response:
Added user message to memory: What is (121 * 3) + 42?
=== Calling Function ===
Calling function: multiply with args: {"a": 121, "b": 3}
Got output: 363
========================
=== Calling Function ===
Calling function: add with args: {"a": 363, "b": 42}
Got output: 405
========================Conclusion
While leading-edge software concepts may not always be deemed viable for current production implementations, these developments often serves as a foundational to establish future methods and approaches.
Understanding these fundamentals can provide valuable insights into the evolution of software development and what the future might look like.
And by studying methodologies from platforms like LlamaIndex and LangChain, developers can gain a deeper appreciation for the principles that underpin modern innovations.
Ultimately, bridging the gap between cutting-edge technologies and traditional methods creates a balanced and informed approach to software development.
I’m currently the Chief Evangelist @ Kore AI. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.