LangChain Search AI Agent Using GPT-4o-mini

In this article is an end-to-end example of a LangChain Agent using OpenAI’s new Small Model for Web Search & Question Answering With LangSmith Integration.

In Short

The LangChain Agent makes use of web search to answer user questions. This agent has access to a single tool, which is a Tavily API to search the web.

tools = [TavilySearchResults(max_results=1)]

Considering the question: What year was IBM founded and in what year was Apple founded? and the response to the question (as seen below), a total number of 165 tokens were used at $0.0000594.

With 88 prompt tokens at $0.0000132 and 77 completion tokens at $0.0000462. The time to first token 1389 ms, which is important considering that responses may be streamed for a better user experience and hence the time to the first token denotes the wait time for the user. With output tokens being about four times more expensive than input tokens.

Basic Setup

Below I setup the LangSmith integration as defined in the Python code. The langChain API Key can be gleaned from the LangSmith console, and the project name is an arbitrary name given to the project.

import os
from uuid import uuid4
unique_id = uuid4().hex[0:8]
os.environ[“LANGCHAIN_TRACING_V2”] = “true”
os.environ[“LANGCHAIN_PROJECT”] = f”OpenAI_SM_Agent_1"
os.environ[“LANGCHAIN_ENDPOINT”] = “https://api.smith.langchain.com"
os.environ[“LANGCHAIN_API_KEY”] = “<LangSmith API Key Goes Here>”

The OpenAI and Tavily web search API keys need to be added…

os.environ[“OPENAI_API_KEY”] = getpass.getpass()
os.environ[“TAVILY_API_KEY”] = getpass.getpass()

The model is set to:

llm = ChatOpenAI(model=”gpt-4o-mini”)

LangSmith

Within LangSmith, the OpenAI_SM_Agent_1 project is accessed, from there five runs visible. Selecting the first run, each step in the chain is visible, with the cost of each step and the execution time/latency.

LangSmith also allows for the creation of datasets, output can be annotated, set to correct and incorrect and auto evaluations can be run to determine the correctness.

Agent Execution

The agent decomposes the compound user question into sub-questions which are the then individually answered.

This code demonstrates how to set up and use a language model (LLM) from OpenAI, specifically with the LangChain framework. It also integrates the Tavily search tool for enhanced information retrieval.

Web search LangChain agent:

### Install Necessary Packages
pip install -qU langchain-openai langchain langchain_community
### Import Required Modules
import getpass
import os
### Set Environment Variables for API Keys
os.environ["OPENAI_API_KEY"] = getpass.getpass()
os.environ["TAVILY_API_KEY"] = getpass.getpass()
### Initialize the OpenAI LLM
from langchain_openai import ChatOpenAI
###
llm = ChatOpenAI(model="gpt-4o-mini")
### Import Necessary LangChain Components
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_core.prompts import ChatPromptTemplate
### Set Up the Tavily Search Tool
tools = [TavilySearchResults(max_results=1)]
### Create a Chat Prompt Template
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant. Make sure to use the tavily_search_results_json tool for information.",
        ),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
        ("placeholder", "{agent_scratchpad}"),
    ]
)

### Construct the Tools agent
agent = create_tool_calling_agent(llm, tools, prompt)

### Create an agent executor by passing in the agent and tools
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

### Invoke the Agent Executor:
agent_executor.invoke({"input": "What year was IBM founded and in what year was Apple founded?"})

And the output from the agent. It is clear how there are two queries, year IBM founded and year Apple founded with the final output being consolidated with web references.

> Entering new AgentExecutor chain...

Invoking: `tavily_search_results_json` with `{'query': 'year IBM founded'}`


[{'url': 'https://en.wikipedia.org/wiki/History_of_IBM', 'content': 'Recognizing this trend, management, with the support of the Board of Directors, began to implement a plan to split IBM into increasingly autonomous business units (e.g. processors, storage, software, services, printers, etc.) to compete more effectively with competitors that were more focused and nimble and had lower cost structures.[citation needed]\nIBM also began spinning off its many divisions into autonomous subsidiaries (so-called "Baby Blues") in an attempt to make the company more manageable and to streamline IBM by having other investors finance those companies.[180][181] These included AdStar, dedicated to disk drives and other data storage products (on creation the largest data storage business in the world);[182] IBM Application Business Systems, dedicated to mid-range computers; IBM Enterprise Systems, dedicated to mainframes; Pennant Systems, dedicated to mid-range and large printers; Lexmark, dedicated to small printers, keyboards, and typewriters (such as the Selectric); and more.[183] Lexmark was acquired by Clayton & Dubilier in a leveraged buyout shortly after its formation.[184]\nIn September 1992, IBM combined and spun off their various non-mainframe and non-midrange, personal computer manufacturing divisions into an autonomous wholly owned subsidiary known as the IBM Personal Computer Company (IBM PC Co.).[185][186] Purchases were often instigated by middle managers and senior staff who saw the potential – once the revolutionary VisiCalc spreadsheet, the killer app, had been surpassed by a far more powerful and stable product, Lotus 1-2-3.[citation needed]\nIBM\'s dominance of the mainframe market in Europe and the US encouraged existing customers to buy the PC,[171][173] and vice versa; as sales of what had been an experiment in a new market became a substantial part of IBM\'s financials, the company found that customers also bought larger IBM computers.[174][167][162] Unlike the BUNCH and other rivals IBM quickly adjusted to the retail market,[171][175] with its own sales force competing with outside retailers for the first time.[162] The US government\'s case sustained by four US Presidents and their Attorneys General was dropped as "without merit" in 1982 by William Baxter, US President Reagans\' Assistant Attorney General in charge of the Antitrust Division of the U.S. Department of Justice.[315]\nCDC filed an antitrust lawsuit against IBM in Minnesota\'s federal court alleging that IBM had monopolized the market for computers in violation of section 2 of the Sherman Antitrust Act by among other things announcing products it could not deliver.[316] A 1965 internal IBM memo by an IBM attorney noted that Control Data had publicly blamed its declining earnings on IBM, "and its frequent model and price changes. By 1981 its stock price had declined by 22%.[166] IBM\'s earnings for the first half of the year grew by 5.3% – one third of the inflation rate – while those of DEC grew by more than 35%.[165] Although IBM began selling minicomputers,[170] in January 1982 the Justice Department ended the antitrust suit because, The New York Times reported, the government "recognized what computer experts and securities analysts had long since concluded: I.B.M. no longer dominates the computer business".[147]\nIBM wished to avoid the same outcome with the new personal computer industry.[169] He asserts that the company was not plundered, its leased machinery was not confiscated, and IBM continued to receive funds through its Geneva-based subsidiary.[86] Black argues that IBM persisted in its business relations with the Nazi regime beyond the point where they should have ceased, maintaining and expanding services to the Third Reich[86] until the seizure of Dehomag following the United States\' declaration of war against Germany in 1941.[citation needed]\nIBM countered these claims by stating that the allegations were based on known facts and previously disclosed documents, asserting the absence of new revelations.'}]
Invoking: `tavily_search_results_json` with `{'query': 'year Apple founded'}`


[{'url': 'https://en.wikipedia.org/wiki/History_of_Apple_Inc.', 'content': 'The head of a retail chain said "It appears that IBM had a better understanding of why the Apple II was successful than had Apple".[78] Gene Amdahl predicted that Apple would be another of the many "brash young companies" that IBM had defeated.[88]\nBy 1984 the press called the two companies archrivals,[89] but IBM had $4 billion in annual PC revenue, more than twice that of Apple and as much as the sales of it and the next three companies combined.[90] A Fortune survey found that 56% of American companies with personal computers used IBM PCs, compared to 16% for Apple.[91] Small businesses, schools, and some homes became the II\'s primary market.[76]\nXerox PARC and the Lisa[edit]\nApple Computer\'s business division was focused on the Apple III, another iteration of the text-based computer. In 1979, the Apple II was chosen to be the desktop platform for the first "killer application" of the business world: VisiCalc, a spreadsheet.[55] So important that the Apple II became what John Markoff described as a "VisiCalc accessory",[57] the application created a business market for the computer and gave home users an additional reason to buy it: compatibility with the office.[55] Before VisiCalc, Apple had been a distant third place competitor to Commodore and Tandy.[58][59]\nThe Apple II was one of the three "1977 Trinity" computers generally credited with creating the home computer market (the other two being the Commodore PET and the Tandy Corporation TRS-80).[60] After giving their results for the first quarter of 2011, Microsoft\'s net profits of $5.2 billion were lower for the quarter than those of Apple, which earned $6 billion in net profit for the quarter.[182][183] The late April announcement of profits by the companies marked the first time in 20 years that Microsoft\'s profits had been lower than Apple\'s,[184] a situation described by Ars Technica as "unimaginable a decade ago".[182]\nThe Guardian reported that one of the reasons for the change was because PC software, where Microsoft dominates, has become less important compared to the tablet and smartphone markets, where Apple has a strong presence.[184] One reason for this was a surprise drop in PC sales in the quarter.[184] Nonetheless, he kept his word and paid the two Steves the money promised.[39][37][38][40]\nThe Apple I went on sale in July 1976 as an assembled circuit board with a retail price of $666.66.[41][42][43] Wozniak later said he had had no idea about the relation between the number and the mark of the beast, and that he came up with the price because he liked repeating digits.[39] About 200 units of the Apple I were eventually sold.[44]\nThe Apple I computer had some notable features, including the use of a TV display, whereas many machines had no display at all. The new corporation bought out the partnership the two Steves had formed nine months earlier.[46]\nIn February 1977, Markkula recruited Michael Scott from National Semiconductor to serve as the first president and CEO of Apple Computer, as the two Steves were both insufficiently experienced and he was not interested in taking that position himself.[47][48]\nThat same month, Wozniak resigned from his job at Hewlett-Packard to work full-time for Apple.[46][49]\nApple II[edit]\nAlmost as soon as Apple had started selling its first computers, Wozniak moved on from the Apple I and began designing a greatly improved computer: the Apple II.[45] Wozniak completed a working prototype of the new machine by August 1976.[38][50]'}]IBM was founded in 1911, originally as the Computing-Tabulating-Recording Company (CTR), and was renamed International Business Machines Corporation (IBM) in 1924. 

Apple Inc. was founded in 1976. 

For more detailed histories, you can visit the following links: 
- [IBM History](https://en.wikipedia.org/wiki/History_of_IBM)
- [Apple History](https://en.wikipedia.org/wiki/History_of_Apple_Inc.)

> Finished chain.
{'input': 'What year was IBM founded and in what year was Apple founded?',
 'output': 'IBM was founded in 1911, originally as the Computing-Tabulating-Recording Company (CTR), and was renamed International Business Machines Corporation (IBM) in 1924. \n\nApple Inc. was founded in 1976. \n\nFor more detailed histories, you can visit the following links: \n- [IBM History](https://en.wikipedia.org/wiki/History_of_IBM)\n- [Apple History](https://en.wikipedia.org/wiki/History_of_Apple_Inc.)'}

Below just the input and output text from the agent.

{'input': 'What year was IBM founded and in what year was Apple founded?',
 'output': 'IBM was founded in 1911, originally as the Computing-Tabulating-Recording Company (CTR), and was renamed International Business Machines Corporation (IBM) in 1924. \n\nApple Inc. was founded in 1976. \n\nFor more detailed histories, you can visit the following links: \n- [IBM History](https://en.wikipedia.org/wiki/History_of_IBM)\n- [Apple History](https://en.wikipedia.org/wiki/History_of_Apple_Inc.)'}

In summary, this code sets up an LLM-based agent using the LangChain framework, integrates a custom search tool, and uses it to answer a specific query.

Finally

GPT-4o-mini Considerations

• Local Control: Open-source SLMs offer the advantage of running models locally with full control over inferencing, which is not applicable for OpenAI’s commercial hosted API model.

• OpenAI Focus: OpenAI emphasises speed, cost, and capability while following the trend of smaller models.

• Competitors: Highly capable open-source text-based SLMs like Orca-2, Phi3, and TinyLlama are notable competitors.

• Differentiators: GPT-4o-mini stands out with its cost, speed, capability, and available modalities.

Advantages of GPT-4o-mini

• Text & Vision Support: GPT-4o-mini supports text and vision inputs in both the API and playground.

• Future Expansions: Upcoming features include handling text, image, video, and audio inputs and outputs.

• Large Context Window: The model boasts a context window of 128K tokens and has knowledge up to October 2023.

• Multi-Language Capabilities: The model supports multiple languages.

• Enhanced Inference Speeds: Improved speeds make it suitable for various applications.

• Cost-Effective: The combination of speed and cost makes it ideal for agentic applications with multiple parallel calls. It costs 15 cents per million input tokens and 60 cents per million output tokens.

• Fine-Tuning: Fine-tuning for GPT-4o-mini will be available soon.