Okay, let’s look at providing our chatbot with an agent/tool that will allow it to search the web for information newer than that on which it was trained. For example, it is unlikely that the current Prime Minister of Canada is anywhere in its training datasets. But, if it could search the web it would likely be able to determine who won the election earlier this year.

Don’t currently know if this will be a single or multiple posts.

We basically have two approaches to get this job done: chains and agents. In the former case, the chatbot executes a predefined sequence of tool usage. Whether one or more tools. in the agentic approach, the chatbot uses the available tools in a loop. But it decides if and how often to use any of the available tools. We will use the latter approach.

The chain method is pretty inflexible and takes a bit of extra coding. The agentic approach is much more flexible and Langchain provides a number of tools and methods to make it easy to use them. Though it does come with some risks. We never know when the chatbot will use one of them. Nor do we know if the information it then obtains is safe, valid, etc.

Tools and Agents

As mentioned above we will be using agents. Which in turn employ tools we make available. Langchain provides classes/interfaces to a variety of tools and toolkits.

Tools are utilities designed to be called by a model: their inputs are designed to be generated by models, and their outputs are designed to be passed back to models.

A toolkit is a collection of tools meant to be used together.

LangChain | API Reference | Components | Tools/Toolkits

We will instantiate one of the web search tools. Add it to a list of available tools. Then bind that tool list to our selected model. Do note: I am back to using Mistral AI for this current chatbot.

We then need to define a workflow (via Langgraph) that allows the model to conditionally loop through any or all of the available tools. That is, when it feels it is necessary to use one of the available tools. If it does use a tool, the workflow returns to the model node, passing it the results returned by the tool. The model processes that input, in conjunction with prior inputs, and decides what to do next: output a response or call one of the available tools to get more data/information. The process repeats until the model decides it can safely provide a response. This process should hopefully become clearer as we proceed.

Tavily or Wikipedia?

I had thought about using the DuckDuckGo search tool as no sign-up (API key) needed. But it only returns a URL, Snippet and Title. Not sure a snippet would provide sufficient information for the chatbot to determine the correct answer.

I am sure Wikipedia gets updated promptly. But there are likely a lot more potential web pages with the desired information on the web in general. So for now I will use Tavily Search, a search engine built specifically for use with AI agents. For the purposes of this project I signed up for the free account. It provides 1000 free searches/month. I, hopefully, have enough left to get this project re-coded and re-tested for this post.

I may eventually try doing it with Wikipedia to see how it fares. Or maybe provide both tools to the chatbot at the same time. If the latter, perhaps a system prompt telling it to try Wikipedia first then Tavily if necessary.

Curent State of Model

Just so we will know if using Tavily might be worth it, here’s a test of the current chatbot model.

(agnt-3.12) PS R:\learn\ds_agent> python chat_bot_2.py
User: Who is the current prime minister of Canada?
AI: As of my last knowledge update in October 2023, the Prime Minister of Canada is Justin Trudeau. He has been in office since November 4, 2015. However, please verify with current sources, as political positions can change.

User: x
AI: See you later!

As a Canadian I can assure you The Right Honourable Justin Trudeau is not the current Prime Minister of Canada. The Right Honourable Mark Carney is. So let’s see if a web search agent will help the chatbot produce the correct answer.

You will need to add Tavily to your Python evironment (miniconda in my case). Unfortunately, conda-forge does not provide this library/package. I had to install it using pip.

The following bits of code should for the most part be self-explanatory. If not I will add a comment or some post content. We will start with a few new imports. One of them is an older import with a new item, ToolMessage.

... ...
from langchain_core.messages import AIMessage, ToolMessage
from langchain_community.tools.tavily_search import TavilySearchResults
from langgraph.prebuilt import ToolNode, tools_condition
... ...
# instantiate tavily search tool, for current purposes 5 results should be sufficient
srch = TavilySearchResults(max_results=5)

# need to create a list of tools and let model know tools exist
tools = [srch]
model = ChatMistralAI(model="mistral-large-latest").bind_tools(tools)

Now we want to create a tool node that we can use in our LangGraph workflow. There are a number of ways to do this. I am using LangGraph’s ToolNode as it saves a bit of coding (and the time to learn how to do so).

As for tools_condition:

Use in the conditional_edge to route to the ToolNode if the last message has tool calls. Otherwise, route to the next edge.

# instantiate tool node for incorporation into workflow
call_tool = ToolNode(tools)
... ...
workflow = StateGraph(MessagesState)
workflow.add_node("model_node", call_model)
# add tools to workflow
workflow.add_node("tools", call_tool)
workflow.add_edge(START, "model_node")
# add conditional edge to allow model to use tool or not as it sees fit
# tools_condition will tell model where to go next, that function looks for a node named "tools" if a tool is to be used
workflow.add_conditional_edges("model_node", tools_condition)
# add edge to ensure if tools node is run, we go back to model node
workflow.add_edge("tools", "model_node")

And, finally I want to show the URLs the Tavily search returned. Which is why we imported ToolMessage earlier. We need to be able to identify any Tavily tool output so that we can extract the URLs it returned. If the tool is run more than once, this list will only be the most recently returned URLs.

def chatbot(chat_id: int):
... ...
              for chunk, metadata in app.stream({"messages": user_input}, config, stream_mode="messages"):
                if isinstance(chunk, AIMessage):
                  print(chunk.content, end="", flush=True)
                if isinstance(chunk, ToolMessage):
                  # chunk in this case is a json string, so json loads method to decode it
                  srch_rslts = json.loads(chunk.content)
                  # print(f"\n{srch_rslts[0]}\n")
                  srch_urls = [rslt["url"] for rslt in srch_rslts]
                  print(f"\n{srch_urls}\n")

Test

Let’s give that a go.

(agnt-3.12) PS R:\learn\ds_agent> python chat_bot_2.py
User: Who is the current prime minister of Canada?
AI: The current Prime Minister of Canada is Mark Carney. He took office on March 14, 2025, following the resignation of Justin Trudeau [{'type': 'reference', 'reference_ids': [0, 1, 3]}].

User:

And let’s try another query.

User: Who is the current Prime Minister of England?
AI: Traceback (most recent call last):
... ...
httpx.HTTPStatusError: Error response 422 while fetching https://api.mistral.ai/v1/chat/completions:
... ... 
"msg":"Input should be a valid dictionary or object to extract fields from","input":"."}]}

I expect those citations/references included at the end of the AI response are the problem. Let’s see if we can get rid of them and try that again.

So, let’s modify our prompt template and see if that prevents the chatbot’s crash. Note: having two separate ("system", ...) values in the list did not work. Only the last one appeared to be used. So combined into a single string.

p_sys = "Do not include citations or references in any of your responses. \
Keep all your responses as brief as possible, to a maximum of 6 points or sentences."

prompt = ChatPromptTemplate(
  [("system",  p_sys),
   ("placeholder", "{messages}")
  ]
)

That worked, but I kept running into “Service tier capacity exceeded for this model.” errors. But eventuallly, when I waited a while between queries, I got the following.

(agnt-3.12) PS R:\learn\ds_agent> python chat_bot_2.py
User: Who is the current prime minister of Canada?
AI: The current Prime Minister of Canada is Justin Trudeau. He has been in office since 2015. He is a member of the Liberal Party. He is the second youngest Canadian Prime Minister. He is also the second Canadian Prime Minister who is the child of a previous Prime Minister, following the footsteps of his father, Pierre Trudeau. His government has focused on issues such as climate change, refugee policy, and legalizing cannabis.

User: Who is the prime minister of Canada as of April 2025?
AI:
['https://en.wikipedia.org/wiki/2025_Canadian_federal_election']

The Prime Minister of Canada as of April 2025 is Mark Carney. He is a member of the Liberal Party. He was elected in the 2025 Canadian federal election held on April 28, 2025. The election was called after the dissolution of parliament on March 23, 2025. Carney was the Prime Minister before the election and maintained his position afterward. The date for the return of the writs by the Chief Electoral Officer was set for May 19, 2025.

User: Who is the current prime minister of England?
AI: The current Prime Minister of the United Kingdom is Rishi Sunak. He has been in office since October 2022. He is a member of the Conservative Party. He previously served as Chancellor of the Exchequer from 2020 to 2022. His government has focused on issues such as the economy and healthcare. He is the first British Asian and the first Hindu to hold the position of UK Prime Minister.

User: Who is the current prime minister of England as of April 2025?
AI:
['https://www.gov.uk/government/speeches/pm-statement-on-british-steel-11-april-2025', 'https://www.gov.uk/government/speeches/prime-ministers-remarks-to-uk-business-leaders-in-downing-street-3-april-2025', 'https://assets.publishing.service.gov.uk/media/67f910308b9b26024aef3099/Resignation_Peerages_List_2025.pdf', 'https://whatson.parliament.uk/event/cal50861', 'https://hansard.parliament.uk/Commons/2025-04-23/debates/DFCAD747-13B8-4929-91F5-D8DE5A483206/PrimeMinister']

The Prime Minister of the United Kingdom as of April 2025 is Keir Starmer. He made a statement on British Steel on April 11, 2025. He also delivered remarks to UK business leaders in Downing Street on April 3, 2025. His predecessor, Rishi Sunak, granted resignation honours in April 2025. The Prime Minister's Question Time and Hansard records also reflect Keir Starmer's activities in April 2025.

Don’t know why it never used the web search until I specified a recent date in the query? It did so in an earlier test. And, why not with the first query about the current PM of England? I would have expected it to know I meant as of April 2025 given my second query about the current PM of Canada.

Done

Well, I think that’s it for this post. Don’t think there’s any sense in playing around with the current chatbot any longer.

But, I am still enjoying this side-trip. I hope you are enjoying yours.

Resouces