## Introduction [00:00] Hey, what's up? My name is Greg and Enthropic just announced web search via the API. So, it used to be if you wanted to build a research bot that would go gather up-to-date information via the web, you'd have to use some form of external tool or an MCP server. This now allows you to build a web research agent without using any external tools, just solely using the cloud API. So, let's walk through a demo of what it looks like to use web search via the cloud API in Python. And then I'll walk you through how to build your own with a getting started. Then we'll talk about some of the options that you can turn on and off and some more complex use cases. I'm recording this on Thursday, May 8th, 2025. Just a few hours ago, the first American Pope was selected. Uh this is not information that is included in the training data for claude uh sonnet 3.7 given that it just happened a few hours ago. Uh so in order to have AI produce a report on this event, it would need to use web search. So here is an example of what we'll be building towards in this video. I asked the question, who is the ## Example [01:03] new pope? uh and it went out and it searched for information and gave me back text about that question with citations both the URL of where it pulled the information from and a snippet of the cited text. So let's look ## Hello World [01:19] at how you use the web search tool. Uh this just to start off with would be a hello world with the anthropic API. We have defined a question. We are instantiating a client. Make sure that you set the enthropic API key as an environment variable. And if you do have that set in your environment, then this client will pick it up by default without you needing to call it out explicitly. Uh then you just simply uh create a new message using the client. You choose the model that you want and you can pass along max tokens. This is an optional parameter just to help you control costs. Uh and then you pass in a list of messages. uh and you define the user message uh with the role user and then you pass in along a string that has the content of that message. So so if I were to run the script without the web search functionality, I get a response that says that as of the last update uh it was Pope Francis. So now let's add in ## Web Search Tool [02:15] the web search tool. And we just add an additional parameter called tools. This takes a list of tools. Here we're only going to define a single one. Uh and we're telling it to use the uh web search tool. Uh, I'm going to use ice cream here to print out the full response. This just is kind of like a very nice pretty print. Super useful helper library if you haven't used it before. Let's run this again. And I'm going to start a stopwatch here. I don't know if I'm going to edit this out or not, but I just do want you to see what the total time is. All right, so about 20 seconds to get our response back. Let's take a look at what we got. So we look our response now is uh considerably more uh fuller than what we had the first time we did. It starts off ## Web Search Response [03:07] by saying that it needs to search for the latest information. Uh and then you can start seeing some tool usage. And so uh Claude makes a query uh who is the current pope May 2025. Uh and then it finds a page here uh from CNN. Uh the page was from 13 minutes ago. Uh and it has a URL here. I do think processing through this response can be a little bit of a challenge just wrapping your head around the structure of it. And I've played with a few different ways to do this. Uh here initially I'm converting the response to a dictionary. Uh but let's run this again and take a look at what sort of the raw response looks like. All right. And here because of ice ## Search Results [03:51] cream, we get a little bit prettier of a response. Um uh but you see here that we do get an initial uh text back from Claude basically telling us that it's going to initiate the search. Then we see a lot of search results. Each one of those has a fairly large encrypted content block there. I assume that this is just because they don't want to give uh users, they basically don't want to get in the scraping game, right? They don't want to give users all of the content that they went out and scraped from websites. So, it's encrypted so that uh Anthropic can use it later, but they are not just going and scraping uh other people's content and then just passing that right off to the user uh for your use. uh which I think is sort of interesting, but it does sort of pollute the uh the responses when you're just trying to like look through these things. Um so what you need to notice is that each of the tool calls is going to return some information and then once ## Citations [04:53] the tool calls have completed, Claude then will try to sort of summarize that information uh and it will provide citations. And so this is actually the information that you will probably use as you start building with this tool. So it says here uh we have a new text block. Based on the search results I can provide you with information about the new pope. And then it gives us citations. Here's uh the cited text. And so this is just a snippet of that big block of encrypted text. uh and then it gives us the title of the page and it gives us a link uh to that page. With the Python library, they use these text blocks where they're using pyantic objects uh to format all the the data that's coming back, which does give you a lot of nicities in interacting with it. Uh but it can also it's not as straightforward as this JSON here. So if you do want to uh just work directly with JSON for instance you can run uh to JSON and if we run that again then you can see that we get uh just a big string back uh or I've also found it helpful to just run uh two dict here and here you can see the Python dictionary. Let's just look at how we did the report in the beginning of the video. Um, and I had a function here called ask claude that has a standard tool use. And then really the rest of it was just about how are we going to process through the responses. Um, is that we're going to ask the question. We get the response back. Uh, and then I'm basically constructing the report in peace meal. So I have this function called get assistant messages. I'm looking for the block type called text. And if we do see text, then I'm adding in the text of that block to my content. And then if that block contains citations then I'm grabbing the citations. And that very simply looks like this. So if there is is uh a attribute called citations on that block then I iterate through each one of those citations and I grab the URL, the title uh and the cited text. Uh and so here's my initial question and then I am just pasting in the text and the citations as they come in. And uh this would then allow me to come and click on uh these different sources to actually confirm uh the reports that I'm getting back. Okay, ## Streaming [07:19] let's talk about a few different ways that you can use this thing. Uh first, you know, it took about 20 seconds for me to get a response back there. Uh you are often going to want to implement streaming. You're going to use the async anthropic client and then you're going to create an async function to ask the question. uh and then you get the results from that as a stream and as the events come in from the string you can print those out and so if I were to uh run this and you can see here that we're just streaming the results as they're coming in and if you want to see any of ## Allowed Domains [07:52] the streaming code you can check out hih high.ai/cloud web and you'll find all of the information there the links in the description. All right let's look at another option. Uh this is another thing that people were pretty excited about. Uh there's an optional parameter here called allowed domains. So you can restrict your search to only a few different domains. Uh so for instance, if we wanted to run this uh and we only want to use Reuters, then we can get information back from just Reuters. And so then here you can see that all of the URLs cited are Reuters URLs. Now, uh it might not be surprising to you, you can't scrape all websites. So for instance, if I were to change this from Reuters to BBC.co.uk and I try running this again, I'm going to get an error saying that the following domains are not accessible to our agent. Uh you're going to get the same error if you run this on reddit.com. Uh similar to allowed domains, you can do the inverse. There's a blocked domains parameter if you want to do that as well. Uh it you can use either allowed domains or blocked domains uh in a request, but you can't use both. Uh last bit, let's talk a ## Pricing [09:09] little bit about price. Uh so Anthropics web search is going to come in at $10 per 1,000 searches plus the tokens that you pay to do all the standard API stuff that you typically would. Uh that is a quite a bit cheaper than OpenAI's solution. and theirs is uh $30 to $50 per 10,000 calls. So that's curious. I'd be interested to know uh what Anthropic is doing to be able to offer that so much cheaper. Uh and also Anthropic does not give you the difference between this search context size low, medium, high. Uh and uh whereas OpenAI is giving you a little bit more fine-tuned control there. Uh so I wonder what default Tanthropic has made here. uh I have to imagine that they're just operating with much lower search context uh in order to be able to provide that pricing for you. Uh but uh out of the box, if we're just comparing, you know, apples to apples or pricing page to pricing page, uh Anthropic does say uh $10 per thousand requests and the cheapest one for OpenAI is $30 per thousand requests.