by April 12, 2005 0 comments



Yahoo has given in to the long-standing demand of many developers and finally launched its Web services. The Yahoo Search Web Services allows developers to query Yahoo’s search engine from different programming languages. Google has been providing this facility for quite some time now with the Google Web API. The Yahoo Search Web Services, however, differs significantly from
Google’s.

For starters, Google’s API is SOAP-based while Yahoo chose to go with REST. The Google API can be used only for Web search meaning that services such as Google News, Google Image Search etc. cannot be accessed via the API. Yahoo, however, allows developers to search five ‘channels’ of content-the regular Web search, Image search, Local search, News search and Video search. While Google restricts developers to 1000 queries a day, Yahoo offers a generous 5000 queries per day in each of the channels. Both, however, require you to register for access to the APIs and are free of cost. Let’s see how it works. 

You can access these services at their respective URLs and obtain results by specifying your query as a parameter. The URLs of the various services are as below (at the time of writing this article). 

In addition to the query, you need to specify your application ID (visit http://developer.yahoo.net 
to get one free). Then there are certain other parameters that depend on the service you’re using. A list of various parameters that Web search supports/requires is given in Table 2.

Direct Hit!
Applies to: Developers 
USP:
Understand how to write code in ASP.NET to query various channels in Yahoo’s search engine
Links:
www.developer.yahoo.net 

One thing to note here with the query parameter is that it can include the full search language like ‘site:’, which makes it quite powerful. Let’s take an example to make things a bit clearer. 

Let’s say you want to search for the word ‘technology’ on the site pcquest.com and you want to fetch results number 8 and 9. This is how your request URL may look like:

http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=technology+site:pcquest.com&results=2&start=8 
Let’s see what response do we get to the above request.

<?xml version=”1.0″ encoding=”UTF-8″?>
<ResultSet xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance
" xmlns=”urn:yahoo:srch” xsi:schemaLocation="urn:yahoo:srch http://api.search.yahoo.com/WebSearchService/V1/WebSearchResponse.xsd
" totalResultsAvailable=”778″ totalResultsReturned=”2″ firstResultPosition=”8″>
<Result>
<Title>Technology in 2001</Title>
<Summary> … </Summary>
<Url>http://www.pcquest.com/content/topstories/100120301.asp</Url>
<ClickUrl> … </ClickUrl>
<ModificationDate>1106294400</ModificationDate>
<MimeType>text/html</MimeType>
<Cache>
<Url> … </Url>
<Size>33788</Size>
</Cache>
</Result>
<Result>
<Title>PCQuest : Technology : Communicate Better with ADSL2</Title>
<Summary> … </Summary>
<Url>http://www.pcquest.com/content/technology/103020705.asp</Url>
<ClickUrl> … </ClickUrl>
<ModificationDate>1106294400</ModificationDate>
<MimeType>text/html</MimeType>
<Cache>
<Url> … </Url>
<Size>32446</Size>
</Cache>
</Result>
</ResultSet>

<!– ws01.search.scd.yahoo.com uncompressed/chunked Sun Mar 13 13:30:35 PST 2005 –>

(Note: In the above results the actual values of Summary, ClickUrl and Url have been replaced by ellipsis (… ) due to space constraints.)

Table
1: URLs of various services
Service Request
URL
Web http://api.search.yahoo.com/WebSearchService/V1/webSearch 
Image http://api.search.yahoo.com/ImageSearchService/V1/imageSearch 
Local http://api.local.yahoo.com/LocalSearchService/V1/localSearch 
News http://api.search.yahoo.com/NewsSearchService/V1/newsSearch 
Video http://api.search.yahoo.com/VideoSearchService/V1/videoSearch 

Let’s process these results to come out with a neat list of results for use on our website. We’ll use ASP.NET for this example, but you can do the same just as easily in any other programming language, especially if it has libraries/functions that allow for easy processing of
XML.

<%@ Page Language=”C#” %>
<%@ import Namespace=”System.Xml” %>
<script runat=”server”>

// function to process XML
void ProcessXML(XmlNode objNode) {
XmlNodeType objDocumentNodeType = objNode.NodeType;
switch(objNode.NodeType) {

Yahoo has a separate site for developers’ , where it provides sample code for accessing its web services

case XmlNodeType.Text:
switch(objNode.Name) {
case ‘Title’:
output.Text += “<li>” + objNode.Value.Trim() + ” – “;
break;
case ‘Url’:
output.Text += “<a href=\”” + objNode.Value.Trim() +”\”</a></li>”;
break;
default:
break;
}
break;

default:
break;
}
// get child nodes of current element, if any
XmlNodeList objChildren = objNode.ChildNodes;

// loop through the child node(s), if present
for(int i = 0; i < objChildren.Count; i++) {

// call the function to process each child node recursively
ProcessXML(objChildren[i]);
}
}
void Page_Load() {
// location of XML file – the request URL goes here
string strXmlFile = " http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=technology+site:pcquest.com&results=2&start=8
";
// create an instance of XmlDocument object
XmlDocument objXml = new XmlDocument();
// load the XML file in the XmlDocument object
objXml.Load(strXmlFile);
// get the root element of our Xml file
XmlElement objDocumentRoot = objXml.DocumentElement;
//Present the search results as an unordered list
output.Text += “<ul>”;
// Process the file
ReadXmlFile(objDocumentRoot);
output.Text += “</ul>”;
}
</script>
<html>
<head>
</head>
<body>
<asp:label id=”output” runat=”server” />
</body> 
</html>

This code creates an unordered list of the form Title – URL of all results returned by the query. The same code can be adapted to search Yahoo News or any of the other channels with minor changes. 

Table
2: Parameters used by Web search
Parameter Value Description
appid string
(required)
The
application ID.
query string
(required)
The
query to search for.
type all
(default), any, or phrase
The
kind of search to submit: all returns results with all query terms. any
returns results with one or more of the query terms. phrase returns
results containing the query terms as a phrase.
results Integerdefault
10, max 50
The
number of results to return.
start Integer:
default 1
The
starting result position to return (1-based). The finishing position
(start + results – 1) cannot exceed 1000.
format all
(default), html, msword, pdf,
ppt, rss, txt, xls
Specifies
the kind of files to search for.
adult_ok no
value (default), or 1
Specifies
whether to allow results with adult content.
similar_ok no
value (default), or 1
Specifies
whether to allow multiple results with similar content
language string:
default en
The
language the results are written in.

The introduction of Yahoo Search Web Services is really a welcome step for developers and surely there are going to be many useful applications based on this in the near future.

Kunal Dua

No Comments so far

Jump into a conversation

No Comments Yet!

You can be the one to start a conversation.

Your data will be safe!Your e-mail address will not be published. Also other data will not be shared with third person.