As researchers we need to keep up to date with the best tools for desk research. Today desk research mainly means internet searching. And internet searching mainly means Google. Or does it? Google certainly is a favoured search tool and we all use it. However, there are new search tools that are available that may be more appropriate for the type of questions we are asking, the information we are looking for or the markets that we are working in. It is our job to stay in close touch with these. We reproduce for you an article by Paul Taylor that was published in the Financial Times last week and which offers us an excellent analysis of the latest tools.

New tools to vie with Google

By Paul Taylor

Financial Times: March 29 2007

Like many web users, I rely heavily on Google’s keyword-based search tools. The Google toolbar is installed in my browser, a customised Google web page is set as my default home page and I often begin research projects with a Google search.

Google has become embedded in users’ web behaviour, but in some areas, such as desktop search, the company has arguably been outflanked. Its desktop search tool, for instance, is good, but I pre­fer Yahoo’s (desktop.yahoo.com).

Also, in my own ad hoc tests, another search technology company, Ask.com (www.ask.com), often provides better, more relevant results to basic queries framed as questions.

Ask.com uses a proprietary subject-specific popularity ranking technology, called Ex­pertRank. It goes beyond ranking pages based on the volume of links pointing to them and determines popularity among clusters of pages based on the views of “experts” on a search topic. This enables Ask.com to deliver consistently relevant results. Ask.com is also my favourite for searching blogs and RSS news feeds.

I have also been taking a closer look at search technology start-ups, some of whose products are still in “beta” or test mode, and others that are still in development.

Many of these use natural language or semantic search techniques in an effort to deliver more meaningful results to ordinary questions such as “Who was the first man on the moon?” or “What is crackberry?”

Others are trying a new direction by focusing on specialist vertical markets, or combining search technology with other technologies.

Most established search engines rely heavily on indexing web pages based on keywords. The results are variable. Keyword-based search tools in particular generally do not cope well with ordinary questions – which is where the start-ups believe they have an edge.

Among those pushing the limits of natural language processing is Powerset (www.powerset.com). This Silicon Valley-based start-up aims to use the structure and nuances of natural language to develop a search engine that “breaks the confines of keyword search”.

Powerset’s search engine has yet to launch, so we’ll have to wait and see, but some start-ups already have trial services or are about to launch them. They include AskMeNow (www.askmenow.com), Hakia (www.hakia.com) and TextDigger’s Digger (www.digger.com).

AskMeNow already has a US-based mobile phone search service that uses natural language semantic search technology. Mobile phone subscribers text a question to “ASKME” and wait a few seconds for the answer. Alternatively, users on some networks can download a small software package to most mobile phones and pay a monthly $5 (£2.50) fee for unlimited access.

AskMeNow is poised to launch a web-based service using the same technology, which will mean users can ask natural language questions and get accurate and relevant results.

Hakia, fund­ed by a consortium of European venture capital companies, is running a trial service and is due to launch commercially this year – one of the first of many expected “semantic web” companies.

Hakia claims that it “understands” questions and performs “meaning-based ­searches”. I found it produced good and often extremely comprehensive results to relatively complex questions, although not always better than Google. It provides search examples, but to get a feel for it you need to use it for a while.

It may not replace Google, but Hakia could appeal to those re­searching knowledge intensive subjects such as medicine, law, finance, science and literature.

TextDigger takes a different approach but also delivers impressive results. Currently, Digger is a private beta service so you need a log-on and password to test it, but you can request an invitation to use the service via the website.

Like other semantic-based tools, TextDigger lets users ask ordinary questions, which it interprets to produce results based on its understanding. It tells you how it is treating the words and allows you to tweak the results and correct misunderstandings. I found this generally yielded the right ans­wer to basic questions, though it slows down the search.

Some search services are focusing on market segments. For example, if you are looking for a particular video clip, Blinkx (www.blinkx.com) delivers unrivalled results. Blinkx has already indexed more than 7m hours of audio, video, viral and television content.

Another start-up, Retrivo (www.retrivo.com), aims to make it easier for users to research, install, use and fix consumer electronics products.

Kosmix (www.kosmix.com) helps con­duct deep searches in categories such as health, travel, finance and US politics.

Like next generation semantic search services, these specialised search engine tools prove that, while Google may be the king of search, there is still room for innovative start-ups and niche search companies.

Short cut to the cheapest deals

Anyone looking for the best price for a laptop computer, camera or book can use a slew of specialist comparison shopping tools, or “shopping bots”.

These specialised search services have an estimated 60m users a month and include Pricegrabber, Shopzilla and Shopping.com. Most provide comparison guides for different geographic markets including the US, UK and other European countries.

Generally, shopping bots help users search quickly for goods across a wide variety of sites and can be useful for researching products and finding the best deal. However, most services also include “pay for placement” retailers at the top of the results lists, so you usually need to reorder results by price.

Most also include user ratings and shop ratings that help users to avoid dodgy stores.

Shopzilla, formerly Bizrate, was one of the first to include user ratings and feedback. It also has services tailored to the US, UK, French, German and Australian markets and draws results from a large number of stores.

Shopping.com (previously Dealtime) also offers international sites and includes extensive product details, making it a good research tool as well as a buying guide. But typically the first results are from “featured merchants” so, again, you should re-sort by price.

My own favourite is Pricegrabber (www.pricegrabber.co.uk). Pricegrabber’s innovations include bottom-line price calculations that take account of tax and shipping, making direct comparisons between stores much easier. Pricegrabber also supplies lots of product information and user feedback on merchants. Once again, however, you need to reorder results by price.

Worth watching

Two of the most promising services up and running are:

?Hakia – “Understands” questions and performs “meaning-based searches”. Especially suitable for researching knowledge- intensive subjects, such as medicine or law.

?TextDigger – Interprets questions, yielding results based on its understanding of the words. Tells users how it is making the interpretation and allows corrections to misunderstandings.

For more information on this topic, visit our post from last year entitled “Sniffing out the right information“

from b2bsee * B2B Blog