My Search Sucks! Part 2

Part 2 in the 4-part series, “My Search Sucks,” discussing why search, well, sucks.

There are three key principles that explain why site search just doesn’t perform like we expect it to. Over the next few weeks, I’ll dive deeper into each issue surrounding traditional search and offer my insights and experiences to help you understand why your search sucks, and how you can improve it.

Reason #1:  The critical information is not in the document.

All full-text search technologies basically work the same way: they look for a match between the words in a user’s query and the words in the text of the documents searched.  That said, there are lots of fancy layers that can be added from simple stemming to complex natural language processing (NLP), but the fundamental assumption is that the engine can figure out which documents best meet a user’s needs by looking inside the document.

While this is a start, it’s just not enough. The critical information isn’t in the document; it’s in someone’s head.  But whose head is it in? Let’s look at some examples.

A favorite example comes to mind involving one of our customers, a large online appliance retailer.  Users were coming to their website looking for a “stove” over and over again, and the search results had “stove-top safe” kettles and pots, but no stoves.  Turns out the reason for this was that this retailer’s website was using the manufacturer terminology, “cooktops” and “ranges.”  The word “stove” was nowhere to be found.  The community was using a different vocabulary than the site.

Sounds like a simple fix, right?  All you need to do is to create a synonym to tell the search engine that a “stove” is the same thing as a “range.”  And sure, once you’ve found and addressed the discrepancy, customers searching for “stoves” will find the “ranges” they’re really looking for.  But what about all of those long-tail terms and content—and what about when things change?

Sam Mefford, an expert in the deployment of enterprise search technologies, commented on last week’s blog.  In his search practice, he sees this challenge surface on a regular basis and provided an example from one of his clients.  The company re-branded one of its products, and made the appropriate changes in its marketing materials and documentation.   Afterwards, field agents and customers could no longer find the products and information they needed, because they continued to search using the old name.  This problem took months to discover.

Another great example is from a customer that’s a well-known wireless provider.  They launched a new LG phone called the “Incite.”  Suddenly, one of the most popular search queries on their site became “insight.”  The search results included lots of business-type documents about how to achieve great “insight” into your business operations, but nothing that matched what users wanted – information on this exciting new phone. Sure, searching for “insight” while the product is called “incite” was technically the user’s mistake, but does that matter when you’re losing opportunities?

Let’s say the words do exist in the document.  It’s often not enough.  There may be 1000s of documents that contain the search terms, but which documents are the best?  A traditional search engine will assume that the one with the most occurrences of the keywords is the most valuable, but this is very often not the case.  Obviously, the technology is more sophisticated than this, but the fundamental basis is along these lines.  The most useful document may only have one instance of the keyword and therefore may be buried on page 10 of the results.  So, how do you get the most useful document to the top of the search results?

Manual tuning is the traditional “solution” to all of these site search issues, but as we discussed earlier, it’s nearly impossible to catch all discrepancies and adapt rapidly—not to mention the time and effort involved.  I’ve even mentioned the spirit of the solution: it’s fundamentally a recognition that the needed information is not in the document, it’s in someone’s head.  But whose head is it really in?

Many companies have experts that manually tune and tweak search.  But that’s a labor-intensive way to temporarily solve the problem and certainly doesn’t guarantee that the expert’s view on what’s right matches with users’.  Why take that chance?  Better to go straight to the source of the information: the user!

Next week: Part 3 in the 4-part series, My Search Sucks! where we’ll explore how actions speak louder than words.