Monthly Archives: October 2012

Intelligent Search: What Google’s New “Semantic Search” Means for Search on your e-Commerce Site

Google has recently announced that it is adding more “semantic search” techniques into its otherwise largely keyword search. This means matching on the meaning of words, rather than just the occurrence of words. Since nearly all of your customers also use Google, their expectations for search are conditioned by Google. Over time, there is a trickle-down in the expectation that shoppers have of search, based largely on their experience on Google.

Therefore, it’s a reasonable question to ask: “What changes should I make in search at my commerce site to keep pace with customer expectations?” Beyond keeping pace with expectations, there is another even more important reason to invest in semantic search on your site — increased conversion rate. Analysis of Neilsen netRatings conversion rate studies across similar e-commerce sites has not only confirmed the impact of natural language semantic search, it has actually measured it!

What is Semantic Search?
The literal definition of semantic search is searching on meaning rather than searching on words. Google is now knocking at the door of semantic search by associating word groups as concepts. If some people search on “beach sandals” and other people search on “beach flip-flops”, while both groups click to show interest in the same item set, then the concept “sandal” and “flip-flop” may be related. The distillation of words into concepts is one part of the greater field of Natural Language Processing (NLP). Searching on concepts in their various forms delivers more complete results and is more tolerant of user search variations. As you have seen, semantic search is quite valuable – but there is more power available when you go deeper using more NLP techniques.

A semantic search with deeper NLP (let’s call this Natural Language Search, or NLS) support brings even more converting power to a commerce site. Lets look at these two commerce searches, “return policy” and “sweaters under $100”. Searching all your product descriptions for the words “return” and “policy” will clearly lead to ridiculous results. Clearly, the intent of this search is to display your policy on returns – treating this as a phrase and recognizing its special nature are important to the shopper, and easy with NLS.

Similarly, treating “under $100” as a keyword search will yield undesirable results. The intent of the user is to restrict the products based on price. Recognizing that “$100” is not a word, but rather a price requires something smarter than a keyword search. This occurs in other forms when the user wants to express a range restriction, not just on price, but any other numerical product attribute such as length, weight or wattage.

Units of measure commonly stump keyword search engines. For example, keyword searching for “12 volt 24 amp motor” will unfortunately return all motors with 12 or a 24 anywhere in the description. Thus, both 24 volt 12 amp motors as well as 24 watt .5 amp motors with a 12″ shaft will be shown! If your site gets lots of dimensional/size searches, the capabilities of NLS is absolutely critical. A semantic search with NLP is aware of units of measure, such as “volt”, “v” or “amp”, “A”. This unit of measure awareness automatically creates a phrase around “12 volt”, and to include searches on variations like “12V” or “12 V”. When a shopper searches for “Nike size 10”, NLS will recognize that “size” is an attribute with numeric values & therefore select the products with “size=10”. These capabilities impact countless unique searches that would otherwise stump almost all search engines.

These examples illustrate how easy it is for dumb keyword searches to yield embarrassing results. Have you ever searched a site only to see hundreds of irrelevant results? This not only reflects poorly on your brand, but can actually cause you to lose customers! Nearly all of us have had the experience of getting such poor results from a search on a commerce site. We get frustrated and leave the site altogether to buy from another site. This illustrates how improving search can improve conversion rate.

In order to measure the correlation between semantic search and conversion rate, we used Nielsen netRatings to compare the conversion rates of sites that were similar except for their use of semantic search. We compared sites for catalog companies and non-catalog companies separately. In both groups, the sites using semantic NLP search had about 20% higher conversion rate than the sites using keyword search. Of course, there are many other phenomena that impact conversion rate, but these would generally balance out across all the groups. Furthermore, the 20% improvement is consistent with the uplift we see when customers switch from keyword search to semantic search. Details of the Nielsen study are available on request.

Google is moving the world towards semantic search. Eventually user expectations will demand it from your commerce site as well. Switch sooner rather than later – you’re leaving money on the table every day until you make the switch!

About the Author:
Craig S. Bassin is the CEO of EasyAsk, the leading provider of natural language technology and solutions. Mr. Bassin has an extensive career in emerging technology software, previously founding B2Systems and choreographing a leveraged buyout of Sapiens Corporation. Before that, Mr. Bassin also served as Vice President & Partner at GenevaGroup International, the premier boutique retained search firm, specializing in Computer Software and Internet Technology companies. He began his career in Software in the early days of the RDBMS boom, was the top sales manager at Ingres Corporation later the first Director of National Accounts at Oracle Corporation.

Benefits of Semantic Natural Language Search for E-Commerce

How this paradigm shift will change Web and mobile e-commerce forever

Advancement in communication and technology over the last two decades has been dramatic, and the way people consume information has evolved in parallel.  Not long ago, people turned to libraries, dictionaries, reference journals, books, phone books and printed newspapers for insight, but now they simply turn to “The Web.”  Answering complex questions used to take hours or days – if we could figure out how to answer them at all.  Now we are accustom to executing Internet searches in seconds.

ACCURACY, however, is the issue.

The next step is to provide the correct response on the very first page.  To take this next step, we’ll consider some words and phrases that were once outside of mainstream vocabulary, more commonly used in academic and research circles at MIT and Stanford labs – things like Natural Language Processing (NLP) and Semantic Search (per Wikipedia: semantic search uses semantics, or the science of meaning in language, to produce highly relevant search results. In most cases, the goal is to deliver the information queried by a user rather than have a user sort through a list of loosely related keyword results.).  Search will not evolve without these important concepts because even with all the great digital information available today, it still takes too long for people to find exactly what they’re searching for – whether on the Internet, on their phone, in an e-commerce store, or in a corporate applications like CRM and Business Intelligence.

It is interesting to think about where we started with search boxes – Yahoo, Excite, Netscape, to name but a few, and most recently Google, have all taught us to search using “keywords.”  We know that search engines can’t understand the way we speak or think, so we had to adapt our behavior to make use of the services they provide.  When we hit the search button, we hope that the algorithms, machines and logic in some distant server farm send us back a bunch of links that we can comb through to find what we are looking for.  Search engines essentially provide us a starting point – lists of results – but we still have to manually navigate the final mile.  We get streams of results in seconds, but it takes considerably longer to find the right thing, or often we get frustrated and stop looking.  Google has learned from user interactions and are now developing semantic capabilities, and WolframAlpha takes it further by computing answers from a knowledge base of curated, structured data but still today ‘search results’ are simply a starting point to begin looking for answers.

Also, semantic search is a great step in the right direction, but it doesn’t have a full understanding of all possible responses. That’s where natural language processing completes the loop, understanding both the searcher’s intent and a deep understanding of the data to deliver the best possible response.  Essentially, Semantic search provides understanding of the intent, or context, of the search.  Natural Language provides knowledge both of intent AND content.

For the first time, you can have better technology than the search engine giants – who have certainly spotted this trend and are moving in the semantic direction.   Recently Google shared its Knowledge Map plans.  Jack Menzel, product management director at Google, in a very articulate video, questioned: “Wouldn’t it be amazing if Google could understand that the words that you use when you are doing a search, well they aren’t just words, they refer to real things in the world. That a building is a building, and an animal is an animal and that they are not just random strings of characters, and if we could understand that those words are talking about those real world things, than we could do a better job of getting you the content you want off the web…”

Google is obviously a large company and has the time and resources to integrate changes in stages, especially considering that their revenue model is still based on keyword advertising.  You and the e-commerce industry do not have that luxury – we need to act now to improve the Web e-commerce search experience and to accommodate the growing number of mobile e-commerce shoppers.

Given where we are today, understanding the intent of what is being searched for has become a competitive advantage – especially when deployed in e-commerce environments.   Understanding intent even helps when shoppers enter only a few keywords, because each single word carries so much value. Natural Language Processing (NLP) use techniques like relevancy, association, disambiguation and many more to understand what a shopper is actually looking for, and can deliver the most relevant options from your product catalog.

Again, semantic search can understand the searcher’s intent, but NLP understands their intent and all possible results, then processes requests and delivers the best possible results. This is an important distinction, especially for e-commerce sites, which need to present the most relevant items, even when search requests don’t match up nicely with what is in your product catalog.

Some general e-commerce industry statistics suggest that 20% of searches are now long-tail searches. A long-tail search is a more descriptive phrase that contains three or more words.  It often contains a main concept, which are one or two words in length. For example, “London Olympic t-shirt under $20,” the main concept would be Olympic and the other terms can help us identify the most relevant item with the additional details.  Now we can look at t-shirts from the 2012 Olympics in London and not t-shirts from 2008 in Beijing.  Cost is yet another filter, but again intent is important.  Keyword search will  return items with ‘Olympic, t-shirt’, ‘under’ or ‘$20’ (potentially t-shirt underwear) while the searcher intent is to find any shirts under $20.

As an e-commerce retailer, you have to address long-tail searches, otherwise you will miss out on a key source of revenue and likely degrade existing traffic.

Hopefully you are beginning to see some of the benefits semantic natural language search can provide Web-based e-commerce, but more importantly you need to consider how this will support your growth into mobile e-commerce.

Since the iPhone was launched, that small screen has become an important window into the world for most users.  Androids and others followed suit and smart phones have become a common entry point into e-commerce.  Analysts from research firm Gartner Inc. say the shift from e-commerce to m-commerce will reach something of a tipping point by 2015. According to Gartner’s analysts, mobile applications and social media will account for 50 percent of Web sales by then. Additionally, Gartner said that e-commerce merchants will start offering “context-aware, mobile-based application capabilities that can be accessed via a browser or installed as an application on a phone” at that point. “E-commerce organizations will need to scale up their operations to handle the increased visitation loads resulting from customers not having to wait until they are in front of a PC to obtain answers to questions or place orders,” said Gene Alvarez, research vice president at Gartner, in a statement.

Additionally, because of Siri, Nuance Dragon, Google Voice Search and others, speech is now an integral way we interact with these little devices. As people become more conversational with these devices, the search terms will naturally become more descriptive.  Again, with limited screen size and long-tail searches, natural language search functionality will not just be a nice feature; it will be mandatory if you want to provide the most relevant result quickly and efficiently on mobile devices.  Imagine connecting to your favorite e-commerce site, hitting the microphone on your smartphone and SPEAKING, ‘ladies blue blouses under $35’ and immediately seeing your results.  That’s taking e-commerce mobile.

Natural Language and Semantic Search are concepts you need to become familiar with in the next few months.  If you learn how to integrate them properly, you’ll be able to provide your shoppers the right information at the right time to improve conversion rates and drive revenue.  Regardless if you do or don’t, your competitors will.  So… Where do YOU think your shoppers will turn the next time they pull out their iPhone?

About the Author:

Craig S. Bassin is the CEO of EasyAsk, the leading provider of natural language technology and solutions.  Mr. Bassin has an extensive career in emerging technology software, previously founding B2Systems and choreographing a leveraged buyout of Sapiens Corporation.  Before that, Mr. Bassin also served as Vice President & Partner at GenevaGroup International, the premier boutique retained search firm, specializing in Computer Software and Internet Technology companies. He began his career in Software in the early days of the RDBMS boom, was the top sales manager at Ingres Corporation later the first Director of National Accounts at Oracle Corporation.