Improving Search Relevancy in the Age of the Customer

Technology is dramatically changing consumers' attitudes toward, and their expectations for, speed and personalization.

Empowered customers expect to get exactly what they want, the second they want it, forcing the companies serving them to evolve their business processes. This is especially true in online retail, where retailers have mere fractions of a second to engage customers - or risk losing them to competitors, who are just a mouse click away.

As one of the first interaction points, site search is critical to a retailer's success. Basic keyword match-based search is not enough anymore. In fact, search based only on keyword matching can present major challenges as organizations strive to deliver the fastest, most convenient, frictionless customer experiences.

Here, we'll explore why keyword-based search is insufficient, and several new capabilities that can enhance relevancy, satisfying customers' growing service demands and ultimately enhancing the conversion-driving power of commerce-oriented sites.

The Limitations of Keyword-Based Search

Many ecommerce platforms and site search solutions leverage open source, keyword-based search technologies, including ElasticSearch or SOLR. These solutions make it much easier for developers to build faceted search and navigation shopping experiences. They also significantly improve the query performance of previous approaches that were built on top of legacy database technology. The search functionality works by identifying products on a site containing search terms in the shopper's query, and favoring those items with the most "interesting" search term matches.

The challenge with this approach is that it is often at odds with the inherent nature of product search - finding the best items among many similar items that are relevant to the shopper's search. Popular open source search technology today is built on top of an algorithm called TF-IDF, which determines keyword matches that are most "interesting." This algorithm was originally designed to search large collections of unique documents like blog posts or an encyclopedia, where there's usually only one best matching item for any given search query, not several. That algorithm shines when looking for something like "Thailand's capital" where there's really only one possible best matching answer - in ecommerce, this is almost the opposite problem when searching for products.

In the age of the customer, the goal of search is to quickly hone in on items that best meet shopper's wants and needs - which often varies greatly between different product categories. For example, apparel shoppers want the newest and in-season clothing in their size, whereas electronics shoppers what the best-selling items with good reviews at a great price. Modern search engines must be capable of this deeper understanding, or risk confusing the customer and losing the sale.

Product Awareness

Product awareness is a technique that can be used in concert with keyword search to more precisely narrow down what users are looking for. Product awareness leverages machine learning to group similar products together while also making it easier for shoppers to find those items they're specifically seeking.

For example, a site visitor may search for "red nike running sneakers." Basic keyword search on its own will uncover results that include some or all of these terms, although the results may not necessarily be sneakers. They may include a red Nike T-shirt, for example, or a Nike running windbreaker. Product awareness intelligently dissects both the product information in the catalog and search terms, extracting the brand, attributes and meaningful details to figure out what is the most important part of the query - which in this case would be sneakers. Any item that may contain the terms red, Nike and/or running - though is not a pair of sneakers - will be eliminated from the search results, or ranked significantly lower.

"Wisdom of the Crowds" Behavioral Understanding

Once product awareness helps determine which search results are relevant, behavioral analysis goes a step further and ranks search results based on how shoppers using the same search term have engaged with the results in the past. Products that typically generate a higher level of conversion can then be ranked higher, increasing their exposure.

Behavioral understanding accomplishes several valuable goals. First, it gets "inside the head" of collective customers and prioritizes search results based on a knowledge of which results are the most successful in driving conversion. Second, it helps sites promote greater profitability, by favoring results that customers are more likely to act upon. Finally, automating search results based on certain key criteria - such as product views, click-throughs and add-to-cart for example - can enable a greater degree of marketing and merchandising consistency across various digital touchpoints (websites, mobile sites and apps). Certain automation rules can be applied, such as eliminating any out-of-stock items from search results, which can bring greater efficiency and ease to online merchandising tasks.

Natural Language Processing (NLP)

Natural Language Processing, or NLP, allows site visitors to search for products in their own language, thus making for an easier, more convenient interaction. In most cases, it works by breaking down and intelligently interpreting conversational lexicon. For example, NLP technologies can break down the phrase, "show me sneakers for long-distance running" and intuitively understand that the customer is searching for running shoes.

The challenge with this approach is that this is not typically how users search, especially customers looking to buy something. Most shoppers want to get in and out of an online transaction as quickly as possible, using short search terms (one or two words) as opposed to the types of longer phrases or sentences that they may use with Siri or Google Assistant.

The more common NLP challenge is recognizing search term variances for the same product. In the previous examples, U.S. customers may prefer the term "running shoes" while their U.K. counterparts will prefer "running sneakers." Also, customers searching for a 12-inch frying pan may use the terms '12 inch frying pan,' '12 in. frying pan' or '12" frying pan,' all of which mean the same thing. Advanced NLP techniques can now accommodate these differences, directing users to the product(s) they seek while accommodating various types of linguistic variances, as well as improving spelling correction and synonym detection.

Personalization and Individualization

Finally, the last step in creating a relevant customer experience is to tailor results based on individual tastes and attributes such as size, color, age, gender, location, brand affinity and style. The goal here is to implicitly understand a shopper's preferences from previous site behavior, in-store purchase history and other third-party data sources to create a one-to-one individualized experience - just as a shopper would experience in a physical store. Shoppers should also be encouraged to provide explicit feedback on their tastes and individual circumstances (such as where they live), to increase the depth of the shopper's preferences profile.

Driving Business from the Search Box

As the customer experience takes center stage, ecommerce sites must deliver a high degree of relevancy across search and navigation, delivering exactly the products customers are looking for quickly. Sites must also present those products that have the highest probability of conversion and revenue-generating potential, in order to drive their businesses.

Keyword-based search capabilities are not enough and must be augmented with greater intelligence, to improve findability, entice the highest degree of customer interaction, recognize natural language-based nuances and hyper-personalize search results. This will help guide customers to exactly what they want and the most popular options in their own terms, and based on their own individual preferences. Ultimately, sites will grow more successful in meeting customers' ever-growing service expectations, and increasing conversions and revenues.