Making information searchable has never really been the point. Instead, our goal as online publishing specialists is to make our client’s information findable! After all it isn’t really the users’ fault if they can’t find relevant results. Even if they’re not using quite the right search terms or operators, it is our job to deliver them the most pertinent information in the right order, maximising the possibility that they will find the information they need.
Search should be clairvoyant: like a magical librarian who somehow correctly guesses what it was you were looking for; offering it up within a fraction of a second, along with a wealth of additional filtering options and navigational possibilities.
Without wishing to destroy the magic, here are my seven steps to improving findability:
1. Define what relevancy means in this context
Relevancy is a difficult thing to pin down. A set of search results is more or less relevant on the basis of how well the information retrieved meets the need of the user. Arriving at a definition of relevancy therefore means doing some fairly detailed analysis of your users and content. Some BIG questions need answering.
Is recall or precision more important? How do you go about catering to the competing needs of different user groups?
Clearly, discussions need to be had and decisions made. During this process it will be worth considering a range of scenarios where you might like to boost certain results over others:
- Field weighting
e.g. results within titles are more relevant
- Recency of data
e.g. results from recent data are more relevant
- Search phrase density
e.g. results which contain the most number of uses of the search phrase are the most relevant
- Search phrase term proximity
e.g. results where multiple terms are nearer to each other are more relevant
- Records which have been bookmarked, cited or linked to
e.g. results which have already proved themselves to be useful to other users are more relevant
2. Provide lots of options, not just lots of results
Findability is not just about returning relevant results. A good search implementation will also provide lots of additional further options to the user, which they can use to hone in on exactly what they were searching for. Examples of this include providing:
- Meaningful facets through which the results can be filtered
- ‘Did you mean…?’ option – for alternative spellings
- ‘Users who searched for x also searched y’ option – for related searches
- Clustering of search results, e.g. by topic
- Sort options
- Hit-highlighting – to highlight the phrase terms in context
3. Enriching the data
Data not only has to be marked up consistently and correctly, it can often benefit from some enhancement before it goes online. In practice, this can mean additional classification processing or entity extraction through text mining. The goal is to ensure the content itself is rich enough to support the sort of advanced searching and filtering that we want to build within the site.
4. Measure relevancy
It’s worth setting up some relevancy metrics to monitor how search is performing over time. A good method is Mean Reciprocal Rank. To implement this you track click-throughs to search results, giving each click-through to a first result a score of 1, each click-through to a second result a score of 1/2, each click-through to a third result a score of 1/3, and so on. Adding all these together will give you the ability to track an overall relevancy score, with a higher score meaning that top links are performing better.
It is also a good idea to monitor searches that return zero results. A monthly list should be reviewed in case there are some sensible search queries in there which will have resulted in user frustration.
Regular reviews of search analytics are a vital part of ensuring that search is still performing well as the site and its content change over time.
5. Improving the user’s query
Normalising the user’s search phrase (and indeed the search index data) can help to improve findability. The following are all ways in which you do this:
- Converting all letters to lower or upper case
- Removing punctuation, accent marks or diacritics
- Expanding abbreviations
- Removing stopwords or “too common” words
Recall might also be improved upon in certain scenarios by converting the users query into a fuzzy query (to return results for close matches to the search terms in order of how well they match). It may also be worth expanding the user’s search to include synonyms using a thesaurus (to return results where matches have been found for the same or similar concept).
In these ways it is possible to enhance the input query before it has even been sent to the search engine.
6. Tuning the site for third party findability
Lots of users (yes, perhaps even MOST) start their search using a third party search engine. It is therefore essential that the site is Search Engine Optimised, meaning lots of quality in-bound links, the use of semantic markup, micro formats and much, much more.
It may also be worth creating an Open Search API – so that third party use of the site’s search facility is possible.
7. Finding also means re-finding
There is a very good chance that users will want to re-use the entries that satisfy their information need. Consequently, improving fundability should also mean making it as easy as possible for users to re-find what they found before. Helping users in this way can be done with features such as:
- Saved searches
- Direct exporting to citation software
So there you have it: seven steps to findability. It is a BIG topic and I’m certain to have missed out important considerations. Please do feel free to publicly rub my nose in some of them by responding below!