SharePoint intranet search: planning refiners for faceted navigation

Faceted navigation is part of every major intranet search engine. In SharePoint, this feature is called Refiners, and the navigation panels name is Refinement Panel. It is responsible for allowing the users to access the search results filtered and organised according to a pre-defined classification system.

To create refiners on a SharePoint intranet (on-premises or online), there are several steps to follow. In this blog post, Im summarising these practical steps and also provide best practices.

The Process

Search refiners are always discrete properties of the content presented as search results. First, they can be metadata in the source system, which can be used as is. For example, the author of a document, project ID, etc.
In other cases, we have to prepare the metadata, by using an auto-classification system, for example. In this case, the metadata we need in search is not available in the source system, we have to extract it from the content. The extraction of the metadata can be done by a specific dictionary, or according to pre-defined rules. For example, location info, the topic of a document, etc. Which is not available as metadata in the source system but implicit included in the content.

Once the metadata is available (either as is, or generated automatically), we have to consider if any normalisation is needed. Normalisation might be required in the following cases:

  • We have multiple source systems, and the metadata is available (or has been prepared in the previous step), but the different systems use different names for the same property. For example, author, creator, authored by, writer, created by, etc. can be used for the same thing, but with different names. The end users dont mind the propertys name in the source systems, what they want is to be able to refine the search results by the property. In this case, what we have to do is to create one property in the index and map it to each of these various instances, then use it on the user interface.
  • We want to transform the data. For example, instead of displaying ZIP numbers on the refinement panels, we need the names of the cities instead. Another typical example is transforming e-mail addresses to person names, for example displaying Agnes Molnar on the refinement panel instead of the e-mail address of [email protected]
  • The properties have to be combined. For example, if we have two separate properties for Last Name and First Name, a good practice to combine them into one refiner value in [FirstName LastName] (Agnes Molnar) or [LastName, FirstName] (Molnar, Agnes) format.

Of course, these transformations might be combined as well. In SharePoint though, the capabilities are limited. Although we have a feature called entity extraction, this has a very limited functionality. In the majority of use cases, we need additional, third party solutions.

Terminology

Content Metadata
Content Metadata is the set of properties, which the content has in the Source System (implicit or explicit). In most cases, different systems (and subsystems) have different set of properties, therefore normalising and unifying them is a critical step in every Enterprise Search implementation.

Search Metadata
Search Metadata is defined in the Search Schema, and consists of Crawled Properties and Managed Properties (see below). Search Metadata is essential for every Search Application, as it describes the results, can be used for filtering and sorting the results, refiners, and also display on the Result Set or Hover Panel.

Crawled Property
Crawled Properties are the representations of the Content Metadata in SharePoint and Office 365 Search Schema.

Managed Property
Managed Properties are the Search Properties which can be used on the User Interface, and in Search Applications. To define where to get their values from, they have to be mapped to the proper Crawled Properties.

User Experience Considerations

Once the metadata is prepared and added to the search index, we can start using it on the user interface. In SharePoint, Refinement Panel is a Web Part available out-of-the-box, which can be customised in an easy way.
The Refinement Panel We Part has various data types to display as refiners (facets): numbers, text values as well as dates. Some configuration can be done in Display Templates, which describes how the refiners have to be displayed. For example, whether to display the item counts or not (see https://searchexplained.com/refiners-in-sharepoint-2013-search/).

We can also create (or purchase) custom developed refiners for advanced scenarios. For example, when we need charts or maps to be used as refiners, but also to create refiner hierarchies (for example, based on a taxonomy).

Conclusions

When planning refiners for faceted navigation, the primary thing to keep an eye on has to be the user needs and requirements. Driven by them, we have to plan the properties accordingly, as well as the user experience. In several cases, we have to use the out-of-the-box capabilities, which are powerful although limited. In other cases, its possible to extend these capabilities with custom and/or 3rd party tools, therefore, we have more freedom during the planning phase.
Whatever your situation is, planning refiners has to be a thorough process, as it is a major part of the overall user experience.

Top 10 Search Features in SharePoint 2013

As someone who has been focusing on Enterprise Search in SharePoint for years, I can say I now know it inside out. There are things I like a lot, there are others I dont like too much. In this blog post, I decided to collect 10 new or improved features that are my top favorites and that make SharePoint 2013 Search a real enterprise solution.

1 One, Integrated Enterprise Search Core

In SharePoint 2010, there was a Search Engine (a.k.a. SharePoint Search), but we also had the opportunity to install FAST Search for SharePoint (a.k.a. FS4SP) in order to get real, enterprise level features. FS4SP had to get licensed and installed as a separate product, as a separate farm, and then we could integrate it with SP2010.

In SharePoint 2013, the whole story is much easier: the big FAST Search engine got 100% integrated into SharePoint, therefore no separate installation and maintenance is needed. As soon as you install SharePoint 2013, you get the big engine instantly.

2 Content Processing, Entity Extraction

Content Processor is a component that sits in between the Crawler and the Indexer. It is responsible for processing the crawled content. It does all sorts of clever stuff including language detection, extracting security descriptions (to determine who in your organization is allowed to see the content), parsing, linguistic processing (to understand the real meaning of the content), entity and metadata extraction, etc.

SharePoint search - content processing and entity extraction

There are two things Id like to highlight here. First is the Web Service Callout step. This option is very useful if you need to perform custom operations on the crawled items before they are processed further.

The second step to highlight is the Custom Entity Extraction. Most organisations have specific terms (a.k.a entities) that are commonly used in everyday business. Its useful to tell the search engine to look out for some of these words because they carry particular significance for that company . For example, product names or regions where the company operates. The Custom Entity Extraction process extracts words (entities) from the content and use them as metadata in the index. This metadata can be used for filtering, ordering as well as facets on the Refinement Panel. The entities are pre-defined in a dictionary which is created by the organisation. See below an screengrab which shows how custom entities can be useful on the search results page to help the user zero in on what he is looking for.

SharePoint search - entity extraction and meta data

Both Web Service Callouts and Entity Extraction work on any type of Content Source, therefore can be used to unify and standardize the metadata in the index.

3 Continuous Crawl

Besides Full and Incremental Crawl, theres a new option in SharePoint 2013 called Continuous Crawl. This is a very dynamic and agile way of crawling that uses SharePoints change log to pick up the changes and enumerate the items which have to get crawled. One of its biggest benefits is in its flexibility and agility: the new and changed items can get indexed in minutes or even seconds, therefore we get a good basis for real, always up-to-date Search Based Applications.

Second, Continuous Crawl can rut at the same time as Full Crawl, therefore it can be used to keep the index refreshed or up-to-date, even if the Full Crawl takes a long time (days or weeks).

Continuous Crawl is available on SharePoint content sources only.

4 Search Administration on Multiple Levels

Due to the complexity of Search in SharePoint 2013, search administrators have complex tasks and responsibilities. Delegating some of these tasks might become essential.

In SharePoint 2013, search administration tasks can be delegated to Site Collection administrators and even to Site administrators.

5 Troubleshooting Enhancements

As Murphys Law says, If anything can go wrong, it will. Enterprise Search is really complex, and any of its components can go wrong. The better troubleshooting tools we have, the easier to fix these issues.

In SharePoint 2013, we have enhanced logs and reports on the server-side that can be used to debug and identify the causes of issue. The enhanced Developer Dashboard can be also used for debugging, and despite its name, its not for developers only.

SharePoint search - developer dashboard troubleshooting

6 PowerShell

PowerShell is Microsofts scripting technology that has modules for SharePoint administration and automation, too. A huge improvement in SharePoint 2013 is that we have more than 150 commands for Enterprise Search management, including setup and deployment, topology management, crawling, query processing, metadata, etc.

7 UI Enhancements

One of the most important UI enhancements is the new Hover Panel, where the search results metadata and related actions can be displayed, as well as its outline and preview if the result is a document. Besides the Hover Panel, I also like how easy it is to customize the way search results are displayed: Display Templates are responsible for the display of the results, the Hover Panel and the refiners. Display Templates are simple HTML and JavaScript files, with structures that are easy to understand. Customization is easier than ever.

SharePoint search - display templates and UI enhancements

8 Result Sources

Result Sources are used to define the index to be used in our queries (is it a local SharePoint index or a remote one from a separate content source such as Lotus Notes?). They also describe the subset of results to retrieve (these were called search scopes in SharePoint 2010). Results Sources can be very useful to define verticals for our Search and ultimately help the user focus her search.

9 Query Rules

Query Rules help us to define rules that are based on the users intent when searching. For example if I search for Harrods department store there is a high likelihood that I want to know the location or see a map; view opening and closing times; or to get a link to their online store. Technically speaking, Query Rules contain conditions and actions. A condition can be based on the query itself (contains one or more specific keywords, matches terms defined in a Managed Metadata Term Set, etc.) or on the user (for example the department he or she works, job title, location, etc.). Of course, these conditions can be combined.

Actions are all about promoting the right results to the user – displaying specific Result Blocks or modifying the current query.

Some examples:

  • If the user is based in Europe (condition), display a Result Block that highlights the latest documents related to the European market (action).
  • If the query contains the keyword define (condition), display the results (definitions) from the Company Knowledge Base (action).

10 Search Query Builder

Last but not least, Id like to highlight the wizard that is used in every Search Web Part in SharePoint 2013. This is the Search Query Builder that helps not only in building the query to be used, but also in choosing the result source as well as filters, ranking models, etc. It also gives us the opportunity to test the results of the current settings before saving anything. This can speed up the configuration of search dramatically.

SharePoint search - custom query builder

Summary

As you can see, Search in SharePoint 2013 has a lot of components that can be and have to be used in order to get a real, enterprise-level Search Application. In this blog post I highlighted what I consider as the top 10, but of course, there are many more, and the beauty of Enterprise Search is always in the details!

Agnes Molnar is the founder of Search Explained, a Content Formula partner.

http://SearchExplained.com
[email protected]

We use cookies to give you the best experience on our site. By continuing to use our website, you are agreeing to our use of cookies. To find more about the cookies, please see our Privacy Policy