Did you ever wish you had more control over how your content is indexed and presented as search results? In SharePoint 2013, you have the option to add a callout from the content processing component to your own web service so you can manipulate the content before it gets indexed. This makes it possible to create new ways to pivot over the search results. Here are some examples of what you could do:

  • Normalize the case for refinable managed properties to make the refiners look cleaner.

  • Create new refiners by extracting data from unstructured text.

  • Calculate new refiners based on managed property values.

This feature is similar to the pipeline extensibility stage in FAST Search for SharePoint 2010, but there are some differences:

  • In FAST Search for SharePoint 2010, the code was executed in a sandbox, whereas SharePoint 2013 lets you put the code in a web service. This reduces the overhead and lets you run the code anywhere.
  • The web service processes and returns managed properties, not crawled properties. Managed properties correspond to what actually gets indexed and are therefore easier to manage.
  • You can define a trigger to limit the set of items that are processed by the web service. This will optimize the overall performance if you only want to process a subset of the content.

In this blog post, we’ll go through a basic scenario where we have a list of popular movies. The metadata is not consistent with regards to casing, and there is limited metadata to use for pivoting over the search results.

We will walk you through how you can go from this search experience …

… to the one below, without modifying the original content. Notice that the list Title and the Director refiner have been normalized to title case, and that there is now a new refiner called “YearsSinceRelease”.

To understand what’s possible to achieve with the content enrichment web service, it is important to have a good conceptual understanding of what goes on during content processing, and where the web service callout takes place in relation to everything else.

This blog post goes into detail on what happens during content processing and how the web service callout fits into the overall picture. We’ll then show you how to create a sample list and create a web service that manipulates the data in the list to create new search refiners.

What happens during content processing

The content processing component receives crawled Properties from the crawler component and outputs managed properties to the index component, but what goes on in-between?

Inside the content processing component, there are “flows” that process one crawled item at a time. When an item has been indexed, a callback is sent back to the crawler to acknowledge whether the item is searchable or not. The success or failure of an item can then be inspected by the administrator in the Search Administration (navigate to Crawl Log and then to Error Breakdown).

The flow is the specification for how the crawled item should be processed to prepare it for indexing. The flow has branches that handle different operations, like inserts, deletes, and partial updates. The branches that handle deletions and partial updates do not have a web service callout.

The main branch of the flow handles insertion of new and updated documents and contains different stages that extract information from the crawled item and create managed properties.

At the start of the flow, new crawled properties are registered in the Search administration database. This is followed by a stage that parses binary document formats such as Office or PDF documents. During document parsing, there is a callout to IFilters for certain document types. Document parsing adds new crawled properties from the parsers. A crawled property cannot be mapped to a managed property until it has been registered in the admin database. After all crawled properties have been collected, another stage maps crawled properties to managed properties according to the Search schema. All stages after this one work on managed properties only.

The next stage processes security descriptors and converts them to the internal format used by the index component. Automatic Language detection takes place before the web service callout, and sets the value of the languages managed property. After the web service callout, there is processing related to people search that creates phonetic name variations.

Next, word breaking is done on all managed properties that are marked for word breaking in the Search schema. This is followed by entity extraction and other stages, like metadata extraction and document summarization. Links that are discovered within the document are written to the Analytics reporting database for later processing. Finally, the managed properties are indexed.

To summarize, it is important to note that the web service callout can only read managed properties. Any crawled property value that the web service needs as input must first be mapped to a managed property. The web service callout can only Access managed properties that exist before the web service callout, and not managed properties that are set further down in the flow. The web service callout can pass managed properties back to the flow, but only if they are a part of the Search schema.

The “Popular Movie” web service

Let’s go through a full scenario that covers everything from creating the content and the web service, to customizing the search experience with new refiners. The first thing we’ll do is to create a list of some all-time popular movies with information about the release year, director, and whether the film received an Oscar. We’ll then create a web service that does a couple of things to improve the search experience:

  • Calculates the years since the movie was released by looking at the ReleaseYear managed property

  • Normalizes the Director managed property to title case

  • Creates a time stamp for when the web service processed the item

These are the steps that we’ll walk you through:

  • Create a sample list

  • Create the web service

  • Configure crawled and managed properties

  • Crawl and search the content

Create a sample list

Open the SharePoint 2013 Management Shell as administrator. Make sure that the user you are logged on as is a member of the WSS_ADMIN_WG
group, because we need write access to SharePoint resources.

Type the following commands to create a new list:

$SPWeb = Get-SPWeb http://myserver 

$CustomListTemplate = $SPWeb.ListTemplates | where {$_.Name -eq "Custom List"}

$listUrl = "PopularMovies"

$description = "A list of popular movies"

$lists = $SPWeb.Lists

$lists.Add($listUrl, $description, $CustomListTemplate)

You now can browse to your list by going to http://myserver/lists/PopularMovies

$list = $lists | where {$_.Title -eq "PopularMovies"}  

Below are the commands to create the different fields (columns) in the list.

$spFieldType = [Microsoft.SharePoint.SPFieldType]::Text $list.Fields.Add("Director", $spFieldType, $false) $spFieldType = [Microsoft.SharePoint.SPFieldType]::Integer $list.Fields.Add("ReleaseYear", $spFieldType, $false) $spFieldType = [Microsoft.SharePoint.SPFieldType]::Boolean $list.Fields.Add("WonOscar", $spFieldType, $false) $spFieldType = [Microsoft.SharePoint.SPFieldType]::DateTime $list.Fields.Add("ReleaseDate",$spFieldType, $false) $list.Update()  

If you want the newly created fields to be part of the default list view, you can add them to the default view. This is an example of how we did it for the Director field.

$spView = $spWeb.GetViewFromUrl("/Lists/PopularMovies/Allitems.aspx”) $spfield = $list.Fields["Director"] 

$spview.ViewFields.Add($spfield)

$spview.Update()

Now we are ready to start populating the list. Each list item represents a movie, and we start off with “Pulp Fiction”.

$spListItem1 = $list.AddItem() $spListItem1["Title"] = "Pulp Fiction" 
 
$spListItem1["ReleaseYear"]=1994 
 
$spListItem1["Director"]="Quentin Tarantino" 
 
$date = Get-Date "10/21/1994" 
$spListItem1["ReleaseDate"]=$date 
 
$spListItem1.Update()

The screen shot below shows the list after we have added a few more movies:

The Title and the Director have been added with inconsistent casing to show how we can fix this automatically in the web service.

Create the web service

Now let’s create a web service that can read the list we created, and create some new managed properties that we can use as refiners.

using System;  using System.Collections.Generic;  using System.IO;  using Microsoft.Office.Server.Search.ContentProcessingEnrichment;  using Microsoft.Office.Server.Search.ContentProcessingEnrichment.PropertyTypes;  using System.Globalization;  using System.Threading;   namespace PopularMovieService {  public class PopularMovieService : IContentProcessingEnrichmentService {  // Define variables to hold the managed properties that the  // web service will populate.  private Property<Int64> NewIntegerMP = new Property<Int64>();  private Property<DateTime> NewDateTimeMP = new Property<DateTime>();  private readonly ProcessedItem processedItemHolder = 
new ProcessedItem { ItemProperties = new List<AbstractProperty>() };  public ProcessedItem ProcessItem(Item item) {  // Iterate over all managed properties passed to the web service.  foreach (var property in item.ItemProperties) {  var s = property as Property; if (s != null) {  // The value of the new text managed property is the  // string in title case.  CultureInfo cultureInfo = Thread.CurrentThread.CurrentCulture;  TextInfo textInfo = cultureInfo.TextInfo;  string normalizedString = textInfo.ToTitleCase(s.Value.ToLower());  s.Value = normalizedString;  processedItemHolder.ItemProperties.Add(s);  }  var l = property as Property; if (l != null) {  // The value of the new integer managed property the  // number of years since the release date.  int CurrentYear = DateTime.Now.Year; NewIntegerMP.Name = "YearsSinceRelease";  NewIntegerMP.Value = CurrentYear - l.Value;  processedItemHolder.ItemProperties.Add(NewIntegerMP); }  // Set the time for when the properties where added by the  // web service. NewDateTimeMP.Name = "ModifiedByWebService";  NewDateTimeMP.Value = DateTime.Now;  processedItemHolder.ItemProperties.Add(NewDateTimeMP); }  return processedItemHolder; }  }} 

Configure crawled and managed properties

When you create a list in SharePoint, the column names are picked up by the crawler as crawled properties. The name of the crawled property is the same as the list column name, but with “ows_” in front, so “ReleaseYear” becomes “ows_ReleaseYear”. There are several ways a new crawled property can be registered as part of the Search schema: by crawling or by adding it programmatically, for example, through a Windows PowerShell cmdlet. In this blog post, we stick to Windows PowerShell whenever possible.

$cp = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $ssa -Category SharePoint -Name "ows_ReleaseYear" -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346" –VariantType 0

$mp = New-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $ssa -Name "ReleaseYear" -Type 2 -Queryable $True

$mp.Refinable = $True

$mp.Update()

New-SPEnterpriseSearchMetadataMapping -SearchApplication $ssa -ManagedProperty $mp -CrawledProperty $cp

We need to do the same for the other managed properties (Director, WonOscar, and ReleaseDate). For easy reference, these are the types to use when creating the managed property: Text = 1, Integer = 2, DateTime = 4, and Boolean (YesNo) = 5.

We also need to create the new managed properties that the web service populates:

$mp = New-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $ssa –Name "ModifiedByWebService" –Type 4 –Queryable $True

$mp.Refinable = $True

$mp.Update()

$mp = New-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $ssa –Name "YearsSinceRelease" –Type 2 –Queryable $True

$mp.Refinable = $True

$mp.Update

Crawl and search the content

Before starting the crawl, we must first enable and configure the web service callout. This is done in Windows PowerShell.

$config = New-SPEnterpriseSearchContentEnrichmentConfiguration

$config.Endpoint = "http://localhost:817/PopularMovieService.svc"

$config.InputProperties = "Director", "Title", "ReleaseYear"

$config.OutputProperties = "Director", "Title", "YearsSinceRelease", "ModifiedByWebService"

Set-SPEnterpriseSearchContentEnrichmentConfiguration -SearchApplication $ssa -ContentEnrichmentConfiguration $config

We are now ready to kick off a full crawl of the list we created, and observe what gets sent to the web service.

If you don’t already have an Enterprise Search Center, you can create one using the site collection template called “Enterprise Search Center” under the Enterprise tab.

To make it easier to see the effect of the web service, we’ve customized the search center to show refiners for the managed Properties that the web service modifies.

To show the new refiners, we will edit the result template in the search site collection. First we do a query to see the result page with refiners. If we search for “Pulp Fiction”, this will show a refiner for Modified Date, which is one of the out-of-the box refiners. Click the tools icon at the upper-right, and then click Edit Page.

Click Edit webpart for the Refinement Web Part. This brings up a configuration panel, which lets you choose refiners for the Web Part. Add all the new managed properties as refiners.

We now need to check in and publish our changes.

If we now try out a query for “ReleaseYear>0”, we get back all three results with refiners, just like the screen shot at the beginning of the blog post.

Looking for more information?

Check out the MSDN documentation for content enrichment . There is also a How to article that gives you an example of how to create a content enrichment web service in Visual Studio.

Finally, look out for our upcoming blog posts on that will provide best practices for debugging the content enrichment service and show you how to use WCF Routing to do content-based routing and load balancing

原文地址:

http://blogs.msdn.com/b/sharepointdev/archive/2012/11/13/customize-the-sharepoint-2013-search-experience-with-a-content-enrichment-web-service.aspx

Customize the SharePoint 2013 search experience with a Content Enrichment web service的更多相关文章

  1. SharePoint 2013 Search 配置总结

    前言:SharePoint 2013集成了Fast搜索以后,搜索的配置有了些许改变,自己在配置过程中,也记录下了一些入门的东西,希望能够对大家有所帮助. 1.配置搜索服务,首先需要启用搜索功能,基本思 ...

  2. SharePoint 2013 Search REST API 使用示例

    前言:在SharePoint2013中,提供Search REST service搜索服务,你可以在自己的客户端搜索方法或者移动应用程序中使用,该服务支持REST web request.你可以使用K ...

  3. 配置SharePoint 2013 Search 拓扑结构

    在单台服务器上安装了 SharePoint Server 2013,并且创建了具有默认搜索拓扑的 Search Service 应用程序.在默认搜索拓扑中,所有搜索组件都位于承载管理中心的服务器上.S ...

  4. 修复SharePoint 2013 Search 拓扑错误

    Problem 当创建和配置SharePoint 2013 Search Service Application成功之后,进入详细配置页后,在Search Application Topology节点 ...

  5. 探索 SharePoint 2013 Search功能

    转载: http://www.cnblogs.com/OceanEyes/p/SharePont-2013-Search-Develop.html SharePoint 2013的搜索功能很强大,特别 ...

  6. SharePoint 2013: Search Architecture in SPC202

    http://social.technet.microsoft.com/wiki/contents/articles/15989.sharepoint-2013-search-architecture ...

  7. 转载 SharePoint 2013 Search功能

    转载原出处: http://www.cnblogs.com/OceanEyes/p/SharePont-2013-Search-Develop.html 阅读目录 启用Search Service A ...

  8. sharepoint 2013 search configuration

    在建立search application之前完成以下命令 $hostA = Get-SPEnterpriseSearchServiceInstance -Identity "WTCSPS0 ...

  9. SharePoint 2013 开发——搜索架构及扩展

    博客地址:http://blog.csdn.net/FoxDave SharePoint 2013高度整合了搜索引擎,在一个场中只有一个搜索服务应用程序(SSA).它集成了FAST,只有一个代码库 ...

随机推荐

  1. java WeakReference

    在Java 1.2中就引入了java.lang.ref这个包,WeakReference就属于这个包.WeakReference是干嘛的呢,一言弊之,它是和Java中的垃圾回收相关的.如果一个对象只有 ...

  2. 17.1.1.5 Creating a Data Snapshot Using mysqldump

    一种方式创建一个数据库的快照在一个存在的master 数据库是使用mysqldump 来创建你需要复制的所有数据库的dump. 一旦数据dump 是完成,你然后倒入数据到slave 在开始复制过程前 ...

  3. 【HDOJ】2546 饭卡

    01背包,需要先对数据升序排序.这样保证优先购买最贵的东西,才满足背包条件. #include <stdio.h> #include <string.h> #include & ...

  4. (转载)php循环检测目录是否存在并创建(循环创建目录)

    (转载)http://www.jb51.net/article/25917.htm php循环检测目录是否存在并创建,需要的朋友可以参考下. 循环创建目录方法 这个会生成image.gif目录 代码如 ...

  5. Bash字符串的处理

    # 定义几个变量用于测试目的 FILE_NAME="/usr/local/app/world.txt" TIME_NOW="`date +"%Y%m%d%H%M ...

  6. time.h

    目录 1简介 2代码示例 3从系统时钟获取时间方式 4time函数介绍 函数名称: localtime 函数名称: asctime 函数名称: ctime 函数名称: difftime 函数名称: g ...

  7. .Net 调式案例—实验4 高CPU(High CPU)回顾

    原文地址:http://blog.csdn.net/directionofear/article/details/8033506 如果Web应用程序经常遇到的问题按频率排名的话,我觉得 第一名unha ...

  8. POJ 3186Treats for the Cows (区间DP)

    详见代码 #include <stdio.h> #include <algorithm> #include <string.h> using namespace s ...

  9. JavaScript运行机制浅析

    从一个简单的问题谈起: <script type="text/javascript"> alert(i); var i = 1; </script> 输出结 ...

  10. Shell until循环

    until 循环执行一系列命令直至条件为 true 时停止.until 循环与 while 循环在处理方式上刚好相反.一般while循环优于until循环,但在某些时候,也只是极少数情况下,until ...