sphinx （coreseek）——2、区段查询实例

• 首先需要知道区段查询的定义：

索引系统需要通过主查询来获取全部的文档信息，一种简单的实现是将整个表的数据读入内存，但是这可能导致整个表被锁定并使得其他操作被阻止（例如：在MyISAM格式上的INSERT操作），同时，将浪费大量内存用于存储查询结果，诸如此类的问题吧。 为了避免出现这种情况，CoreSeek/Sphinx支持一种被称为 区段查询的技术. 首先，CoreSeek/Sphinx从数据库中取出文档ID的最小值和最大值，将由最大值和最小值定义自然数区间分成若干份，一次获取数据，建立索引。现举例如下：

例 3.1. 范围查询用法举例

# in sphinx.conf

sql_query_range    = SELECT MIN(id),MAX(id) FROM documents
sql_range_step =
sql_query = SELECT * FROM documents WHERE id>=$start AND id<=$end

如果这个表（documents）中，字段ID的最小值和最大值分别是1 和2345，则sql_query将执行3次：

1. 将 $start 替换为1，并且将 $end 替换为 1000;
2. 将 $start 替换为1001，并且将 $end 替换为 2000;
3. 将 $start 替换为2001，并且将 $end 替换为 2345.

显然，这对于只有2000行的表，分区查询与整个读入没有太大区别，但是当表的规模扩大到千万级（特别是对于MyISAM格式的表），分区区段查询将提供一些帮助。

• 以上是coreseek 文档上的定义，分享一下本机测试实例：

之前做一套域名MX解析系统的时候获取过几百万的域名www title 信息，下面就用检索www 网站titile 数据来测试。

编辑用于测试的 coreseek 配置文件 csft.range.conf

source src
{
        type                    = mysql
        # some straightforward parameters for SQL source types
        sql_host                = localhost
        sql_user                = root
        sql_pass                = xxxxxxxxxxxxx
        sql_db                  = whomx
        sql_port                =   # optional, default is 3306
        sql_query_pre           = SET NAMES utf8
        sql_query_pre           = SET SESSION query_cache_type=OFF
　　　　 sql_query               = \
        SELECT i.id,title \
            FROM  mx_domain_wwwinfo i \
            WHERE id>=$start AND id<=$end    
         sql_query_range        = SELECT MIN(id),MAX(id) FROM mx_domain_wwwinfo
}

index 配置只需要配置中文字符编码   还有中文词库的位置就可

indexer searchd 不需要更改。

接下来测试一下，

• 生成索引：

root@timeless-HP-Pavilion-g4-Notebook-PC:/usr/local/coreseek/etc# /usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/csft.range.conf   --all --rotate
Coreseek Fulltext 4.1 [ Sphinx 2.0.-dev (r2922)]
Copyright (c) -,
Beijing Choice Software Technologies Inc (http://www.coreseek.com)

 using config file '/usr/local/coreseek/etc/csft.range.conf'...
WARNING: failed to open pid_file '/usr/local/coreseek/var/log/searchd.pid'.
indexing index 'src'...
WARNING: Attribute count is : switching to none docinfo
collected  docs, 92.2 MB
sorted 17.5 Mhits, 100.0% done
total  docs,  bytes
total 34.680 sec,  bytes/sec, 43275.54 docs/sec
total  reads, 0.023 sec, 3631.5 kb/call avg, 1.4 msec/call avg
total  writes, 0.105 sec, 912.1 kb/call avg, 0.7 msec/call avg

• 根据某个关键词测试：

• root@timeless-HP-Pavilion-g4-Notebook-PC:/usr/local/coreseek/bin# ./search -c /usr/local/coreseek/etc/csft.range.conf  济南
  Coreseek Fulltext 4.1 [ Sphinx 2.0.-dev (r2922)]
  Copyright (c) -,
  Beijing Choice Software Technologies Inc (http://www.coreseek.com)
  
   using config file '/usr/local/coreseek/etc/csft.range.conf'...
  index 'src': query '济南 ': returned  matches of  total in 0.005 sec
  
  displaying matches:
  . document=, weight=
      id=
      domain_id=
      title=?????????????,??????,??????,?????????????????????????????????????,??????,??????,????,????,??????,??????,??????,??????,????????,????,??????,??????,??????,??????,????????,????????,?????????,?????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=??--??????,?????,????????,????????,??????,??????,?????,?????,???,?????,?????,?????,?????,????,????,??????,??????,??????,?????,?????,?????,???,???????,
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=???????-???.??.???.???????/?????/?????/?????????/?????/?????/???? ????? ????? ????? ????? ????? ????? ??POS??? ????? ????? ????? ??POS?
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=???????-???.??.???.???????/?????/?????/?????????/?????/?????/???? ????? ????? ????? ????? ????? ????? ??POS??? ????? ????? ????? ??POS?
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=?????? - ???? | ????? | ????? | ?????? | ?????? | ?????? | ?????? | ?????? | ?????? | ?????? | ?????? | ?????? | ?????? | ??????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????????|????????|?????????|?????????|??????|?????|????????????????|????????|?????????|?????????|??????|?????|????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=???????????????????????????????????????????????????????????????????????????????????????????????????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????|??????|????????|??????|??????|??????|??????|??????|??????|??????|??????-????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????????????????????????????????????????????????????????????????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=??????|??????|??SKF??|??NSK??|??FAG??|??NTN??|??KOYO??|??TIMKEN??|??FAG??|????|????????|????????|??????|-??
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=???????|??????????|???????|??????????|??????|???????|?????????|????????|?????????|???????|??????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=???? ?????? ?????? ?????? ????? ?????? ???? ???? ?????? ???? ??????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=??????|????|?????|?????????|???????????|??????|??????|????|?????|?????|?????|?????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=??????????????????????????????????????????????????????????????????????????????_??????????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=?????????????????????????????????????????????????????????????????????????_??????????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????????????????????????????????????????????????????????????????????????????_??????????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=??????|??????|??????|??????|??????|?????|??????|????????|???????????|????????????|???--
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????????? ???????????? ??????? ?????????? ??????? ??????? ????????? ??????? ??????? ???????_?????????
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????-|??????|??????????????|????????|???????|??????|????T1|????T3|????T6|????U8
      addtime=
  . document=, weight=
      id=
      domain_id=
      title=????|?????|?????|?????|?????|?????|?????|?????|????|??????????
      addtime=
  
  words:
  . '济南':  documents,  hits

  可以看到结果 ：1. '济南': 11040 documents, 22214 hits  以上显示只是 编码问题。

　　    接下来还有个问题  比如 现在 要增量索引跟区段查询综合在一起怎么办？  下面文章根据百度文库里找到的一篇关于

《千万级Discuz!数据全文检索方案（Sphinx）》 综合使用coreseek 实现检索。