高CPU业务场景下的任务分发方案Gearman搭建一览

　　

　   Gearman是当年LiveJournal用来做图片resize的，大家也明白图片resize是一个高CPU的操作，如果让web网站去做这个高CPU的功能，有可能会拖垮你的

web应用，那本篇我们来看看gearman是如何解决这个问题的，它的架构图类似下面这样：

从上面这张图，你应该会看到，Gearman是由三个部分组成：

1. Job Server

这个就是Gearman的Job Server，通过它对Client 和 jobwork 进行桥接，是不是想起来了中介者模式。。。

2. Client

Gearman提供了Client API 给客户端调用，Client只需要将一个高CPU的业务函数名丢给Job Server，然后等待JobServer的返回执行结果。

3. jobwork

Gearman提供了work API 给work客户端进行调用。jobserver会根据后端的work集群的负载情况，分发给一个合适的work去执行，并等待结果。

说到这里，你应该就明白了，本质上它属于那种分布式的RPC调用，而且非常牛逼的地方在于Client 和 Work 可以用不同的语言实现。

一：安装部署

1.  下载地址：https://github.com/gearman/gearmand/releases

目前gearman的JobServer 有C，JAVA，Perl三种语言实现，由于C版本的JobServer是最活跃的，所以这里采用目前最新的1.1.17版本的gearmand在CentOS

上进行安装部署。

2.  快速安装

可以通过官网http://gearman.org/getting-started/中的getting-started进行快速安装。

<1> 基础依赖库安装和gearmand下载

 yum -y install boost-devel gperf libevent-devel libuuid-devel gcc44 gcc-c++
 wget https://github.com/gearman/gearmand/releases/download/1.1.17/gearmand-1.1.17.tar.gz
 cd gearmand-1.1..tar.gz
 tar xzvf gearmand-1.1..tar.gz
 cd gearmand-1.1.
 [root@localhost gearmand-1.1.]# ls
 aclocal.m4  build-aux     configure.ac  gear_config.in  libgearman-1.0     libhashkit-1.0  Makefile.am  rpm      THANKS
 AUTHORS     ChangeLog     COPYING       gearmand        libgearmancore     libhostile      Makefile.in  scripts  util
 benchmark   configmake.h  docs          HACKING         libgearman-server  libtest         man          support  version.m4
 bin         configure     examples      libgearman      libhashkit         m4              NEWS         tests

<2> 然后就是常规的./configure --prefix=/usr/myapp/gearman && make && make install  这个过程超级慢，可以出去抽跟烟，

顺便再去拉泡屎。。。

 ./configure --prefix=/usr/myapp/gearman && make && make install

<3> 若干年后，当你看到这个就算安装成功了。。。还是得恭喜一下。。。。至少没让你踩到缺少各种依赖包的界面。

 See any operating system documentation about shared libraries for
 more information, such as the ld() and ld.so() manual pages.
 ----------------------------------------------------------------------
  /usr/bin/mkdir -p '/usr/myapp/gearman/sbin'
  /usr/bin/install -c -m  man/gearman_worker_create. man/gearman_worker_define_function. man/gearman_worker_echo. man/gearman_worker_errno. man/gearman_worker_error. man/gearman_worker_free. man/gearman_worker_function_exist. man/gearman_worker_grab_job. man/gearman_worker_options. man/gearman_worker_register. man/gearman_worker_remove_options. man/gearman_worker_remove_servers. man/gearman_worker_set_context. man/gearman_worker_set_log_fn. man/gearman_worker_set_namespace. man/gearman_worker_set_options. man/gearman_worker_set_timeout. man/gearman_client_has_option. man/gearman_client_options_t. man/gearman_task_attr_init. man/gearman_task_attr_init_background. man/gearman_task_attr_init_epoch. man/gearman_task_attr_t. man/gearman_worker_set_identifier. man/gearman_worker_set_workload_free_fn. man/gearman_worker_set_workload_malloc_fn. man/gearman_worker_st. man/gearman_worker_timeout. man/gearman_worker_unregister. man/gearman_worker_unregister_all. man/gearman_worker_wait. man/gearman_worker_work. man/libgearman. '/usr/myapp/gearman/share/man/man3'
   /bin/sh ./libtool   --mode=install /usr/bin/install -c gearmand/gearmand '/usr/myapp/gearman/sbin'
 libtool: install: /usr/bin/install -c gearmand/gearmand /usr/myapp/gearman/sbin/gearmand
  /usr/bin/mkdir -p '/usr/myapp/gearman/bin'
   /bin/sh ./libtool   --mode=install /usr/bin/install -c bin/gearman bin/gearadmin '/usr/myapp/gearman/bin'
 libtool: install: /usr/bin/install -c bin/.libs/gearman /usr/myapp/gearman/bin/gearman
 libtool: install: /usr/bin/install -c bin/gearadmin /usr/myapp/gearman/bin/gearadmin
 make[]: Leaving directory `/usr/myapp/gearmand-1.1.'
 make[]: Leaving directory `/usr/myapp/gearmand-1.1.'
 make[]: Leaving directory `/usr/myapp/gearmand-1.1.'

<4> 启动gearmand，你也可以用 -d 开启后台运行的模式，这里加上DEBUG只是看一下实时的DEBUG信息，如下所示：

 [root@localhost myapp]# cd /usr/myapp/gearman
 [root@localhost gearman]# ls
 bin  include  lib  sbin  share
 [root@localhost gearman]# cd bin
 [root@localhost bin]# ls
 gearadmin  gearman
 [root@localhost bin]# cd /usr/myapp/gearman
 [root@localhost gearman]# cd sbin
 [root@localhost sbin]# ls
 gearmand
 [root@localhost sbin]# ./gearmand --verbose DEBUG
 ./gearmand: Could not open log file "/usr/myapp/gearman/var/log/gearmand.log", from "/usr/myapp/gearman/sbin", switching to stderr. (No such file or directory)
   DEBUG -- ::10.796259 [  main ] THREADS:  -> libgearman-server/gearmand.cc:
    INFO -- ::10.796374 [  main ] Initializing Gear on port  with SSL: false
    INFO -- ::10.796487 [  main ] Starting up with pid , verbose is set to DEBUG
   DEBUG -- ::10.796637 [  main ] Method for libevent: epoll -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.798874 [  main ] Trying to listen on 0.0.0.0: -> libgearman-server/gearmand.cc:
    INFO -- ::10.800151 [  main ] Listening on 0.0.0.0: ()
   DEBUG -- ::10.800175 [  main ] Trying to listen on ::: -> libgearman-server/gearmand.cc:
    INFO -- ::10.800307 [  main ] Listening on ::: ()
   DEBUG -- ::10.800333 [  main ] Creating wakeup pipe -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.800344 [  main ] Creating  threads -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.800357 [  main ] Initializing libevent for IO thread -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800406 [  main ] Creating IO thread wakeup pipe -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800467 [  main ] Thread  created -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800507 [  main ] Initializing libevent for IO thread -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800550 [  main ] Creating IO thread wakeup pipe -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800585 [  main ] Thread  created -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800594 [  main ] Initializing libevent for IO thread -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800632 [  main ] Creating IO thread wakeup pipe -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800669 [  main ] Thread  created -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800677 [  main ] Initializing libevent for IO thread -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800714 [  main ] Creating IO thread wakeup pipe -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800753 [  main ] Thread  created -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.800761 [  main ] replaying queue: begin -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.800766 [  main ] __replay -> libgearman-server/plugins/queue/default/queue.cc:
   DEBUG -- ::10.800774 [  main ] replaying queue: end -> libgearman-server/gearmand.cc:
    INFO -- ::10.800780 [  main ] Adding event for listening socket ()
    INFO -- ::10.800787 [  main ] Adding event for listening socket ()
   DEBUG -- ::10.800794 [  main ] Adding event for wakeup pipe -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.800801 [  main ] Entering main event loop -> libgearman-server/gearmand.cc:
   DEBUG -- ::10.801186 [      ] Entering thread event loop -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.801277 [      ] Entering thread event loop -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.801507 [  main ] staring up Epoch thread -> libgearman-server/timer.cc:
   DEBUG -- ::10.801635 [      ] Entering thread event loop -> libgearman-server/gearmand_thread.cc:
   DEBUG -- ::10.802426 [      ] Entering thread event loop -> libgearman-server/gearmand_thread.cc:

<5> 最后通过netstat，lsof, ps -ef 三板斧可以找出来gearmand大概占用的端口号，就如你看到的默认占用的4370端口，

当然你也可以在启动的时候用help命令也是能够知道的。

 [root@localhost ~]# netstat -tln
 Active Internet connections (only servers)
 Proto Recv-Q Send-Q Local Address           Foreign Address         State
 tcp               192.168.122.1:        0.0.0.0:*               LISTEN
 tcp               0.0.0.0:              0.0.0.0:*               LISTEN
 tcp               127.0.0.1:           0.0.0.0:*               LISTEN
 tcp               127.0.0.1:            0.0.0.0:*               LISTEN
 tcp        0      0 0.0.0.0:4730            0.0.0.0:*               LISTEN
 tcp6              :::                 :::*                    LISTEN
 tcp6              :::                 :::*                    LISTEN
 tcp6              :::                   :::*                    LISTEN
 tcp6              :::                 :::*                    LISTEN
 tcp6              :::                  :::*                    LISTEN
 tcp6              :::                 :::*                    LISTEN
 tcp6              127.0.0.1:          :::*                    LISTEN
 [root@localhost ~]# ps -ef | grep gearmand
 root      40299  15869  0 02:31 pts/1    00:00:00 ./gearmand --verbose DEBUG
 root           : pts/    :: grep --color=auto gearmand
 [root@localhost ~]# lsof -i :
 COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
 gearmand 40299 root    8u  IPv4 322550      0t0  TCP *:gearman (LISTEN)
 gearmand  root    9u  IPv6       0t0  TCP *:gearman (LISTEN)
 [root@localhost ~]# 

二：Java Driver 在 Gearman上的使用

为了演示，我可以做一个简单的 “字符串.ToUpper”的业务逻辑来验证一下这个架构是否可以跑的起来。

1. java 充当 Gearman 的 work

首先需要在mvn仓库中拉一下jar包：http://www.mvnrepository.com/artifact/org.gearman/gearman-java/0.6。

<1> UpperFunction类，这个类用于定义work具体的业务逻辑：

 package com.datamip.gearmanwork;

 import java.text.SimpleDateFormat;
 import java.util.Date;

 import org.gearman.client.GearmanJobResult;
 import org.gearman.client.GearmanJobResultImpl;
 import org.gearman.util.ByteUtils;
 import org.gearman.worker.AbstractGearmanFunction;

 //字符串大写的业务Function
 public class UpperFunction extends AbstractGearmanFunction {

     @Override
     public GearmanJobResult executeFunction() {

         String param = ByteUtils.fromUTF8Bytes((byte[]) this.data);

         byte[] mybytes = param.toUpperCase().getBytes();

         GearmanJobResultImpl result = new GearmanJobResultImpl(mybytes, true, mybytes, null, null, -1, -1);

         SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

         String dateString = formatter.format(new Date());

         System.out.println(String.format("当前时间:%s, 过来的字符串:%s，返回的字符串:%s", dateString, param,new String(mybytes)));

         return result;
     }
 }

<2>  将UpperFunction注册到gearmand中，从红色代码可以看到，其实是一个kv模式，这里的key="myUpperFunc”的对应执行业务就是new UpperFunction。

这样Client只需要传递一个"myUpperFunc"，Gearmand就知道这个“字符串”对应是哪一个处理函数。。。

 public class App {
     public static void main(String[] args) {

         GearmanWorker worker = new GearmanWorkerImpl();

         GearmanNIOJobServerConnection conn = new GearmanNIOJobServerConnection("192.168.23.170", 4730);
         worker.addServer(conn);

         // 将‘将转大写的函数注册’ 到gearmand中
10         worker.registerFunctionFactory(new GearmanFunctionFactory() {
11
12             public String getFunctionName() {
13                 return "myUpperFunc";
14             }
15
16             public GearmanFunction getFunction() {
17                 return new UpperFunction();
18             }
19         });

         System.out.println("启动服务。。。。");

         worker.work();
     }
 }

2. java 充当 Gearman 的 client

<1> GearSubmit类【简单的一个包装类，随便定义】

 package com.datamip.gearmanclient;

 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
 import java.util.concurrent.Future;

 import org.gearman.client.GearmanClient;
 import org.gearman.client.GearmanClientImpl;
 import org.gearman.client.GearmanJob;
 import org.gearman.client.GearmanJobImpl;
 import org.gearman.client.GearmanJobResult;
 import org.gearman.common.GearmanJobServerConnection;
 import org.gearman.common.GearmanNIOJobServerConnection;
 import org.gearman.util.ByteUtils;

 public class Gearsubmit {

     public  void process() throws InterruptedException, ExecutionException {

         GearmanJobServerConnection conn = new GearmanNIOJobServerConnection("192.168.23.170", 4730);

         GearmanClient client = new GearmanClientImpl();

         client.addJobServer(conn); // 添加连接

         String functionName = "myUpperFunc";

         byte[] data = ByteUtils.toUTF8Bytes("hello,world");

         // 创建后台任务
         GearmanJob job = GearmanJobImpl.createJob(functionName, data, null);

         GearmanJobResult jobResult = null;

         Future<GearmanJobResult> gearmanJobResult = client.submit(job);

         jobResult = gearmanJobResult.get();

         byte[] resultBytes = jobResult.getResults();

         // 获取job的返回值
         String value = ByteUtils.fromUTF8Bytes(resultBytes);

         System.out.println(value);

         System.out.println("执行结束");

         client.shutdown();
     }
 }

<2> 主程序，开多线程并发的去执行。

 public class App {
     public static void main(String[] args) throws InterruptedException, ExecutionException, IOException {

         ExecutorService executorService = Executors.newFixedThreadPool(100);

         for (int i = 0; i < 10000; i++) {
             executorService.execute(new Runnable() {

                 @Override
                 public void run() {
                     Gearsubmit submit=new Gearsubmit();

                     try {
                         submit.process();
                     } catch (InterruptedException | ExecutionException e) {
                         // TODO Auto-generated catch block
                         e.printStackTrace();
                     }
                 }
             });
         }

         System.in.read();
     }
 }

好了，一切都准备好了，接下来为了演示，演示就是解释，我用Jar2Exe把work程序导出成jar再转换成exe，如下图：

然后我把3.exe开成5个实例，client用100个线程的线程池并发调用，当然一切都是模拟。。。。可以看到，当我client启动的时候，5个work都在执行，

如果这个时候，你把某一个work停止了，jobserver也不再将任务丢给它，而是转给其他负载相对小的work继续执行。

好了，本篇就说到这里，希望对你有帮助。