kettle从入门到精通 第四十五课 ETL之 kettle redis

1、kettle 9.3/9.4 spoon客户端中默认是没有redis步骤的，首先想到在kettle的插件市场进行下载redis步骤。

2、可能因为网络原因，直接下载失败了。索性放弃redis原有插件步骤，改为自己通过java代码进行实现，（有java基础的可以通过java代码步骤结合自定义jar包实现一切功能）。

3、将jedis-2.10.2.jar（jedis版本最好和jdk1版本匹配） 放到kettle的lib目录下，重启spoon客户端。

步骤【生成记录】生成一笔测试数据，步骤【Java代码-redis-set】往redis里面写数据，步骤【Java代码-redis-get】从redis读取数据，如下图所示。

4、步骤【java代码-redis-set】实现逻辑如下：

代码中import 包时，一定要放到最顶部。下面代码中标红的部分为关键代码，其他为模版代码。

import redis.clients.jedis.Jedis;

public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException {
  if (first) {
    first = false;

    /* TODO: Your code here. (Using info fields)

    FieldHelper infoField = get(Fields.Info, "info_field_name");

    RowSet infoStream = findInfoRowSet("info_stream_tag");

    Object[] infoRow = null;

    int infoRowCount = 0;

    // Read all rows from info step before calling getRow() method, which returns first row from any
    // input rowset. As rowMeta for info and input steps varies getRow() can lead to errors.
    while((infoRow = getRowFrom(infoStream)) != null){

      // do something with info data
      infoRowCount++;
    }
    */
  }

  Object[] r = getRow();

  if (r == null) {
    setOutputDone();
    return false;
  }

Jedis jedis = new Jedis("localhost", 6379);
// 设置键值对
jedis.set("name", "Java小金刚");
// 关闭连接
jedis.close();

  // It is always safest to call createOutputRow() to ensure that your output row's Object[] is large
  // enough to handle any new fields you are creating in this step.
  r = createOutputRow(r, data.outputRowMeta.size());

  /* TODO: Your code here. (See Sample)

  // Get the value from an input field
  String foobar = get(Fields.In, "a_fieldname").getString(r);

  foobar += "bar";

  // Set a value in a new output field
  get(Fields.Out, "output_fieldname").setValue(r, foobar);

  */
  // Send the row on to the next step.
  putRow(data.outputRowMeta, r);

  return true;
}

5、步骤【java代码-redis-get】实现逻辑如下：

代码中import 包时，一定要放到最顶部。下面代码中标红的部分为关键代码，其他为模版代码。

import redis.clients.jedis.Jedis;

public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException {
  if (first) {
    first = false;

    /* TODO: Your code here. (Using info fields)

    FieldHelper infoField = get(Fields.Info, "info_field_name");

    RowSet infoStream = findInfoRowSet("info_stream_tag");

    Object[] infoRow = null;

    int infoRowCount = 0;

    // Read all rows from info step before calling getRow() method, which returns first row from any
    // input rowset. As rowMeta for info and input steps varies getRow() can lead to errors.
    while((infoRow = getRowFrom(infoStream)) != null){

      // do something with info data
      infoRowCount++;
    }
    */
  }

  Object[] r = getRow();

  if (r == null) {
    setOutputDone();
    return false;
  }

Jedis jedis = new Jedis("localhost", 6379);
// 获取键值对
String name = jedis.get("name");

// 关闭连接
jedis.close();

  // It is always safest to call createOutputRow() to ensure that your output row's Object[] is large
  // enough to handle any new fields you are creating in this step.
  r = createOutputRow(r, data.outputRowMeta.size());

  /* TODO: Your code here. (See Sample)

  // Get the value from an input field
  String foobar = get(Fields.In, "a_fieldname").getString(r);

  foobar += "bar";

  // Set a value in a new output field
  get(Fields.Out, "output_fieldname").setValue(r, foobar);

  */
  get(Fields.Out, "name").setValue(r, name);
  // Send the row on to the next step.
  putRow(data.outputRowMeta, r);

  return true;
}