streamsets mongodb destinations 使用

测试集成了directory（excel） 以及redis && field splitter 组件

pipeline flow

docker-compose 配置

redis 服务&& streamsets

version: "3"
services:
   sets:
     image: streamsets/datacollector
     volumes:
     - "./ms/data:/data"
     - "./ms/logs:/logs"
     - "./ms/tmp:/tmp"
     ports:
     - "8000:8000"
     - "18630:18630"
   redis:
     image: redis
     ports:
     - "6379:6379"
   mongodb:
     image: mongo
     ports:
     - "27017:27017"

directory 配置

参考https://www.cnblogs.com/rongfengliang/p/9509467.html 使用了excel 格式
excel 格式

field splitter

添加expression Evaluator 配置

添加crud header

mongodb 配置

运行&&效果

• 运行
  
• mongodb
  

参考资料

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Destinations/Redis.html#concept_ktc_gw2_gw
https://github.com/rongfengliang/streamsets-demos