【NIFI】 开发自定义Nifi Processor

　　本例需要基础知识：【NIFI】 Apache NiFI 安装及简单的使用

　　Nifi不光可以使用自带的Processor，还可以自定义Processor。本例简单介绍开发一个Processor

开发

　　1、新建一个Maven工程，这里采用的是eclipse的模板原型来创建。

　　　　a、创建

　　　　

　　　　b、添加模板，内容：

• • • Archetype Group Id：org.apache.nifi
    • Archetype Artifact Id：nifi-processor-bundle-archetype
    • Archetype Version：1.2.0

　　　　

　　　　c、根据模板创建，maven项目

　　　　

　　2、创建后，项目目录如下：

　　　　

　　　　其中3个pom文件如下

 <?xml version="1.0" encoding="UTF-8"?>
 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements. See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License. You may obtain a copy of the License at
   http://www.apache.org/licenses/LICENSE-2.0
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>

     <parent>
         <groupId>org.apache.nifi</groupId>
         <artifactId>nifi-nar-bundles</artifactId>
         <version>1.2.0</version>
     </parent>

     <groupId>com.test</groupId>
     <artifactId>my-processor</artifactId>
     <version>1.0-SNAPSHOT</version>
     <packaging>pom</packaging>

     <modules>
         <module>nifi-my-processor-processors</module>
         <module>nifi-my-processor-nar</module>
     </modules>

 </project>

my-processor pom.xml　

　　

 <?xml version="1.0" encoding="UTF-8"?>
 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements. See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License. You may obtain a copy of the License at
   http://www.apache.org/licenses/LICENSE-2.0
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>

     <parent>
         <groupId>com.test</groupId>
         <artifactId>my-processor</artifactId>
         <version>1.0-SNAPSHOT</version>
     </parent>

     <artifactId>nifi-my-processor-nar</artifactId>
     <packaging>nar</packaging>
     <properties>
         <maven.javadoc.skip>true</maven.javadoc.skip>
         <source.skip>true</source.skip>
     </properties>

     <dependencies>
         <dependency>
             <groupId>com.test</groupId>
             <artifactId>nifi-my-processor-processors</artifactId>
             <version>1.0-SNAPSHOT</version>
         </dependency>
     </dependencies>

 </project>

nifi-my-processor-nar pom.xml

 <?xml version="1.0" encoding="UTF-8"?>
 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements. See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License. You may obtain a copy of the License at
   http://www.apache.org/licenses/LICENSE-2.0
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>

     <parent>
         <groupId>com.test</groupId>
         <artifactId>my-processor</artifactId>
         <version>1.0-SNAPSHOT</version>
     </parent>

     <artifactId>nifi-my-processor-processors</artifactId>
     <packaging>jar</packaging>

     <dependencies>
         <dependency>
             <groupId>org.apache.nifi</groupId>
             <artifactId>nifi-api</artifactId>
         </dependency>
         <dependency>
             <groupId>org.apache.nifi</groupId>
             <artifactId>nifi-utils</artifactId>
         </dependency>
         <dependency>
             <groupId>org.apache.nifi</groupId>
             <artifactId>nifi-mock</artifactId>
             <scope>test</scope>
         </dependency>
         <dependency>
             <groupId>org.slf4j</groupId>
             <artifactId>slf4j-simple</artifactId>
             <scope>test</scope>
         </dependency>
         <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <scope>test</scope>
         </dependency>
     </dependencies>
 </project>

nifi-my-processor-processors pom.xml

　　

　　3、修改项目，因环境引起的错误

　　　　a、删除nifi-my-processor-processors子项目中，src/test中的测试文件（打包可能出现错误）

　　　　b、在org.apache.nifi.processor.Processor文件中配置自己的Porcessor

　　　　　　

　　4、代码编写，编辑MyProcessor.java文件，文件在项目创建的时候已经生成，做适当修改即可。其中有设置状态，属性，及处理方法（onTrigger）等

　　　　

　　　　内容：

 /*
  * Licensed to the Apache Software Foundation (ASF) under one or more
  * contributor license agreements.  See the NOTICE file distributed with
  * this work for additional information regarding copyright ownership.
  * The ASF licenses this file to You under the Apache License, Version 2.0
  * (the "License"); you may not use this file except in compliance with
  * the License.  You may obtain a copy of the License at
  *
  *     http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
 package com.test.processors;

 import org.apache.nifi.components.PropertyDescriptor;
 import org.apache.nifi.flowfile.FlowFile;
 import org.apache.nifi.annotation.behavior.ReadsAttribute;
 import org.apache.nifi.annotation.behavior.ReadsAttributes;
 import org.apache.nifi.annotation.behavior.WritesAttribute;
 import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.lifecycle.OnScheduled;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.SeeAlso;
 import org.apache.nifi.annotation.documentation.Tags;
 import org.apache.nifi.processor.exception.ProcessException;
 import org.apache.nifi.processor.AbstractProcessor;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
 import org.apache.nifi.processor.ProcessorInitializationContext;
 import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.util.StandardValidators;

 import java.io.InputStreamReader;
 import java.io.StringWriter;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
 import java.util.concurrent.atomic.AtomicReference;

 @Tags({"example"})
 @CapabilityDescription("Provide a description")
 @SeeAlso({})
 @ReadsAttributes({@ReadsAttribute(attribute="", description="")})
 @WritesAttributes({@WritesAttribute(attribute="", description="")})
 public class MyProcessor extends AbstractProcessor {

     public static final PropertyDescriptor MY_PROPERTY = new PropertyDescriptor
             .Builder().name("MY_PROPERTY")
             .displayName("My property")
             .description("Example Property")
             .required(true)
             .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
             .build();

     public static final Relationship MY_RELATIONSHIP_SUCCESS = new Relationship.Builder()
             .name("sucess")
             .description("Example relationship Success")
             .build();

     public static final Relationship MY_RELATIONSHIP_FAILURE = new Relationship.Builder()
             .name("failure")
             .description("Example relationship Failure")
             .build();

     private List<PropertyDescriptor> descriptors;

     private Set<Relationship> relationships;

     @Override
     protected void init(final ProcessorInitializationContext context) {
         final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
         descriptors.add(MY_PROPERTY);
         this.descriptors = Collections.unmodifiableList(descriptors);

         final Set<Relationship> relationships = new HashSet<Relationship>();
         relationships.add(MY_RELATIONSHIP_SUCCESS);
         relationships.add(MY_RELATIONSHIP_FAILURE);
         this.relationships = Collections.unmodifiableSet(relationships);
     }

     @Override
     public Set<Relationship> getRelationships() {
         return this.relationships;
     }

     @Override
     public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
         return descriptors;
     }

     @OnScheduled
     public void onScheduled(final ProcessContext context) {

     }

     @Override
     public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
         FlowFile flowFile = session.get();
         if ( flowFile == null ) {
             return;
         }
         // TODO implement
         final AtomicReference<String> value = new AtomicReference<>();
         session.read(flowFile, in -> {
             try{
                 StringWriter sw = new StringWriter();
                 InputStreamReader inr = new InputStreamReader(in);
                 char[] buffer = new char[1024];
                 int n = 0;
                 while (-1 != (n = inr.read(buffer))) {
                     sw.write(buffer, 0, n);
                 }
                 String str = sw.toString();

                 String result = "处理了：" + str + context.getProperty("MY_PROPERTY").getValue();
                 value.set(result);
             }catch(Exception ex){
                 ex.printStackTrace();
                 getLogger().error("Failed to read json string.");
             }
         });

         String results = value.get();
         if(results != null && !results.isEmpty()){
             flowFile = session.putAttribute(flowFile, "match", results);
         }

         flowFile = session.write(flowFile, out -> out.write(value.get().getBytes()));

         session.transfer(flowFile, MY_RELATIONSHIP_SUCCESS);

     }
 }

　　5、打包，使用maven命令：mvn clean package

　　6、将nifi-my-processor-nar工程target目录中的 nifi-my-processor-nar-1.0-SNAPSHOT.nar 文件，拷贝到 nifi\lib 目录中

　　7、启动 NIFI 项目，使用自定义的Process：MyProcessor

　　　　配置如下：

　　　　　　a、拉入三个Processor

　　　　　　　　

　　　　　　b、配置三个Processor，

　　　　　　　　下图是GenerateFlowFile的配置，主要配置了执行的时间（10s）及产生的字符串（123）

　　　　　　　　  

　　　　　　　　下图是MyProcessor配置

　　　　　　　　

　　　　　　c、启动三个Processor

　　　　　　d、查看输出，可以看到字符串 123 经过处理成 ： "处理了：123abc"