DolphinScheduler3.1.7集成SAP HANA
源码地址：GitHub - apache/dolphinscheduler at 3.1.7-release
个人fork gitee地址：DolphinScheduler:Gitee)
后端代码更改项：
修改项1:DataSourceConstants.java
路径：dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/DataSourceConstants.java
public class DataSourceConstants {
public static final String COM_PRESTO_JDBC_DRIVER = "com.facebook.presto.jdbc.PrestoDriver";
public static final String COM_REDSHIFT_JDBC_DRIVER = "com.amazon.redshift.jdbc42.Driver";
public static final String COM_ATHENA_JDBC_DRIVER = "com.simba.athena.jdbc.Driver";
//add HANA driver message
public static final String COM_HANA_DB_JDBC_DRIVER = "com.sap.db.jdbc.Driver";
/**
* validation Query
*/
public static final String PRESTO_VALIDATION_QUERY = "select 1";
public static final String REDHIFT_VALIDATION_QUERY = "select 1";
public static final String ATHENA_VALIDATION_QUERY = "select 1";
// add HANA Connect check sql
public static final String HANA_VALIDATION_QUERY = "select 1";
/**
* jdbc url
*/
public static final String JDBC_PRESTO = "jdbc:presto://";
public static final String JDBC_REDSHIFT = "jdbc:redshift://";
public static final String JDBC_ATHENA = "jdbc:awsathena://";
// add HANA jdbc url modual
public static final String JDBC_HANA = "jdbc:sap://";
修改项2:前端映射枚举值
路径：dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java
H2(9, "h2"),
REDSHIFT(10,"redshift"),
ATHENA(11,"athena"),
HANA(12,"hana"),
;
修改项3:dolphinscheduler-datasource-hana模块
路径dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana，模块层级如下
pom文件
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-plugin</artifactId>
<version>3.1.8-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-datasource-hana</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-api</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.sap.cloud.db.jdbc</groupId>
<artifactId>ngdbc</artifactId>
<version>2.17.10</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<type>jar</type>
<scope>test</scope>
</dependency>
</dependencies>
</project>
HanaDataSourceChannel.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannel.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceClient;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceChannel implements DataSourceChannel {
@Override
public DataSourceClient createDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
return new HanaDataSourceClient(baseConnectionParam, dbType);
}
}
HanaDataSourceChannelFactory.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelFactory.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory;
import com.google.auto.service.AutoService;
@AutoService(DataSourceChannelFactory.class)
public class HanaDataSourceChannelFactory implements DataSourceChannelFactory {
@Override
public String getName() {
return "hana";
}
@Override
public DataSourceChannel create() {
return new HanaDataSourceChannel();
}
}
HanaDataSourceClient.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceClient.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceClient extends CommonDataSourceClient {
public HanaDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
super(baseConnectionParam, dbType);
}
}
HanaConnectionParam.java
路径:dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaConnectionParam.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
public class HanaConnectionParam extends BaseConnectionParam {
@Override
public String toString() {
return "HanaConnectionParam{"
+ "user='" + user + '\''
+ ", password='" + password + '\''
+ ", address='" + address + '\''
+ ", database='" + database + '\''
+ ", jdbcUrl='" + jdbcUrl + '\''
+ ", driverLocation='" + driverLocation + '\''
+ ", driverClassName='" + driverClassName + '\''
+ ", validationQuery='" + validationQuery + '\''
+ ", other='" + other + '\''
+ '}';
}
}
HanaDataSourceParamDTO.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceParamDTO.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceParamDTO extends BaseDataSourceParamDTO {
@Override
public String toString() {
return "HanaDataSourceParamDTO{"
+ "name='" + name + '\''
+ ", note='" + note + '\''
+ ", host='" + host + '\''
+ ", port=" + port
+ ", database='" + database + '\''
+ ", userName='" + userName + '\''
+ ", password='" + password + '\''
+ ", other='" + other + '\''
+ '}';
}
@Override
public DbType getType() {
return DbType.HANA;
}
}
HanaDataSourceProcessor.java
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceProcessor.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.commons.lang3.StringUtils;
import org.apache.dolphinscheduler.common.constants.Constants;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.apache.commons.collections4.MapUtils;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.auto.service.AutoService;
@AutoService(DataSourceProcessor.class)
public class HanaDataSourceProcessor extends AbstractDataSourceProcessor {
private final Logger logger = LoggerFactory.getLogger(HanaDataSourceProcessor.class);
private static final String APPEND_PARAMS = "reconnect=true";
@Override
public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
return JSONUtils.parseObject(paramJson, HanaDataSourceParamDTO.class);
}
@Override
public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
HanaConnectionParam connectionParams = (HanaConnectionParam) createConnectionParams(connectionJson);
HanaDataSourceParamDTO hanaDatasourceParamDTO = new HanaDataSourceParamDTO();
hanaDatasourceParamDTO.setUserName(connectionParams.getUser());
hanaDatasourceParamDTO.setDatabase(connectionParams.getDatabase());
String address = connectionParams.getAddress();
String[] hostSeperator = address.split(Constants.DOUBLE_SLASH);
String[] hostPortArray = hostSeperator[hostSeperator.length - 1].split(Constants.COMMA);
hanaDatasourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
hanaDatasourceParamDTO.setHost(hostPortArray[0].split(Constants.COLON)[0]);
return hanaDatasourceParamDTO;
}
@Override
public BaseConnectionParam createConnectionParams(BaseDataSourceParamDTO dataSourceParam) {
HanaDataSourceParamDTO hanaDatasourceParam = (HanaDataSourceParamDTO) dataSourceParam;
String address = String.format("%s%s:%s", DataSourceConstants.JDBC_HANA, hanaDatasourceParam.getHost(),
hanaDatasourceParam.getPort());
String jdbcUrl = String.format("%s?currentschema=%s", address, hanaDatasourceParam.getDatabase());
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl(jdbcUrl);
hanaConnectionParam.setValidationQuery("select 1 from DUMMY");
hanaConnectionParam.setDatabase(hanaDatasourceParam.getDatabase());
hanaConnectionParam.setAddress(address);
hanaConnectionParam.setUser(hanaDatasourceParam.getUserName());
hanaConnectionParam.setPassword(PasswordUtils.encodePassword(hanaDatasourceParam.getPassword()));
hanaConnectionParam.setDriverClassName(getDatasourceDriver());
hanaConnectionParam.setOther(transformOther(hanaDatasourceParam.getOther()));
return hanaConnectionParam;
}
@Override
public ConnectionParam createConnectionParams(String connectionJson) {
return JSONUtils.parseObject(connectionJson, HanaConnectionParam.class);
}
@Override
public String getDatasourceDriver() {
return DataSourceConstants.COM_HANA_DB_JDBC_DRIVER;
}
@Override
public String getValidationQuery() {
return DataSourceConstants.COM_HANA_DB_JDBC_DRIVER;
}
@Override
public String getJdbcUrl(ConnectionParam connectionParam) {
HanaConnectionParam hanaConnectionParam = (HanaConnectionParam) connectionParam;
String jdbcUrl = hanaConnectionParam.getJdbcUrl();
// get customize connection message,parse it to map
if (MapUtils.isNotEmpty(parseOther(hanaConnectionParam.getOther()))) {
return String.format("%s?%s&%s", jdbcUrl, hanaConnectionParam.getOther(), APPEND_PARAMS);
}
return String.format("%s&%s", jdbcUrl, APPEND_PARAMS);
}
@Override
public Connection getConnection(ConnectionParam connectionParam) throws ClassNotFoundException, SQLException {
HanaConnectionParam hanaConnectionParam = (HanaConnectionParam) connectionParam;
Class.forName(getDatasourceDriver());
String user = hanaConnectionParam.getUser();
String password = PasswordUtils.decodePassword(hanaConnectionParam.getPassword());
return DriverManager.getConnection(getJdbcUrl(connectionParam), user, password);
}
@Override
public DbType getDbType() {
return DbType.HANA;
}
@Override
public DataSourceProcessor create() {
logger.info("create hana datasource processor success");
return new HanaDataSourceProcessor();
}
/**
* transform map to string with &
* @param otherMap connection map
* @return return the connection string format
*/
private String transformOther(Map<String, String> otherMap) {
if (MapUtils.isEmpty(otherMap)) {
return null;
}
List<String> list = new ArrayList<>();
otherMap.forEach((key, value) -> list.add(String.format("%s=%s", key, value)));
return String.join("&", list);
}
/**
* transform string to map
* @param other connection string
* @return return connection map format
*/
private Map<String, String> parseOther(String other) {
if (StringUtils.isEmpty(other)) {
return null;
}
Map<String, String> otherMap = new LinkedHashMap<>();
for (String config : other.split("&")) {
otherMap.put(config.split("=")[0], config.split("=")[1]);
}
return otherMap;
}
}
HanaDataSourceProcessorTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceProcessorTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.HashMap;
import java.util.Map;
@ExtendWith(MockitoExtension.class)
public class HanaDataSourceProcessorTest {
private HanaDataSourceProcessor hanaDataSourceProcessor = new HanaDataSourceProcessor();
@Test
public void testCreateConnectionParams() {
Map<String, String> props = new HashMap<>();
HanaDataSourceParamDTO mysqlDatasourceParamDTO = new HanaDataSourceParamDTO();
mysqlDatasourceParamDTO.setUserName("root");
mysqlDatasourceParamDTO.setPassword("123456");
mysqlDatasourceParamDTO.setHost("localhost");
mysqlDatasourceParamDTO.setPort(30015);
mysqlDatasourceParamDTO.setDatabase("default");
mysqlDatasourceParamDTO.setOther(props);
try (MockedStatic<PasswordUtils> mockedPasswordUtils = Mockito.mockStatic(PasswordUtils.class)) {
Mockito.when(PasswordUtils.encodePassword(Mockito.anyString())).thenReturn("test");
HanaConnectionParam connectionParams = (HanaConnectionParam) hanaDataSourceProcessor
.createConnectionParams(mysqlDatasourceParamDTO);
Assertions.assertEquals("jdbc:sap://localhost:30015", connectionParams.getAddress());
Assertions.assertEquals("jdbc:sap://localhost:30015?currentschema=default", connectionParams.getJdbcUrl());
}
}
@Test
public void testCreateConnectionParams2() {
String connectionJson = "{\"user\":\"root\",\"password\":\"123456\",\"address\":\"jdbc:sap://localhost:30015\""
+ ",\"database\":\"default\",\"jdbcUrl\":\"jdbc:sap://localhost:30015?currentschema=default\"}";
HanaConnectionParam connectionParams = (HanaConnectionParam) hanaDataSourceProcessor
.createConnectionParams(connectionJson);
Assertions.assertNotNull(connectionJson);
Assertions.assertEquals("root", connectionParams.getUser());
}
@Test
public void testGetDatasourceDriver() {
Assertions.assertEquals(DataSourceConstants.COM_HANA_DB_JDBC_DRIVER,
hanaDataSourceProcessor.getDatasourceDriver());
}
@Test
public void testGetJdbcUrl() {
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015?currentschema=default");
Assertions.assertEquals(
"jdbc:sap://localhost:30015?currentschema=default&reconnect=true",
hanaDataSourceProcessor.getJdbcUrl(hanaConnectionParam));
}
@Test
public void testGetDbType() {
Assertions.assertEquals(DbType.HANA, hanaDataSourceProcessor.getDbType());
}
@Test
public void testGetValidationQuery() {
Assertions.assertEquals(DataSourceConstants.HANA_VALIDATION_QUERY,
hanaDataSourceProcessor.getValidationQuery());
}
@Test
public void testGetDatasourceUniqueId() {
HanaConnectionParam mysqlConnectionParam = new HanaConnectionParam();
mysqlConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015?currentschema=default");
mysqlConnectionParam.setUser("root");
mysqlConnectionParam.setPassword("123456");
try (MockedStatic<PasswordUtils> mockedPasswordUtils = Mockito.mockStatic(PasswordUtils.class)) {
Mockito.when(PasswordUtils.encodePassword(Mockito.anyString())).thenReturn("123456");
Assertions.assertEquals("hana@root@123456@jdbc:sap://localhost:30015?currentschema=default",
hanaDataSourceProcessor.getDatasourceUniqueId(mysqlConnectionParam, DbType.HANA));
}
}
}
JDBCDataSourceProviderTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/provider/JDBCDataSourceProviderTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.provider;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class JDBCDataSourceProviderTest {
@Test
public void testCreateJdbcDataSource() {
try (
MockedStatic<JDBCDataSourceProvider> mockedJDBCDataSourceProvider =
Mockito.mockStatic(JDBCDataSourceProvider.class)) {
HikariDataSource dataSource = Mockito.mock(HikariDataSource.class);
mockedJDBCDataSourceProvider
.when(() -> JDBCDataSourceProvider.createJdbcDataSource(Mockito.any(), Mockito.any()))
.thenReturn(dataSource);
Assertions.assertNotNull(
JDBCDataSourceProvider.createJdbcDataSource(new HanaConnectionParam(), DbType.HANA));
}
}
@Test
public void testCreateOneSessionJdbcDataSource() {
try (
MockedStatic<JDBCDataSourceProvider> mockedJDBCDataSourceProvider =
Mockito.mockStatic(JDBCDataSourceProvider.class)) {
HikariDataSource dataSource = Mockito.mock(HikariDataSource.class);
mockedJDBCDataSourceProvider
.when(() -> JDBCDataSourceProvider.createOneSessionJdbcDataSource(Mockito.any(), Mockito.any()))
.thenReturn(dataSource);
Assertions.assertNotNull(
JDBCDataSourceProvider.createOneSessionJdbcDataSource(new HanaConnectionParam(), DbType.HANA));
}
}
}
DataSourceUtilsTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/utils/DataSourceUtilsTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.utils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.PropertyUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.sql.Connection;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ExecutionException;
@ExtendWith(MockitoExtension.class)
public class DataSourceUtilsTest {
@Test
public void testCheckDatasourceParam() {
HanaDataSourceParamDTO hanaDataSourceParamDTO = new HanaDataSourceParamDTO();
hanaDataSourceParamDTO.setHost("localhost");
hanaDataSourceParamDTO.setDatabase("default");
Map<String, String> other = new HashMap<>();
other.put("reconnect", "true");
hanaDataSourceParamDTO.setOther(other);
DataSourceUtils.checkDatasourceParam(hanaDataSourceParamDTO);
Assertions.assertTrue(true);
}
@Test
public void testBuildConnectionParams() {
HanaDataSourceParamDTO hanaDataSourceParamDTO = new HanaDataSourceParamDTO();
hanaDataSourceParamDTO.setHost("localhost");
hanaDataSourceParamDTO.setDatabase("default");
hanaDataSourceParamDTO.setUserName("root");
hanaDataSourceParamDTO.setPort(30015);
hanaDataSourceParamDTO.setPassword("123456");
try (
MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
mockedStaticPasswordUtils.when(() -> PasswordUtils.encodePassword(Mockito.anyString()))
.thenReturn("123456");
mockedStaticCommonUtils.when(CommonUtils::getKerberosStartupState).thenReturn(false);
ConnectionParam connectionParam = DataSourceUtils.buildConnectionParams(hanaDataSourceParamDTO);
Assertions.assertNotNull(connectionParam);
}
}
@Test
public void testBuildConnectionParams2() {
HanaDataSourceParamDTO hanaDatasourceParamDTO = new HanaDataSourceParamDTO();
hanaDatasourceParamDTO.setHost("localhost");
hanaDatasourceParamDTO.setDatabase("default");
hanaDatasourceParamDTO.setUserName("root");
hanaDatasourceParamDTO.setPort(30015);
hanaDatasourceParamDTO.setPassword("123456");
ConnectionParam connectionParam =
DataSourceUtils.buildConnectionParams(DbType.HANA, JSONUtils.toJsonString(hanaDatasourceParamDTO));
Assertions.assertNotNull(connectionParam);
}
@Test
public void testGetConnection() throws ExecutionException {
try (
MockedStatic<PropertyUtils> mockedStaticPropertyUtils = Mockito.mockStatic(PropertyUtils.class);
MockedStatic<DataSourceClientProvider> mockedStaticDataSourceClientProvider =
Mockito.mockStatic(DataSourceClientProvider.class)) {
mockedStaticPropertyUtils.when(() -> PropertyUtils.getLong("kerberos.expire.time", 24L)).thenReturn(24L);
DataSourceClientProvider clientProvider = Mockito.mock(DataSourceClientProvider.class);
mockedStaticDataSourceClientProvider.when(DataSourceClientProvider::getInstance).thenReturn(clientProvider);
Connection connection = Mockito.mock(Connection.class);
Mockito.when(clientProvider.getConnection(Mockito.any(), Mockito.any())).thenReturn(connection);
HanaConnectionParam connectionParam = new HanaConnectionParam();
connectionParam.setUser("root");
connectionParam.setPassword("123456");
connection = DataSourceClientProvider.getInstance().getConnection(DbType.HANA, connectionParam);
Assertions.assertNotNull(connection);
}
}
@Test
public void testGetJdbcUrl() {
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015");
String jdbcUrl = DataSourceUtils.getJdbcUrl(DbType.HANA, hanaConnectionParam);
Assertions.assertEquals(
"jdbc:sap://localhost:30015?reconnect=true",
jdbcUrl);
}
@Test
public void testBuildDatasourceParamDTO() {
HanaConnectionParam connectionParam = new HanaConnectionParam();
connectionParam.setJdbcUrl(
"jdbc:sap://localhost:30015?reconnect=true");
connectionParam.setAddress("jdbc:mysql://localhost:30015");
connectionParam.setUser("root");
connectionParam.setPassword("123456");
Assertions.assertNotNull(
DataSourceUtils.buildDatasourceParamDTO(DbType.HANA, JSONUtils.toJsonString(connectionParam)));
}
@Test
public void testGetDatasourceProcessor() {
Assertions.assertNotNull(DataSourceUtils.getDatasourceProcessor(DbType.HANA));
}
@Test
public void testGetDatasourceProcessorError() {
Assertions.assertThrows(Exception.class, () -> {
DataSourceUtils.getDatasourceProcessor(null);
});
}
}
HanaDataSourceChannelFactoryTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelFactoryTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
public class HanaDataSourceChannelFactoryTest {
@Test
public void testCreate() {
HanaDataSourceChannelFactory sourceChannelFactory = new HanaDataSourceChannelFactory();
DataSourceChannel dataSourceChannel = sourceChannelFactory.create();
Assertions.assertNotNull(dataSourceChannel);
}
}
HanaDataSourceChannelTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class HanaDataSourceChannelTest {
@Test
public void testCreateDataSourceClient() {
HanaDataSourceChannel sourceChannel = Mockito.mock(HanaDataSourceChannel.class);
HanaDataSourceClient dataSourceClient = Mockito.mock(HanaDataSourceClient.class);
Mockito.when(sourceChannel.createDataSourceClient(Mockito.any(), Mockito.any())).thenReturn(dataSourceClient);
Assertions.assertNotNull(sourceChannel.createDataSourceClient(new HanaConnectionParam(), DbType.HANA));
}
}
修改项4:打包相关依赖
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-all/pom.xml
dependencies中新增hana模块依赖
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-hana</artifactId>
<version>${project.version}</version>
</dependency>
dolphinscheduler-datasource-plugin/pom.xml
modules中新增hana模块
<module>dolphinscheduler-datasource-hana</module>
前端UI代码变更
修改项1:types.ts
路径：dolphinscheduler-ui/src/service/modules/data-source/types.ts 新增hana选择
| 'PRESTO'
| 'REDSHIFT'
| 'ATHENA'
| 'HANA'
interface IDataSource {
id?: number
修改项6:
修改项2:use-form.ts
路径：dolphinscheduler-ui/src/views/datasource/list/use-form.ts 新增hana默认端口
value: 'ATHENA',
label: 'ATHENA',
defaultPort: 0
},
HANA: {
value: 'HANA',
label: 'HANA',
defaultPort: 30015
}
}
修改项3:use-datasource.ts
路径：dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-datasource.ts
id: 10,
code: 'ATHENA',
disabled: false
},
{
id: 11,
code: 'HANA',
disabled: false
}
]
编译打包
dolphinscheduler-dist/target/路径下生成tar.gz包
mvn -U clean package -Prelease -Dmaven.test.skip=true
DolphinScheduler3.1.7集成SAP HANA的更多相关文章
- SAP HANA专题分析目录
针对HANA的关键技术领域, 做深度解析. 1. HANA开发规范 HANA 各种对象的应用解析.版本管理,开发规范. 2. HANA系统管理 用户.系统权限.数据权限的深度解析. HANA系统配置. ...
- [转载]大道至简！！！从SAP HANA作为SAP加速器的方式，看ERP on HANA的春天
I AM A ABAPER! 科技的进步,一定会使一些东西变得越来越精简! 大道至简!!! 文章很好!!!!!!!!!!! -------------------------------------- ...
- SAP HANA中的SLT简介
在以SAP系统作为主要ERP的企业中,不同系统之间的数据库数据同步是个重要的工作.对于这种需求,除了开发ABAP接口之外,也有高效的工具可用.SLT就是其中之一. SLT是SAP的第一个ETL(Ext ...
- 【公众号系列】SAP HANA 平台的优势
公众号:SAP Technical 本文作者:matinal 原文出处:http://www.cnblogs.com/SAPmatinal/ 原文链接:[公众号系列]SAP HANA 平台的优势 ...
- 【公众号系列】SAP HANA和区块链
公众号:SAP Technical 本文作者:matinal 原文出处:http://www.cnblogs.com/SAPmatinal/ 原文链接:[公众号系列]SAP HANA和区块链 写在 ...
- 如何在云端部署SAP HANA实战, Azure 上的 SAP HANA（大型实例）概述和体系结构
什么是 Azure 上的 SAP HANA(大型实例)? Azure 上的 SAP HANA(大型实例)是一种针对 Azure 的独特解决方案. 除了提供 Azure 虚拟机以用于部署和运行 SAP ...
- sap hana 数据库 EBS
SAP实时数据平台详解 ************************************************************ EBS是Oracle 公司对原有应用产品整合后的一个产 ...
- SAP HANA 开发模式 - 基于SAP HANA平台的多团队产品研发
“基本”开发模式 Windows: Unix/Linux: 在基本模式下我们可以通过regi来进行激活我们的object.Regi是一个类git功能的,方便和HANA repository交互的一个命 ...
- 基于SAP HANA平台的多团队产品研发
工欲善其事必先利其器.要提高多团队的开发效率,而且还是在SAP HANA平台上,建议大家还是本着“慢就是快”的原则,不要急功近利,在没有准备好团队开发的架构时就匆忙开始功能的开发.匆忙功能开发就算了, ...
- 利用Veeam保护SAP HANA数据库
利用Veeam保护SAP HANA数据库 前言 针对越来越多的SAP HANA备份需求,我们Team翻译.整理.借鉴了Veeam 的SAP HANA 大神 Clemens Zerbe 和 Ali Sa ...
随机推荐
- java处理集合工具
public static <K, V> Map<K, V> parseListToMap(Collection<V> list, Function<V, K ...
- RateLimiter使用
1.maven依赖 <dependency> <groupId>com.google.guava</groupId> <artifactId>guava ...
- golang中一种不常见的switch语句写法
最近翻开源代码的时候看到了一种很有意思的switch用法,分享一下. 注意这里讨论的不是typed switch,也就是case语句后面是类型的那种. 直接看代码: func (s *systemd) ...
- 2022-12-07：删除重复的电子邮箱。删除重复数据后，id=3的数据被删除。请问sql语句如何写？ DROP TABLE IF EXISTS `person`； CREATE TABLE `per
2022-12-07:删除重复的电子邮箱.删除重复数据后,id=3的数据被删除.请问sql语句如何写? DROP TABLE IF EXISTS `person`; CREATE TABLE `per ...
- 一个线上全文索引BUG的排查：关于类阿拉件数字的分词与检索
说到全文检索的分词,多半讲到的是中(日韩)文分词,少有英文等拉丁文系语言,因为英语单词天然就是分词的. 但更少讲到阿拉伯数字.比如金额,手机号码,座机号码等等. 以下不是传统的从0开始针对mysql全 ...
- PHP反序列化常用魔术方法
PHP反序列化 php序列化(serialize):是将变量转换为可保存或传输的字符串的过程 php反序列化(unserialize):就是在适当的时候把这个字符串再转化成原来的变量使用 PHP反序列 ...
- AIGC持续火爆大模型争相推出，庞大市场造就算力供应模式演变
本图由AI生成 黄仁勋说的AI发展迎来iPhone时刻,对NVIDIA有什么影响? 文/王吉伟 近期的AIGC领域仍旧火爆异常. 但火的不只是AIGC应用,还有巨头之间的AI竞赛,以及接连不断上新的A ...
- cv学习总结（10.16-10.23） KNN
本周从周一开始学习cs231n的相关内容,看完了231n的课程介绍,背景介绍,图像分类的KNN和SVM算法,完成了作业中assignment1的KNN部分的代码(附件),思考总结了KNN的实现原理:即 ...
- Pandas 加载数据的方法和技巧
哈喽大家好,我是咸鱼 相信小伙伴们在学习 python 数据分析的过程中或多或少都会听说或者使用过 pandas pandas 是 python 的一个拓展库,常用于数据分析 今天咸鱼将介绍几个关于 ...
- 金三银四抢人季，HR 如何 3 招做到效率为王？
春招伊始,面对队伍庞大的校招人群,蜂拥而入的简历,HR 如何才能快速搞定呢?Bug君总结了一下过往招聘季的一些比较流行的环节: 通过线上宣讲,节省出行成本.时间,老板更认可了 现在大多数企业都会在直播 ...