DolphinScheduler3.1.7集成SAP HANA
源码地址：GitHub - apache/dolphinscheduler at 3.1.7-release
个人fork gitee地址：DolphinScheduler:Gitee)
后端代码更改项：
修改项1:DataSourceConstants.java
路径：dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/DataSourceConstants.java
public class DataSourceConstants {
public static final String COM_PRESTO_JDBC_DRIVER = "com.facebook.presto.jdbc.PrestoDriver";
public static final String COM_REDSHIFT_JDBC_DRIVER = "com.amazon.redshift.jdbc42.Driver";
public static final String COM_ATHENA_JDBC_DRIVER = "com.simba.athena.jdbc.Driver";
//add HANA driver message
public static final String COM_HANA_DB_JDBC_DRIVER = "com.sap.db.jdbc.Driver";
/**
* validation Query
*/
public static final String PRESTO_VALIDATION_QUERY = "select 1";
public static final String REDHIFT_VALIDATION_QUERY = "select 1";
public static final String ATHENA_VALIDATION_QUERY = "select 1";
// add HANA Connect check sql
public static final String HANA_VALIDATION_QUERY = "select 1";
/**
* jdbc url
*/
public static final String JDBC_PRESTO = "jdbc:presto://";
public static final String JDBC_REDSHIFT = "jdbc:redshift://";
public static final String JDBC_ATHENA = "jdbc:awsathena://";
// add HANA jdbc url modual
public static final String JDBC_HANA = "jdbc:sap://";
修改项2:前端映射枚举值
路径：dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java
H2(9, "h2"),
REDSHIFT(10,"redshift"),
ATHENA(11,"athena"),
HANA(12,"hana"),
;
修改项3:dolphinscheduler-datasource-hana模块
路径dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana，模块层级如下
pom文件
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-plugin</artifactId>
<version>3.1.8-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-datasource-hana</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-api</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.sap.cloud.db.jdbc</groupId>
<artifactId>ngdbc</artifactId>
<version>2.17.10</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<type>jar</type>
<scope>test</scope>
</dependency>
</dependencies>
</project>
HanaDataSourceChannel.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannel.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceClient;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceChannel implements DataSourceChannel {
@Override
public DataSourceClient createDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
return new HanaDataSourceClient(baseConnectionParam, dbType);
}
}
HanaDataSourceChannelFactory.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelFactory.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory;
import com.google.auto.service.AutoService;
@AutoService(DataSourceChannelFactory.class)
public class HanaDataSourceChannelFactory implements DataSourceChannelFactory {
@Override
public String getName() {
return "hana";
}
@Override
public DataSourceChannel create() {
return new HanaDataSourceChannel();
}
}
HanaDataSourceClient.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceClient.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceClient extends CommonDataSourceClient {
public HanaDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
super(baseConnectionParam, dbType);
}
}
HanaConnectionParam.java
路径:dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaConnectionParam.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
public class HanaConnectionParam extends BaseConnectionParam {
@Override
public String toString() {
return "HanaConnectionParam{"
+ "user='" + user + '\''
+ ", password='" + password + '\''
+ ", address='" + address + '\''
+ ", database='" + database + '\''
+ ", jdbcUrl='" + jdbcUrl + '\''
+ ", driverLocation='" + driverLocation + '\''
+ ", driverClassName='" + driverClassName + '\''
+ ", validationQuery='" + validationQuery + '\''
+ ", other='" + other + '\''
+ '}';
}
}
HanaDataSourceParamDTO.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceParamDTO.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class HanaDataSourceParamDTO extends BaseDataSourceParamDTO {
@Override
public String toString() {
return "HanaDataSourceParamDTO{"
+ "name='" + name + '\''
+ ", note='" + note + '\''
+ ", host='" + host + '\''
+ ", port=" + port
+ ", database='" + database + '\''
+ ", userName='" + userName + '\''
+ ", password='" + password + '\''
+ ", other='" + other + '\''
+ '}';
}
@Override
public DbType getType() {
return DbType.HANA;
}
}
HanaDataSourceProcessor.java
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/main/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceProcessor.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.commons.lang3.StringUtils;
import org.apache.dolphinscheduler.common.constants.Constants;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.apache.commons.collections4.MapUtils;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.auto.service.AutoService;
@AutoService(DataSourceProcessor.class)
public class HanaDataSourceProcessor extends AbstractDataSourceProcessor {
private final Logger logger = LoggerFactory.getLogger(HanaDataSourceProcessor.class);
private static final String APPEND_PARAMS = "reconnect=true";
@Override
public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
return JSONUtils.parseObject(paramJson, HanaDataSourceParamDTO.class);
}
@Override
public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
HanaConnectionParam connectionParams = (HanaConnectionParam) createConnectionParams(connectionJson);
HanaDataSourceParamDTO hanaDatasourceParamDTO = new HanaDataSourceParamDTO();
hanaDatasourceParamDTO.setUserName(connectionParams.getUser());
hanaDatasourceParamDTO.setDatabase(connectionParams.getDatabase());
String address = connectionParams.getAddress();
String[] hostSeperator = address.split(Constants.DOUBLE_SLASH);
String[] hostPortArray = hostSeperator[hostSeperator.length - 1].split(Constants.COMMA);
hanaDatasourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
hanaDatasourceParamDTO.setHost(hostPortArray[0].split(Constants.COLON)[0]);
return hanaDatasourceParamDTO;
}
@Override
public BaseConnectionParam createConnectionParams(BaseDataSourceParamDTO dataSourceParam) {
HanaDataSourceParamDTO hanaDatasourceParam = (HanaDataSourceParamDTO) dataSourceParam;
String address = String.format("%s%s:%s", DataSourceConstants.JDBC_HANA, hanaDatasourceParam.getHost(),
hanaDatasourceParam.getPort());
String jdbcUrl = String.format("%s?currentschema=%s", address, hanaDatasourceParam.getDatabase());
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl(jdbcUrl);
hanaConnectionParam.setValidationQuery("select 1 from DUMMY");
hanaConnectionParam.setDatabase(hanaDatasourceParam.getDatabase());
hanaConnectionParam.setAddress(address);
hanaConnectionParam.setUser(hanaDatasourceParam.getUserName());
hanaConnectionParam.setPassword(PasswordUtils.encodePassword(hanaDatasourceParam.getPassword()));
hanaConnectionParam.setDriverClassName(getDatasourceDriver());
hanaConnectionParam.setOther(transformOther(hanaDatasourceParam.getOther()));
return hanaConnectionParam;
}
@Override
public ConnectionParam createConnectionParams(String connectionJson) {
return JSONUtils.parseObject(connectionJson, HanaConnectionParam.class);
}
@Override
public String getDatasourceDriver() {
return DataSourceConstants.COM_HANA_DB_JDBC_DRIVER;
}
@Override
public String getValidationQuery() {
return DataSourceConstants.COM_HANA_DB_JDBC_DRIVER;
}
@Override
public String getJdbcUrl(ConnectionParam connectionParam) {
HanaConnectionParam hanaConnectionParam = (HanaConnectionParam) connectionParam;
String jdbcUrl = hanaConnectionParam.getJdbcUrl();
// get customize connection message,parse it to map
if (MapUtils.isNotEmpty(parseOther(hanaConnectionParam.getOther()))) {
return String.format("%s?%s&%s", jdbcUrl, hanaConnectionParam.getOther(), APPEND_PARAMS);
}
return String.format("%s&%s", jdbcUrl, APPEND_PARAMS);
}
@Override
public Connection getConnection(ConnectionParam connectionParam) throws ClassNotFoundException, SQLException {
HanaConnectionParam hanaConnectionParam = (HanaConnectionParam) connectionParam;
Class.forName(getDatasourceDriver());
String user = hanaConnectionParam.getUser();
String password = PasswordUtils.decodePassword(hanaConnectionParam.getPassword());
return DriverManager.getConnection(getJdbcUrl(connectionParam), user, password);
}
@Override
public DbType getDbType() {
return DbType.HANA;
}
@Override
public DataSourceProcessor create() {
logger.info("create hana datasource processor success");
return new HanaDataSourceProcessor();
}
/**
* transform map to string with &
* @param otherMap connection map
* @return return the connection string format
*/
private String transformOther(Map<String, String> otherMap) {
if (MapUtils.isEmpty(otherMap)) {
return null;
}
List<String> list = new ArrayList<>();
otherMap.forEach((key, value) -> list.add(String.format("%s=%s", key, value)));
return String.join("&", list);
}
/**
* transform string to map
* @param other connection string
* @return return connection map format
*/
private Map<String, String> parseOther(String other) {
if (StringUtils.isEmpty(other)) {
return null;
}
Map<String, String> otherMap = new LinkedHashMap<>();
for (String config : other.split("&")) {
otherMap.put(config.split("=")[0], config.split("=")[1]);
}
return otherMap;
}
}
HanaDataSourceProcessorTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/param/HanaDataSourceProcessorTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.param;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.HashMap;
import java.util.Map;
@ExtendWith(MockitoExtension.class)
public class HanaDataSourceProcessorTest {
private HanaDataSourceProcessor hanaDataSourceProcessor = new HanaDataSourceProcessor();
@Test
public void testCreateConnectionParams() {
Map<String, String> props = new HashMap<>();
HanaDataSourceParamDTO mysqlDatasourceParamDTO = new HanaDataSourceParamDTO();
mysqlDatasourceParamDTO.setUserName("root");
mysqlDatasourceParamDTO.setPassword("123456");
mysqlDatasourceParamDTO.setHost("localhost");
mysqlDatasourceParamDTO.setPort(30015);
mysqlDatasourceParamDTO.setDatabase("default");
mysqlDatasourceParamDTO.setOther(props);
try (MockedStatic<PasswordUtils> mockedPasswordUtils = Mockito.mockStatic(PasswordUtils.class)) {
Mockito.when(PasswordUtils.encodePassword(Mockito.anyString())).thenReturn("test");
HanaConnectionParam connectionParams = (HanaConnectionParam) hanaDataSourceProcessor
.createConnectionParams(mysqlDatasourceParamDTO);
Assertions.assertEquals("jdbc:sap://localhost:30015", connectionParams.getAddress());
Assertions.assertEquals("jdbc:sap://localhost:30015?currentschema=default", connectionParams.getJdbcUrl());
}
}
@Test
public void testCreateConnectionParams2() {
String connectionJson = "{\"user\":\"root\",\"password\":\"123456\",\"address\":\"jdbc:sap://localhost:30015\""
+ ",\"database\":\"default\",\"jdbcUrl\":\"jdbc:sap://localhost:30015?currentschema=default\"}";
HanaConnectionParam connectionParams = (HanaConnectionParam) hanaDataSourceProcessor
.createConnectionParams(connectionJson);
Assertions.assertNotNull(connectionJson);
Assertions.assertEquals("root", connectionParams.getUser());
}
@Test
public void testGetDatasourceDriver() {
Assertions.assertEquals(DataSourceConstants.COM_HANA_DB_JDBC_DRIVER,
hanaDataSourceProcessor.getDatasourceDriver());
}
@Test
public void testGetJdbcUrl() {
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015?currentschema=default");
Assertions.assertEquals(
"jdbc:sap://localhost:30015?currentschema=default&reconnect=true",
hanaDataSourceProcessor.getJdbcUrl(hanaConnectionParam));
}
@Test
public void testGetDbType() {
Assertions.assertEquals(DbType.HANA, hanaDataSourceProcessor.getDbType());
}
@Test
public void testGetValidationQuery() {
Assertions.assertEquals(DataSourceConstants.HANA_VALIDATION_QUERY,
hanaDataSourceProcessor.getValidationQuery());
}
@Test
public void testGetDatasourceUniqueId() {
HanaConnectionParam mysqlConnectionParam = new HanaConnectionParam();
mysqlConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015?currentschema=default");
mysqlConnectionParam.setUser("root");
mysqlConnectionParam.setPassword("123456");
try (MockedStatic<PasswordUtils> mockedPasswordUtils = Mockito.mockStatic(PasswordUtils.class)) {
Mockito.when(PasswordUtils.encodePassword(Mockito.anyString())).thenReturn("123456");
Assertions.assertEquals("hana@root@123456@jdbc:sap://localhost:30015?currentschema=default",
hanaDataSourceProcessor.getDatasourceUniqueId(mysqlConnectionParam, DbType.HANA));
}
}
}
JDBCDataSourceProviderTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/provider/JDBCDataSourceProviderTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.provider;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class JDBCDataSourceProviderTest {
@Test
public void testCreateJdbcDataSource() {
try (
MockedStatic<JDBCDataSourceProvider> mockedJDBCDataSourceProvider =
Mockito.mockStatic(JDBCDataSourceProvider.class)) {
HikariDataSource dataSource = Mockito.mock(HikariDataSource.class);
mockedJDBCDataSourceProvider
.when(() -> JDBCDataSourceProvider.createJdbcDataSource(Mockito.any(), Mockito.any()))
.thenReturn(dataSource);
Assertions.assertNotNull(
JDBCDataSourceProvider.createJdbcDataSource(new HanaConnectionParam(), DbType.HANA));
}
}
@Test
public void testCreateOneSessionJdbcDataSource() {
try (
MockedStatic<JDBCDataSourceProvider> mockedJDBCDataSourceProvider =
Mockito.mockStatic(JDBCDataSourceProvider.class)) {
HikariDataSource dataSource = Mockito.mock(HikariDataSource.class);
mockedJDBCDataSourceProvider
.when(() -> JDBCDataSourceProvider.createOneSessionJdbcDataSource(Mockito.any(), Mockito.any()))
.thenReturn(dataSource);
Assertions.assertNotNull(
JDBCDataSourceProvider.createOneSessionJdbcDataSource(new HanaConnectionParam(), DbType.HANA));
}
}
}
DataSourceUtilsTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/utils/DataSourceUtilsTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana.utils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.PropertyUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.sql.Connection;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ExecutionException;
@ExtendWith(MockitoExtension.class)
public class DataSourceUtilsTest {
@Test
public void testCheckDatasourceParam() {
HanaDataSourceParamDTO hanaDataSourceParamDTO = new HanaDataSourceParamDTO();
hanaDataSourceParamDTO.setHost("localhost");
hanaDataSourceParamDTO.setDatabase("default");
Map<String, String> other = new HashMap<>();
other.put("reconnect", "true");
hanaDataSourceParamDTO.setOther(other);
DataSourceUtils.checkDatasourceParam(hanaDataSourceParamDTO);
Assertions.assertTrue(true);
}
@Test
public void testBuildConnectionParams() {
HanaDataSourceParamDTO hanaDataSourceParamDTO = new HanaDataSourceParamDTO();
hanaDataSourceParamDTO.setHost("localhost");
hanaDataSourceParamDTO.setDatabase("default");
hanaDataSourceParamDTO.setUserName("root");
hanaDataSourceParamDTO.setPort(30015);
hanaDataSourceParamDTO.setPassword("123456");
try (
MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
mockedStaticPasswordUtils.when(() -> PasswordUtils.encodePassword(Mockito.anyString()))
.thenReturn("123456");
mockedStaticCommonUtils.when(CommonUtils::getKerberosStartupState).thenReturn(false);
ConnectionParam connectionParam = DataSourceUtils.buildConnectionParams(hanaDataSourceParamDTO);
Assertions.assertNotNull(connectionParam);
}
}
@Test
public void testBuildConnectionParams2() {
HanaDataSourceParamDTO hanaDatasourceParamDTO = new HanaDataSourceParamDTO();
hanaDatasourceParamDTO.setHost("localhost");
hanaDatasourceParamDTO.setDatabase("default");
hanaDatasourceParamDTO.setUserName("root");
hanaDatasourceParamDTO.setPort(30015);
hanaDatasourceParamDTO.setPassword("123456");
ConnectionParam connectionParam =
DataSourceUtils.buildConnectionParams(DbType.HANA, JSONUtils.toJsonString(hanaDatasourceParamDTO));
Assertions.assertNotNull(connectionParam);
}
@Test
public void testGetConnection() throws ExecutionException {
try (
MockedStatic<PropertyUtils> mockedStaticPropertyUtils = Mockito.mockStatic(PropertyUtils.class);
MockedStatic<DataSourceClientProvider> mockedStaticDataSourceClientProvider =
Mockito.mockStatic(DataSourceClientProvider.class)) {
mockedStaticPropertyUtils.when(() -> PropertyUtils.getLong("kerberos.expire.time", 24L)).thenReturn(24L);
DataSourceClientProvider clientProvider = Mockito.mock(DataSourceClientProvider.class);
mockedStaticDataSourceClientProvider.when(DataSourceClientProvider::getInstance).thenReturn(clientProvider);
Connection connection = Mockito.mock(Connection.class);
Mockito.when(clientProvider.getConnection(Mockito.any(), Mockito.any())).thenReturn(connection);
HanaConnectionParam connectionParam = new HanaConnectionParam();
connectionParam.setUser("root");
connectionParam.setPassword("123456");
connection = DataSourceClientProvider.getInstance().getConnection(DbType.HANA, connectionParam);
Assertions.assertNotNull(connection);
}
}
@Test
public void testGetJdbcUrl() {
HanaConnectionParam hanaConnectionParam = new HanaConnectionParam();
hanaConnectionParam.setJdbcUrl("jdbc:sap://localhost:30015");
String jdbcUrl = DataSourceUtils.getJdbcUrl(DbType.HANA, hanaConnectionParam);
Assertions.assertEquals(
"jdbc:sap://localhost:30015?reconnect=true",
jdbcUrl);
}
@Test
public void testBuildDatasourceParamDTO() {
HanaConnectionParam connectionParam = new HanaConnectionParam();
connectionParam.setJdbcUrl(
"jdbc:sap://localhost:30015?reconnect=true");
connectionParam.setAddress("jdbc:mysql://localhost:30015");
connectionParam.setUser("root");
connectionParam.setPassword("123456");
Assertions.assertNotNull(
DataSourceUtils.buildDatasourceParamDTO(DbType.HANA, JSONUtils.toJsonString(connectionParam)));
}
@Test
public void testGetDatasourceProcessor() {
Assertions.assertNotNull(DataSourceUtils.getDatasourceProcessor(DbType.HANA));
}
@Test
public void testGetDatasourceProcessorError() {
Assertions.assertThrows(Exception.class, () -> {
DataSourceUtils.getDatasourceProcessor(null);
});
}
}
HanaDataSourceChannelFactoryTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelFactoryTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
public class HanaDataSourceChannelFactoryTest {
@Test
public void testCreate() {
HanaDataSourceChannelFactory sourceChannelFactory = new HanaDataSourceChannelFactory();
DataSourceChannel dataSourceChannel = sourceChannelFactory.create();
Assertions.assertNotNull(dataSourceChannel);
}
}
HanaDataSourceChannelTest.java
路径：dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-hana/src/test/java/org/apache/dolphinscheduler/plugin/datasource/hana/HanaDataSourceChannelTest.java
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.hana;
import org.apache.dolphinscheduler.plugin.datasource.hana.param.HanaConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class HanaDataSourceChannelTest {
@Test
public void testCreateDataSourceClient() {
HanaDataSourceChannel sourceChannel = Mockito.mock(HanaDataSourceChannel.class);
HanaDataSourceClient dataSourceClient = Mockito.mock(HanaDataSourceClient.class);
Mockito.when(sourceChannel.createDataSourceClient(Mockito.any(), Mockito.any())).thenReturn(dataSourceClient);
Assertions.assertNotNull(sourceChannel.createDataSourceClient(new HanaConnectionParam(), DbType.HANA));
}
}
修改项4:打包相关依赖
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-all/pom.xml
dependencies中新增hana模块依赖
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-hana</artifactId>
<version>${project.version}</version>
</dependency>
dolphinscheduler-datasource-plugin/pom.xml
modules中新增hana模块
<module>dolphinscheduler-datasource-hana</module>
前端UI代码变更
修改项1:types.ts
路径：dolphinscheduler-ui/src/service/modules/data-source/types.ts 新增hana选择
| 'PRESTO'
| 'REDSHIFT'
| 'ATHENA'
| 'HANA'
interface IDataSource {
id?: number
修改项6:
修改项2:use-form.ts
路径：dolphinscheduler-ui/src/views/datasource/list/use-form.ts 新增hana默认端口
value: 'ATHENA',
label: 'ATHENA',
defaultPort: 0
},
HANA: {
value: 'HANA',
label: 'HANA',
defaultPort: 30015
}
}
修改项3:use-datasource.ts
路径：dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-datasource.ts
id: 10,
code: 'ATHENA',
disabled: false
},
{
id: 11,
code: 'HANA',
disabled: false
}
]
编译打包
dolphinscheduler-dist/target/路径下生成tar.gz包
mvn -U clean package -Prelease -Dmaven.test.skip=true
DolphinScheduler3.1.7集成SAP HANA的更多相关文章
- SAP HANA专题分析目录
针对HANA的关键技术领域, 做深度解析. 1. HANA开发规范 HANA 各种对象的应用解析.版本管理,开发规范. 2. HANA系统管理 用户.系统权限.数据权限的深度解析. HANA系统配置. ...
- [转载]大道至简！！！从SAP HANA作为SAP加速器的方式，看ERP on HANA的春天
I AM A ABAPER! 科技的进步,一定会使一些东西变得越来越精简! 大道至简!!! 文章很好!!!!!!!!!!! -------------------------------------- ...
- SAP HANA中的SLT简介
在以SAP系统作为主要ERP的企业中,不同系统之间的数据库数据同步是个重要的工作.对于这种需求,除了开发ABAP接口之外,也有高效的工具可用.SLT就是其中之一. SLT是SAP的第一个ETL(Ext ...
- 【公众号系列】SAP HANA 平台的优势
公众号:SAP Technical 本文作者:matinal 原文出处:http://www.cnblogs.com/SAPmatinal/ 原文链接:[公众号系列]SAP HANA 平台的优势 ...
- 【公众号系列】SAP HANA和区块链
公众号:SAP Technical 本文作者:matinal 原文出处:http://www.cnblogs.com/SAPmatinal/ 原文链接:[公众号系列]SAP HANA和区块链 写在 ...
- 如何在云端部署SAP HANA实战, Azure 上的 SAP HANA（大型实例）概述和体系结构
什么是 Azure 上的 SAP HANA(大型实例)? Azure 上的 SAP HANA(大型实例)是一种针对 Azure 的独特解决方案. 除了提供 Azure 虚拟机以用于部署和运行 SAP ...
- sap hana 数据库 EBS
SAP实时数据平台详解 ************************************************************ EBS是Oracle 公司对原有应用产品整合后的一个产 ...
- SAP HANA 开发模式 - 基于SAP HANA平台的多团队产品研发
“基本”开发模式 Windows: Unix/Linux: 在基本模式下我们可以通过regi来进行激活我们的object.Regi是一个类git功能的,方便和HANA repository交互的一个命 ...
- 基于SAP HANA平台的多团队产品研发
工欲善其事必先利其器.要提高多团队的开发效率,而且还是在SAP HANA平台上,建议大家还是本着“慢就是快”的原则,不要急功近利,在没有准备好团队开发的架构时就匆忙开始功能的开发.匆忙功能开发就算了, ...
- 利用Veeam保护SAP HANA数据库
利用Veeam保护SAP HANA数据库 前言 针对越来越多的SAP HANA备份需求,我们Team翻译.整理.借鉴了Veeam 的SAP HANA 大神 Clemens Zerbe 和 Ali Sa ...
随机推荐
- MySQL中binlog备份脚本分享
关于MySQL的二进制日志(binlog),我们都知道二进制日志(binlog)非常重要,尤其当你需要point to point灾难恢复的时侯,所以我们要对其进行备份.关于二进制日志(binlog) ...
- 如何在 .NET Core WebApi 中处理 MultipartFormDataContent 中的文件
在上一篇文章(如何在 .NET Core WebApi 中处理 MultipartFormDataContent)中,我们有描述过如何以最简单的方式在 .NET Core WebApi 中处理 Mul ...
- Jquery实现复选框的选中和取消
复选框的选中与取消 我在网上看了好多关于这个问题的解答,好多都是一两个按钮的触发事件,有的甚至没有任何效果,经过自己的调试发现这个方法好用一点: 首先我在页面上添加了这样一个复选框 我的复选框是动态加 ...
- P1980 [NOIP2013 普及组] 计数问题
题目链接:https://www.luogu.com.cn/problem/P1980 术语 以下的英文术语均可以翻译为数字. digit: 一个数字字符,十进制就是 0-9 之间的一个字符: num ...
- 「学习笔记」AC 自动机
「学习笔记」AC 自动机 点击查看目录 目录 「学习笔记」AC 自动机 算法 问题 思路 代码 例题 Keywords Search 玄武密码 单词 病毒 最短母串 文本生成器 背单词 密码 禁忌 前 ...
- 自制ASP.NET 本地授权文件
asp.net登录时验证本地ini文件是否正确,主要步骤. 1.导入myini.DLL文件. 下载地址:http://yunpan.cn/cKw9kHJUk9Ui8 提取码 6631 2.添加引用 ...
- 2022-06-16：给定一个数组arr，含有n个数字，都是非负数， 给定一个正数k， 返回所有子序列中，累加和最小的前k个子序列累加和。 假设K不大，怎么算最快？ 来自亚马逊。
2022-06-16:给定一个数组arr,含有n个数字,都是非负数, 给定一个正数k, 返回所有子序列中,累加和最小的前k个子序列累加和. 假设K不大,怎么算最快? 来自亚马逊. 答案2022-06- ...
- 2022-02-23：如何搭建k8s单机环境（用k3s），并且搭建dashboard？
2022-02-23:如何搭建k8s单机环境(用k3s),并且搭建dashboard? 答案2022-02-03: 使用场景:个人电脑.需要安装虚拟机,操作系统是centos. 个人电脑上测试,不需要 ...
- 2021-06-27：给定一个正数数组arr，代表若干人的体重。再给定一个正数limit，表示所有船共同拥有的载重量。每艘船最多坐两人，且不能超过载重，想让所有的人同时过河，并且用最好的分配方法让船尽
2021-06-27:给定一个正数数组arr,代表若干人的体重.再给定一个正数limit,表示所有船共同拥有的载重量.每艘船最多坐两人,且不能超过载重,想让所有的人同时过河,并且用最好的分配方法让船尽 ...
- 基于Expression Lambda表达式树的通用复杂动态查询构建器——《构思篇一》
在上一篇中构思了把查询子句描述出来的数据结构,那么能否用代码将其表达出来,如何表达呢? 再次回顾考察,看下面的查询子句: Id>1 and Id<10 如上所示,有两个独立的条件分别为Id ...