elasticsearch 之编译过程

https://github.com/elastic/elasticsearch/blob/master/CONTRIBUTING.md

不同的版本需要指定JDK

可以下载openJDK版本到服务器上，下载过程比较慢

https://jdk.java.net/archive/

export JAVA_HOME=/xxx/yyy/jdk10.xxx

/gradle assemble

执行命令后

生产的结果在distributions目录下, 启动跟自己下载的版本一样，一样会遇到权限问题，chown xxx -R 目录解决一下

.

├── archives

│   ├── build

│   │   ├── logs-hack

│   │   │   └── logs

│   │   └── plugins-hack

│   │       └── plugins

│   ├── build.gradle

│   ├── integ-test-zip

│   │   ├── build

│   │   │   ├── distributions

│   │   │   │   ├── elasticsearch-6.4.4-SNAPSHOT.pom

│   │   │   │   └── elasticsearch-6.4.4-SNAPSHOT.zip

│   │   │   ├── heapdump

│   │   │   └── publications

│   │   │       ├── nebula

│   │   │       │   └── pom-default.xml

│   │   │       └── nebulaRealPom

│   │   │           └── pom-default.xml

│   │   ├── build.gradle

│   │   └── src

│   │       └── test

│   │           └── java

│   │               └── org

│   │                   └── elasticsearch

│   │                       └── test

│   │                           └── rest

│   │                               ├── CreatedLocationHeaderIT.java

│   │                               ├── DefaultShardsIT.java

│   │                               ├── IntegTestZipClientYamlTestSuiteIT.java

│   │                               ├── NodeRestUsageIT.java

│   │                               ├── RequestsWithoutContentIT.java

│   │                               └── WaitForRefreshAndCloseTests.java

│   ├── oss-tar

│   │   ├── build

│   │   │   └── distributions

│   │   │       └── elasticsearch-oss-6.4.4-SNAPSHOT.tar.gz

│   │   └── build.gradle

│   ├── oss-zip

│   │   ├── build

│   │   │   ├── distributions

│   │   │   │   ├── elasticsearch-oss-6.4.4-SNAPSHOT.pom

│   │   │   │   └── elasticsearch-oss-6.4.4-SNAPSHOT.zip

│   │   │   └── publications

│   │   │       ├── nebula

│   │   │       │   └── pom-default.xml

│   │   │       └── nebulaRealPom

│   │   │           └── pom-default.xml

│   │   └── build.gradle

│   ├── tar

│   │   ├── build

│   │   │   └── distributions

│   │   │       └── elasticsearch-6.4.4-SNAPSHOT.tar.gz

│   │   └── build.gradle

│   └── zip

│       ├── build

│       │   ├── distributions

│       │   │   ├── elasticsearch-6.4.4-SNAPSHOT.pom

│       │   │   └── elasticsearch-6.4.4-SNAPSHOT.zip

│       │   └── publications

│       │       ├── nebula

│       │       │   └── pom-default.xml

│       │       └── nebulaRealPom

│       │           └── pom-default.xml

│       └── build.gradle

├── build

│   ├── cluster

│   │   ├── run\ node0

│   │   │   ├── cwd

│   │   │   └── elasticsearch-6.4.4-SNAPSHOT

│   │   │       ├── bin

│   │   │       │   ├── elasticsearch

│   │   │       │   ├── elasticsearch.bat

│   │   │       │   ├── elasticsearch-certgen

│   │   │       │   ├── elasticsearch-certgen.bat

│   │   │       │   ├── elasticsearch-certutil

│   │   │       │   ├── elasticsearch-certutil.bat

│   │   │       │   ├── elasticsearch-cli

│   │   │       │   ├── elasticsearch-cli.bat

es 的文档告诉大家，请不要把贡献代码作为课程的一部分，希望大家是自愿的贡献代码。同时在之前他们也说明不要为了重构而重构，因为测试，review要花费巨大的资源。如果要提交新的特性，请先开一个issue，讨论是否有必要才开始动手写代码。

Contributing as part of a class

In general Elasticsearch is happy to accept contributions that were created as part of a class but strongly advise against making the contribution as part of the class. So if you have code you wrote for a class feel free to submit it.

Please, please, please do not assign contributing to Elasticsearch as part of a class. If you really want to assign writing code for Elasticsearch as an assignment then the code contributions should be made to your private clone and opening PRs against the primary Elasticsearch clone must be optional, fully voluntary, not for a grade, and without any deadlines.

Because:

• While the code review process is likely very educational, it can take wildly varying amounts of time depending on who is available, where the change is, and how deep the change is. There is no way to predict how long it will take unless we rush.
• We do not rush reviews without a very, very good reason. Class deadlines aren't a good enough reason for us to rush reviews.
• We deeply discourage opening a PR you don't intend to work through the entire code review process because it wastes our time.
• We don't have the capacity to absorb an entire class full of new contributors, especially when they are unlikely to become long time contributors.

Finally, we require that you run ./gradlew check before submitting a non-documentation contribution. This is mentioned above, but it is worth repeating in this section because it has come up in this context.

FAILURE: Build failed with an exception.

* What went wrong:

Could not determine the dependencies of task ':x-pack:qa:full-cluster-restart:with-system-key:v5.6.15-SNAPSHOT#oldClusterTestCluster#node1.copyBwcPlugins'.

> Could not resolve all task dependencies for configuration ':x-pack:qa:full-cluster-restart:with-system-key:v5.6.15-SNAPSHOT#oldClusterTestCluster_elasticsearchBwcPlugins'.

> Could not resolve org.elasticsearch.plugin:x-pack:5.6.15-SNAPSHOT.

Required by:

project :x-pack:qa:full-cluster-restart:with-system-key

> Could not resolve org.elasticsearch.plugin:x-pack:5.6.15-SNAPSHOT.

> Unable to load Maven meta-data from https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml.

> Could not get resource 'https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml'.

> Could not GET 'https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml'.

> sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

　　　　export JAVA_HOME=/usr/share/jdk1.5.0_05
　　　　export PATH=$JAVA_HOME/bin:$PATH
　　　　export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

　　　　javac InstallCert.java

　　　　java InstallCert

　　　 执行过程中，输入1，在当前目录生成一个文件　　
      手动复制到目录：$JAVA_HOME/lib/security/cacerts

/*
 * Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Sun Microsystems nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

import java.io.*;
import java.net.URL;

import java.security.*;
import java.security.cert.*;

import javax.net.ssl.*;

public class InstallCert {

    public static void main(String[] args) throws Exception {
    String host = "10.0.31.32"; //输入服务端地址
    int port = 8443; //输入服务端端口 一般默认443

    String p = "changeit"; //一般默认changeit
    char[] passphrase = p.toCharArray();

    File file = new File("jssecacerts");
    if (file.isFile() == false) {
        char SEP = File.separatorChar;
        File dir = new File(System.getProperty("java.home") + SEP
            + "lib" + SEP + "security");
        file = new File(dir, "jssecacerts");
        if (file.isFile() == false) {
        file = new File(dir, "cacerts");
        }
    }
    System.out.println("Loading KeyStore " + file + "...");
    InputStream in = new FileInputStream(file);
    KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
    ks.load(in, passphrase);
    in.close();

    SSLContext context = SSLContext.getInstance("TLS");
    TrustManagerFactory tmf =
        TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
    tmf.init(ks);
    X509TrustManager defaultTrustManager = (X509TrustManager)tmf.getTrustManagers()[0];
    SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);
    context.init(null, new TrustManager[] {tm}, null);
    SSLSocketFactory factory = context.getSocketFactory();

    System.out.println("Opening connection to " + host + ":" + port + "...");
    SSLSocket socket = (SSLSocket)factory.createSocket(host, port);
    socket.setSoTimeout(10000);
    try {
        System.out.println("Starting SSL handshake...");
        socket.startHandshake();
        socket.close();
        System.out.println();
        System.out.println("No errors, certificate is already trusted");
    } catch (SSLException e) {
        System.out.println();
        e.printStackTrace(System.out);
    }

    X509Certificate[] chain = tm.chain;
    if (chain == null) {
        System.out.println("Could not obtain server certificate chain");
        return;
    }

    BufferedReader reader =
        new BufferedReader(new InputStreamReader(System.in));

    System.out.println();
    System.out.println("Server sent " + chain.length + " certificate(s):");
    System.out.println();
    MessageDigest sha1 = MessageDigest.getInstance("SHA1");
    MessageDigest md5 = MessageDigest.getInstance("MD5");
    for (int i = 0; i < chain.length; i++) {
        X509Certificate cert = chain[i];
        System.out.println
            (" " + (i + 1) + " Subject " + cert.getSubjectDN());
        System.out.println("   Issuer  " + cert.getIssuerDN());
        sha1.update(cert.getEncoded());
        System.out.println("   sha1    " + toHexString(sha1.digest()));
        md5.update(cert.getEncoded());
        System.out.println("   md5     " + toHexString(md5.digest()));
        System.out.println();
    }

    System.out.println("Enter certificate to add to trusted keystore or 'q' to quit: [1]");
    String line = reader.readLine().trim();
    int k;
    try {
        k = (line.length() == 0) ? 0 : Integer.parseInt(line) - 1;
    } catch (NumberFormatException e) {
        System.out.println("KeyStore not changed");
        return;
    }

    X509Certificate cert = chain[k];
    String alias = host + "-" + (k + 1);
    ks.setCertificateEntry(alias, cert);

    OutputStream out = new FileOutputStream("jssecacerts");
    ks.store(out, passphrase);
    out.close();

    System.out.println();
    System.out.println(cert);
    System.out.println();
    System.out.println
        ("Added certificate to keystore 'jssecacerts' using alias '"
        + alias + "'");
    }

    private static final char[] HEXDIGITS = "0123456789abcdef".toCharArray();

    private static String toHexString(byte[] bytes) {
    StringBuilder sb = new StringBuilder(bytes.length * 3);
    for (int b : bytes) {
        b &= 0xff;
        sb.append(HEXDIGITS[b >> 4]);
        sb.append(HEXDIGITS[b & 15]);
        sb.append(' ');
    }
    return sb.toString();
    }

    private static class SavingTrustManager implements X509TrustManager {

    private final X509TrustManager tm;
    private X509Certificate[] chain;

    SavingTrustManager(X509TrustManager tm) {
        this.tm = tm;
    }

    public X509Certificate[] getAcceptedIssuers() {
        throw new UnsupportedOperationException();
    }

    public void checkClientTrusted(X509Certificate[] chain, String authType)
        throws CertificateException {
        throw new UnsupportedOperationException();
    }

    public void checkServerTrusted(X509Certificate[] chain, String authType)
        throws CertificateException {
        this.chain = chain;
        tm.checkServerTrusted(chain, authType);
    }
    }

}
---------------------

6.4 分支能编译，能运行起来，但是单元测试老是提示找不到 5.X的一个包，估计是拿来做兼容测试的。

=======================================
Elasticsearch Build Hamster says Hello!
=======================================
Gradle Version : 4.9
OS Info : Linux 3.10.0-862.el7.x86_64 (amd64)
JDK Version : Oracle Corporation 10.0.2 [OpenJDK 64-Bit Server VM 10.0.2+13]
JAVA_HOME : /data/dev/jdk-10.0.2
Random Testing Seed : 9DAA18E0EFB8B8EE

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':x-pack:qa:full-cluster-restart:with-system-key:v5.6.15-SNAPSHOT#oldClusterTestCluster#node1.copyBwcPlugins'.
> Could not resolve all task dependencies for configuration ':x-pack:qa:full-cluster-restart:with-system-key:v5.6.15-SNAPSHOT#oldClusterTestCluster_elasticsearchBwcPlugins'.
> Could not find org.elasticsearch.plugin:x-pack:5.6.15-SNAPSHOT.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml
- https://repo.maven.apache.org/maven2/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.pom
- https://repo.maven.apache.org/maven2/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.zip
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.pom
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.zip
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.pom
- https://artifacts.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.zip
- https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/maven-metadata.xml
- https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.pom
- https://snapshots.elastic.co/maven/org/elasticsearch/plugin/x-pack/5.6.15-SNAPSHOT/x-pack-5.6.15-SNAPSHOT.zip
Required by:
project :x-pack:qa:full-cluster-restart:with-system-key

git check master

重新执行，发现中文的提示，导致测试用例不过。

Suite: Test class org.elasticsearch.plugins.PluginsServiceTests

1> [2019-04-30T09:42:28,120][INFO ][o.e.p.PluginsServiceTests] [testDesktopServicesStoreFiles] before test

1> [2019-04-30T09:42:28,337][INFO ][o.e.p.PluginsServiceTests] [testDesktopServicesStoreFiles] after test

2> REPRODUCE WITH: ./gradlew :server:test --tests "org.elasticsearch.plugins.PluginsServiceTests.testDesktopServicesStoreFiles" -Dtests.seed=9EA3ED2F64F1D310 -Dtests.security.manager=true -Dtests.locale=hr-BA -Dtests.timezone=EST5EDT -Dcompiler.java=12 -Druntime.java=12

2> java.lang.AssertionError:

Expected: with toString() a string containing "Not a directory"

but: toString() was "java.nio.file.FileSystemException: /data/dev/elasticsearch/server/build/testrun/test/temp/org.elasticsearch.plugins.PluginsServiceTests_9EA3ED2F64F1D310-002/tempDir-002/plugins/.DS_Store/plugin-descriptor.properties: 不是目录"

at __randomizedtesting.SeedInfo.seed([9EA3ED2F64F1D310:C91D147EEA92C17A]:0)

at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)

at org.junit.Assert.assertThat(Assert.java:956)

at org.junit.Assert.assertThat(Assert.java:923)

启动过程如果提示max file open too low可以执行这个
sudo sh -c "ulimit -n 65536 && exec su $LOGNAME"