通往集群的大门

集群由什么用?

高可用

　　高可用(High Availability)是分布式系统架构设计中必须考虑的因素之一，它通常是指，通过设计减少系统不能提供服务的时间。如果系统每运行100个时间单位，会有1个时间单位无法提供服务，我们说系统的可用性是99%。

负载均衡

　　将流量均衡的分布在不同的节点上，每个节点都可以处理一部分负载，并且可以在节点之间动态分配负载，以实现平衡。

高性能

　　将流量分发到不同机器，充分利用多机器多CPU，从串行计算到并行计算提供系统性能。

ES集群的基本核心概念

Cluster集群

　　一个ElasticSearch集群由一个或多个节点(Node)组成，每个集群都有一个共同的集群名称作为标识。

Node节点

　　一个ElasticSearch实例即一个Node，一台机器可以有多个实例，正常使用下每个实例应该会部署在不同机器上。ElasticSearch的配置文件中可以通过node.master、node.data来设置节点类型。

　　node.master：表示节点是否具有称为主节点的资格

　　　　true代表的是有资格竞选主节点

　　　　false代表的是没有资格竞选主节点

　　node.data：表示节点是否存储数据

Node节点组合

主节点+数据节点(master+data)

　　　　节点即有称为主节点的资格，又存储数据

node.master: true

node.data: true

数据节点(data)

　　节点没有成为主节点的资格，不参与选举，只会存储数据

node.master: false

node.data: true

客户端节点(client)

　　不会成为主节点，也不会存储数据，主要是针对海量请求的时候，可以进行负载均衡

node.master: false

node.data: false

分片

　　每个索引有一个或多个分片，每个分片存储不同的数据。分片可分为主分片(primary shard)和复制分片(replica shard)，复制分片是主分片的拷贝。默认每个主分片有一个复制分片，一个索引的复制分片的数量可以动态地调整，复制分片匆匆不与它的主分片在同一个节点上。

搭建ES集群

搭建步骤

拷贝ES7.8.1安装包3份，分别命名es-a，es-b,es-c
分别修改elasticsearch.yml文件
分别启动a、b、c三个节点
打开浏览器输入:ip:port/_cat/health?v，如果返回的nodt.total是3，代表集群搭建成功

配置文件

#集群名称，三台集群，要配置相同的集群名称！！！

cluster.name: my-application

#节点名称

node.name: node-1 #是不是有资格主节点

node.master: true

#是否存储数据

node.data: true

#最⼤集群节点数

node.max_local_storage_nodes: 3 #⽹关地址

network.host: 0.0.0.0

#端⼝

http.port: 9200

#内部节点之间沟通端⼝

transport.tcp.port: 9300

#es7.x 之后新增的配置，写⼊候选主节点的设备地址，在开启服务后可以被选为主节点

discovery.seed_hosts: ["localhost:9300","localhost:9400","localhost:9500"]

#es7.x 之后新增的配置，初始化⼀个新的集群时需要此配置来选举master

cluster.initial_master_nodes: ["node-1", "node-2","node-3"] #数据和存储路径

path.data: /Users/louis.chen/Documents/study/search/storage/a/data

path.logs: /Users/louis.chen/Documents/study/search/storage/a/logs

注意

　　因为我是一台服务器上，搞了3搞ES启动端口号不同，拷贝了3份es，具体还要跟实际情况相应调整

真实配置-a

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

#       Before you set out to tweak and tune the configuration, make sure you

#       understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

# 集群名称

cluster.name: my-application

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

# 集群名称

node.name: node-1

# 是不是有资格主节点

node.master: true

# 是否存储数据

node.data: true

# 最大集群节点数，因为3个集群，所以配置3

node.max_local_storage_nodes: 3

#

# Add custom attributes to the node:

#

#node.attr.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

# 数据存储路径

path.data: /var/soft/es7.8.1/elasticsearch-7.8.1/data

# 日志存储路径

# Path to log files:

#

path.logs: /var/soft/es7.8.1/elasticsearch-7.8.1/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

#bootstrap.memory_lock: true

#

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

# 网关地址

network.host: 0.0.0.0

#

# Set a custom port for HTTP:

# 端口

http.port: 9200

# 内部节点之间沟通端口

transport.tcp.port: 9300

#

# For more information, consult the network module documentation.

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

# es7.x之后新增的配置，写入候选主节点的设备地址，在开启服务后可以被选为主节点

discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]

#

# Bootstrap the cluster using an initial set of master-eligible nodes:

# es7.x之后新增的配置，初始化一个新的集群时需要此配置来选举master

cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

#

# For more information, consult the discovery and cluster formation module documentation.

#

# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

#gateway.recover_after_nodes: 3

#

# For more information, consult the gateway module documentation.

#

# ---------------------------------- Various -----------------------------------

#

# Require explicit names when deleting indices:

#

#action.destructive_requires_name: true

bootstrap.system_call_filter: false

http.cors.allow-origin: "*"

http.cors.enabled: true

http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization

http.cors.allow-credentials: true

真实配置-b

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

#       Before you set out to tweak and tune the configuration, make sure you

#       understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

# 集群名称

cluster.name: my-application

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

# 集群名称

node.name: node-2

# 是不是有资格主节点

node.master: true

# 是否存储数据

node.data: true

# 最大集群节点数，因为3个集群，所以配置3

node.max_local_storage_nodes: 3

#

# Add custom attributes to the node:

#

#node.attr.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

# 数据存储路径

path.data: /var/soft/es-b/data

# 日志存储路径

# Path to log files:

#

path.logs: /var/soft/es-b/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

#bootstrap.memory_lock: true

#

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

# 网关地址

network.host: 0.0.0.0

#

# Set a custom port for HTTP:

# 端口

http.port: 9201

# 内部节点之间沟通端口

transport.tcp.port: 9400

#

# For more information, consult the network module documentation.

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

# es7.x之后新增的配置，写入候选主节点的设备地址，在开启服务后可以被选为主节点

discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]

#

# Bootstrap the cluster using an initial set of master-eligible nodes:

# es7.x之后新增的配置，初始化一个新的集群时需要此配置来选举master

cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

#

# For more information, consult the discovery and cluster formation module documentation.

#

# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

#gateway.recover_after_nodes: 3

#

# For more information, consult the gateway module documentation.

#

# ---------------------------------- Various -----------------------------------

#

# Require explicit names when deleting indices:

#

#action.destructive_requires_name: true

bootstrap.system_call_filter: false

http.cors.allow-origin: "*"

http.cors.enabled: true

http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization

http.cors.allow-credentials: true

只修改了，2个端口、数据和日志存储路径

真实配置-c

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

#       Before you set out to tweak and tune the configuration, make sure you

#       understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

# 集群名称

cluster.name: my-application

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

# 集群名称

node.name: node-3

# 是不是有资格主节点

node.master: true

# 是否存储数据

node.data: true

# 最大集群节点数，因为3个集群，所以配置3

node.max_local_storage_nodes: 3

#

# Add custom attributes to the node:

#

#node.attr.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

# 数据存储路径

path.data: /var/soft/es-c/data

# 日志存储路径

# Path to log files:

#

path.logs: /var/soft/es-c/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

#bootstrap.memory_lock: true

#

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

# 网关地址

network.host: 0.0.0.0

#

# Set a custom port for HTTP:

# 端口

http.port: 9202

# 内部节点之间沟通端口

transport.tcp.port: 9500

#

# For more information, consult the network module documentation.

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

# es7.x之后新增的配置，写入候选主节点的设备地址，在开启服务后可以被选为主节点

discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]

#

# Bootstrap the cluster using an initial set of master-eligible nodes:

# es7.x之后新增的配置，初始化一个新的集群时需要此配置来选举master

cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

#

# For more information, consult the discovery and cluster formation module documentation.

#

# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

#gateway.recover_after_nodes: 3

#

# For more information, consult the gateway module documentation.

#

# ---------------------------------- Various -----------------------------------

#

# Require explicit names when deleting indices:

#

#action.destructive_requires_name: true

bootstrap.system_call_filter: false

http.cors.allow-origin: "*"

http.cors.enabled: true

http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization

http.cors.allow-credentials: true