一、ELK簡(jiǎn)介

ELK是Elasticsearch、Logstash、Kibana的簡(jiǎn)稱，這三者是核心套件，但并非全部。
Elasticsearch是實(shí)時(shí)全文搜索和分析引擎，提供搜集、分析、存儲(chǔ)數(shù)據(jù)三大功能；是一套開放REST和JAVA API等結(jié)構(gòu)提供高效搜索功能，可擴(kuò)展的分布式系統(tǒng)。它構(gòu)建于Apache Lucene搜索引擎庫(kù)之上。
Logstash是一個(gè)用來(lái)搜集、分析、過濾日志的工具。它支持幾乎任何類型的日志，包括系統(tǒng)日志、錯(cuò)誤日志和自定義應(yīng)用程序日志。它可以從許多來(lái)源接收日志，這些來(lái)源包括 syslog、消息傳遞（例如 RabbitMQ）和JMX，它能夠以多種方式輸出數(shù)據(jù)，包括電子郵件、websockets和Elasticsearch。
Kibana是一個(gè)基于Web的圖形界面，用于搜索、分析和可視化存儲(chǔ)在 Elasticsearch指標(biāo)中的日志數(shù)據(jù)。它利用Elasticsearch的REST接口來(lái)檢索數(shù)據(jù)，不僅允許用戶創(chuàng)建他們自己的數(shù)據(jù)的定制儀表板視圖，還允許他們以特殊的方式查詢和過濾數(shù)據(jù)。
Beats：filebeat:日志文件，metricbeat:類似cpu/內(nèi)存度量數(shù)據(jù)，packetbeat:網(wǎng)絡(luò)數(shù)據(jù)，winlogbeat:windows數(shù)據(jù)，heartbeat：健康檢查
官網(wǎng)地址：https://www.elastic.co/cn/

二、ELK安裝環(huán)境

在主機(jī)hosts文件中配置主機(jī)IP解析

192.168.8.120 master-node
192.168.8.121 data-node1
192.168.8.122 data-node2

1.軟件下載

elasticsearch下載：https://www.elastic.co/cn/downloads/elasticsearch

　　　　　　https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz

logstash下載：https://www.elastic.co/cn/downloads/logstash

kibana下載：https://www.elastic.co/cn/downloads/kibana

　　　　　　https://artifacts.elastic.co/downloads/kibana/kibana-7.5.0-linux-x86_64.tar.gz

JDK下載：https://download.oracle.com/otn/java/jdk/8u231-b11/

2.基礎(chǔ)架構(gòu)圖

3.普通用戶創(chuàng)建

[root@Linux7 soft]# useradd elkuser
[root@Linux7 soft]# passwd elkuser
Changing password for user elkuser.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

4.jdk安裝

[root@Linux7 soft]# rpm -ivh jdk-8u231-linux-x64.rpm

5.操作系統(tǒng)環(huán)境配置

[root@Linux7 ~]# vi /etc/security/limits.conf

* soft nproc 204800
* hard nproc 204800
* soft nofile 655360
* hard nofile 655360
* soft memlock unlimited
* hard memlock unlimited

[root@Linux7 ~]# vi /etc/sysctl.conf

vm.max_map_count=1024000

###RHEL6.0:[root@Linux6 data]# vi /etc/security/limits.d/90-nproc.conf  增加配置內(nèi)容：*          soft    nproc     65535

###RHEL7.0：[root@Linux7 ~]# vi /etc/security/limits.d/20-nproc.conf    增加配置內(nèi)容：*          soft    nproc     65535

[root@Linux7 ~]# sysctl  -p

6.elasticsearch安裝

[root@Linux7 soft]# tar -xzvf elasticsearch-7.5.0-linux-x86_64.tar.gz

[root@Linux7 soft]# mv elasticsearch-7.5.0 /usr/local/elasticsearch7.5

[root@Linux7 soft]# chown -R elkuser.elkuser /usr/local/elasticsearch7.5

elasticsearch 參數(shù)配置：

[elkuser@Linux7 elasticsearch7.5]$ vi config/elasticsearch.yml

network.host: 192.168.8.120
cluster.initial_master_nodes: ["node-1"]

elasticsearch 啟動(dòng)：

[root@Linux7 elasticsearch7.5]# su - elkuser
[elkuser@Linux7 ~]$ cd /usr/local/elasticsearch7.5/

[elkuser@Linux7 elasticsearch7.5]$ bin/elasticsearch

elasticsearch 啟動(dòng)狀態(tài)查看：

[2019-12-13T11:03:45,087][INFO ][o.e.h.AbstractHttpServerTransport] [Linux7] publish_address {10.231.35.131:9200}, bound_addresses {10.231.35.131:9200}
[2019-12-13T11:03:45,088][INFO ][o.e.n.Node               ] [Linux7] started
[2019-12-13T11:03:45,336][INFO ][o.e.l.LicenseService     ] [Linux7] license [149d87ef-d8ca-498a-aef8-5f79f95df570] mode [basic] - valid
[2019-12-13T11:03:45,337][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [Linux7] Active license is now [BASIC]; Security is disabled
[2019-12-13T11:03:45,353][INFO ][o.e.g.GatewayService     ] [Linux7] recovered [0] indices into cluster_state

elasticsearch 訪問：

瀏覽器地址欄輸入：192.168.8.120:9200 

7.kibana安裝

[root@Linux7 soft]# tar -xzvf kibana-7.5.0-linux-x86_64.tar.gz
[root@Linux7 soft]# mv kibana-7.5.0-linux-x86_64 /usr/local/kibana7.5
[root@Linux7 soft]# chown -R elkuser:elkuser /usr/local/kibana7.5/

kibana配置修改：

[elkuser@Linux7 kibana7.5]$ vi config/kibana.yml

server.host: "192.168.8.120"
elasticsearch.hosts: ["http://192.168.8.120:9200"]

kibana啟動(dòng)：

[root@Linux7 soft]# su - elkuser
[elkuser@Linux7 kibana7.5]$ bin/kibana

kibana訪問：

http://192.168.8.120:5601

三、總結(jié)

1.安裝elastic stack的需要jdk1.8版本環(huán)境

2.運(yùn)行elastic stack的用戶需要非root

如果是不root模式運(yùn)行，報(bào)錯(cuò)信息如下

[root@Linux7 elasticsearch7.5]# bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-12-13T10:34:41,302][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [Linux7] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.0.jar:7.5.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.5.0.jar:7.5.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.0.jar:7.5.0]
        ... 6 more

3.elastic stack分為生產(chǎn)（配置network.host為非localhost）和非生產(chǎn)模式，其中生產(chǎn)模式要對(duì)運(yùn)行環(huán)境做檢測(cè)，比如用戶的會(huì)話限制、cluster.initail_master_nodes等參數(shù)。

修改config/elasticsearch.yml配置文件network.host為非localhost模式后，啟動(dòng)報(bào)錯(cuò)信息如下

ERROR: [2] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2019-12-13T10:46:44,400][INFO ][o.e.n.Node               ] [Linux7] stopping ...
[2019-12-13T10:46:44,427][INFO ][o.e.n.Node               ] [Linux7] stopped
[2019-12-13T10:46:44,428][INFO ][o.e.n.Node               ] [Linux7] closing ...
[2019-12-13T10:46:44,471][INFO ][o.e.n.Node               ] [Linux7] closed
[2019-12-13T10:46:44,476][INFO ][o.e.x.m.p.NativeController] [Linux7] Native controller process has stopped - no new native processes can be started