测试开发技术网站

博客

ElasticSearch----基于docker-compose方式部署三节点8.1.3版本的ElasticSearch集群 Kibana看板
作者：redrose2100   类别：数据库    日期：2022-06-07 08:24:29    阅读：1577 次   消耗积分：0 分

1 准备好环境变量

内容如下，保存在.env文件中，存放在一个空的目录中，这里注意密码需要给ELASTIC_PASSWORD 和 KIBANA_PASSWORD 配置一个比较强的，不能使用过于简单的密码,
此外，这里暴露端口是通过ES_PORT设置的，默认暴露9200端口，如果是其他端口需要修改下面变量文件中的ES_PORT值
kibana端口通过KIBANA_PORT设置，默认为5601，那么也可以设置为其他端口

  1. # Password for the 'elastic' user (at least 6 characters)
  2. ELASTIC_PASSWORD=xxxxxxxxxxx
  4. # Password for the 'kibana_system' user (at least 6 characters)
  5. KIBANA_PASSWORD=xxxxxxxxxxx
  7. # Version of Elastic products
  8. STACK_VERSION=8.1.3
  10. # Set the cluster name
  11. CLUSTER_NAME=docker-cluster
  13. # Set to 'basic' or 'trial' to automatically start the 30-day trial
  14. LICENSE=basic
  15. #LICENSE=trial
  17. # Port to expose Elasticsearch HTTP API to the host
  18. ES_PORT=9200
  19. #ES_PORT=127.0.0.1:9200
  21. # Port to expose Kibana to the host
  22. KIBANA_PORT=5601
  23. #KIBANA_PORT=80
  25. # Increase or decrease based on the available host memory (in bytes)
  26. MEM_LIMIT=1073741824
  28. # Project namespace (defaults to the current folder name if not set)
  29. #COMPOSE_PROJECT_NAME=myproject

2 准备docke-compose.yml文件

若启用https，使用如下配置文件

  1. version: "2.2"
  3. services:
  4.   setup:
  5.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  6.     volumes:
  7.       - certs:/usr/share/elasticsearch/config/certs
  8.     user: "0"
  9.     command: >
  10.       bash -c '
  11.         if [ x${ELASTIC_PASSWORD} == x ]; then
  12.           echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
  13.           exit 1;
  14.         elif [ x${KIBANA_PASSWORD} == x ]; then
  15.           echo "Set the KIBANA_PASSWORD environment variable in the .env file";
  16.           exit 1;
  17.         fi;
  18.         if [ ! -f config/certs/ca.zip ]; then
  19.           echo "Creating CA";
  20.           bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
  21.           unzip config/certs/ca.zip -d config/certs;
  22.         fi;
  23.         if [ ! -f config/certs/certs.zip ]; then
  24.           echo "Creating certs";
  25.           echo -ne \
  26.           "instances:\n"\
  27.           "  - name: es01\n"\
  28.           "    dns:\n"\
  29.           "      - es01\n"\
  30.           "      - localhost\n"\
  31.           "    ip:\n"\
  32.           "      - 127.0.0.1\n"\
  33.           "  - name: es02\n"\
  34.           "    dns:\n"\
  35.           "      - es02\n"\
  36.           "      - localhost\n"\
  37.           "    ip:\n"\
  38.           "      - 127.0.0.1\n"\
  39.           "  - name: es03\n"\
  40.           "    dns:\n"\
  41.           "      - es03\n"\
  42.           "      - localhost\n"\
  43.           "    ip:\n"\
  44.           "      - 127.0.0.1\n"\
  45.           > config/certs/instances.yml;
  46.           bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
  47.           unzip config/certs/certs.zip -d config/certs;
  48.         fi;
  49.         echo "Setting file permissions"
  50.         chown -R root:root config/certs;
  51.         find . -type d -exec chmod 750 \{\} \;;
  52.         find . -type f -exec chmod 640 \{\} \;;
  53.         echo "Waiting for Elasticsearch availability";
  54.         until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
  55.         echo "Setting kibana_system password";
  56.         until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
  57.         echo "All done!";
  58.       '
  59.     healthcheck:
  60.       test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
  61.       interval: 1s
  62.       timeout: 5s
  63.       retries: 120
  65.   es01:
  66.     depends_on:
  67.       setup:
  68.         condition: service_healthy
  69.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  70.     volumes:
  71.       - certs:/usr/share/elasticsearch/config/certs
  72.       - esdata01:/usr/share/elasticsearch/data
  73.     ports:
  74.       - ${ES_PORT}:9200
  75.     environment:
  76.       - node.name=es01
  77.       - cluster.name=${CLUSTER_NAME}
  78.       - cluster.initial_master_nodes=es01,es02,es03
  79.       - discovery.seed_hosts=es02,es03
  80.       - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
  81.       - bootstrap.memory_lock=true
  82.       - xpack.security.enabled=true
  83.       - xpack.security.http.ssl.enabled=true
  84.       - xpack.security.http.ssl.key=certs/es01/es01.key
  85.       - xpack.security.http.ssl.certificate=certs/es01/es01.crt
  86.       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
  87.       - xpack.security.http.ssl.verification_mode=certificate
  88.       - xpack.security.transport.ssl.enabled=true
  89.       - xpack.security.transport.ssl.key=certs/es01/es01.key
  90.       - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
  91.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  92.       - xpack.security.transport.ssl.verification_mode=certificate
  93.       - xpack.license.self_generated.type=${LICENSE}
  94.     mem_limit: ${MEM_LIMIT}
  95.     ulimits:
  96.       memlock:
  97.         soft: -1
  98.         hard: -1
  99.     healthcheck:
  100.       test:
  101.         [
  102.           "CMD-SHELL",
  103.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  104.         ]
  105.       interval: 10s
  106.       timeout: 10s
  107.       retries: 120
  109.   es02:
  110.     depends_on:
  111.       - es01
  112.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  113.     volumes:
  114.       - certs:/usr/share/elasticsearch/config/certs
  115.       - esdata02:/usr/share/elasticsearch/data
  116.     environment:
  117.       - node.name=es02
  118.       - cluster.name=${CLUSTER_NAME}
  119.       - cluster.initial_master_nodes=es01,es02,es03
  120.       - discovery.seed_hosts=es01,es03
  121.       - bootstrap.memory_lock=true
  122.       - xpack.security.enabled=true
  123.       - xpack.security.http.ssl.enabled=true
  124.       - xpack.security.http.ssl.key=certs/es02/es02.key
  125.       - xpack.security.http.ssl.certificate=certs/es02/es02.crt
  126.       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
  127.       - xpack.security.http.ssl.verification_mode=certificate
  128.       - xpack.security.transport.ssl.enabled=true
  129.       - xpack.security.transport.ssl.key=certs/es02/es02.key
  130.       - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
  131.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  132.       - xpack.security.transport.ssl.verification_mode=certificate
  133.       - xpack.license.self_generated.type=${LICENSE}
  134.     mem_limit: ${MEM_LIMIT}
  135.     ulimits:
  136.       memlock:
  137.         soft: -1
  138.         hard: -1
  139.     healthcheck:
  140.       test:
  141.         [
  142.           "CMD-SHELL",
  143.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  144.         ]
  145.       interval: 10s
  146.       timeout: 10s
  147.       retries: 120
  149.   es03:
  150.     depends_on:
  151.       - es02
  152.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  153.     volumes:
  154.       - certs:/usr/share/elasticsearch/config/certs
  155.       - esdata03:/usr/share/elasticsearch/data
  156.     environment:
  157.       - node.name=es03
  158.       - cluster.name=${CLUSTER_NAME}
  159.       - cluster.initial_master_nodes=es01,es02,es03
  160.       - discovery.seed_hosts=es01,es02
  161.       - bootstrap.memory_lock=true
  162.       - xpack.security.enabled=true
  163.       - xpack.security.http.ssl.enabled=true
  164.       - xpack.security.http.ssl.key=certs/es03/es03.key
  165.       - xpack.security.http.ssl.certificate=certs/es03/es03.crt
  166.       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
  167.       - xpack.security.http.ssl.verification_mode=certificate
  168.       - xpack.security.transport.ssl.enabled=true
  169.       - xpack.security.transport.ssl.key=certs/es03/es03.key
  170.       - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
  171.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  172.       - xpack.security.transport.ssl.verification_mode=certificate
  173.       - xpack.license.self_generated.type=${LICENSE}
  174.     mem_limit: ${MEM_LIMIT}
  175.     ulimits:
  176.       memlock:
  177.         soft: -1
  178.         hard: -1
  179.     healthcheck:
  180.       test:
  181.         [
  182.           "CMD-SHELL",
  183.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  184.         ]
  185.       interval: 10s
  186.       timeout: 10s
  187.       retries: 120
  189.   kibana:
  190.     depends_on:
  191.       es01:
  192.         condition: service_healthy
  193.       es02:
  194.         condition: service_healthy
  195.       es03:
  196.         condition: service_healthy
  197.     image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
  198.     volumes:
  199.       - certs:/usr/share/kibana/config/certs
  200.       - kibanadata:/usr/share/kibana/data
  201.     ports:
  202.       - ${KIBANA_PORT}:5601
  203.     environment:
  204.       - SERVERNAME=kibana
  205.       - ELASTICSEARCH_HOSTS=https://es01:9200
  206.       - ELASTICSEARCH_USERNAME=kibana_system
  207.       - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
  208.       - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
  209.     mem_limit: ${MEM_LIMIT}
  210.     healthcheck:
  211.       test:
  212.         [
  213.           "CMD-SHELL",
  214.           "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
  215.         ]
  216.       interval: 10s
  217.       timeout: 10s
  218.       retries: 120
  220. volumes:
  221.   certs:
  222.     driver: local
  223.   esdata01:
  224.     driver: local
  225.   esdata02:
  226.     driver: local
  227.   esdata03:
  228.     driver: local
  229.   kibanadata:
  230.     driver: local

若禁用https，则使用如下配置文件

  1. version: "2.2"
  3. services:
  4.   setup:
  5.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  6.     volumes:
  7.       - certs:/usr/share/elasticsearch/config/certs
  8.     user: "0"
  9.     command: >
  10.       bash -c '
  11.         if [ x${ELASTIC_PASSWORD} == x ]; then
  12.           echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
  13.           exit 1;
  14.         elif [ x${KIBANA_PASSWORD} == x ]; then
  15.           echo "Set the KIBANA_PASSWORD environment variable in the .env file";
  16.           exit 1;
  17.         fi;
  18.         if [ ! -f config/certs/ca.zip ]; then
  19.           echo "Creating CA";
  20.           bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
  21.           unzip config/certs/ca.zip -d config/certs;
  22.         fi;
  23.         if [ ! -f config/certs/certs.zip ]; then
  24.           echo "Creating certs";
  25.           echo -ne \
  26.           "instances:\n"\
  27.           "  - name: es01\n"\
  28.           "    dns:\n"\
  29.           "      - es01\n"\
  30.           "      - localhost\n"\
  31.           "    ip:\n"\
  32.           "      - 127.0.0.1\n"\
  33.           "  - name: es02\n"\
  34.           "    dns:\n"\
  35.           "      - es02\n"\
  36.           "      - localhost\n"\
  37.           "    ip:\n"\
  38.           "      - 127.0.0.1\n"\
  39.           "  - name: es03\n"\
  40.           "    dns:\n"\
  41.           "      - es03\n"\
  42.           "      - localhost\n"\
  43.           "    ip:\n"\
  44.           "      - 127.0.0.1\n"\
  45.           > config/certs/instances.yml;
  46.           bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
  47.           unzip config/certs/certs.zip -d config/certs;
  48.         fi;
  49.         echo "Setting file permissions"
  50.         chown -R root:root config/certs;
  51.         find . -type d -exec chmod 750 \{\} \;;
  52.         find . -type f -exec chmod 640 \{\} \;;
  53.         echo "Waiting for Elasticsearch availability";
  54.         until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
  55.         echo "Setting kibana_system password";
  56.         until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
  57.         echo "All done!";
  58.       '
  59.     healthcheck:
  60.       test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
  61.       interval: 1s
  62.       timeout: 5s
  63.       retries: 120
  65.   es01:
  66.     depends_on:
  67.       setup:
  68.         condition: service_healthy
  69.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  70.     volumes:
  71.       - certs:/usr/share/elasticsearch/config/certs
  72.       - esdata01:/usr/share/elasticsearch/data
  73.     ports:
  74.       - ${ES_PORT}:9200
  75.     environment:
  76.       - node.name=es01
  77.       - cluster.name=${CLUSTER_NAME}
  78.       - cluster.initial_master_nodes=es01,es02,es03
  79.       - discovery.seed_hosts=es02,es03
  80.       - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
  81.       - bootstrap.memory_lock=true
  82.       - xpack.security.transport.ssl.enabled=true
  83.       - xpack.security.transport.ssl.key=certs/es01/es01.key
  84.       - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
  85.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  86.       - xpack.security.transport.ssl.verification_mode=certificate
  87.       - xpack.license.self_generated.type=${LICENSE}
  88.     mem_limit: ${MEM_LIMIT}
  89.     ulimits:
  90.       memlock:
  91.         soft: -1
  92.         hard: -1
  93.     healthcheck:
  94.       test:
  95.         [
  96.           "CMD-SHELL",
  97.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  98.         ]
  99.       interval: 10s
  100.       timeout: 10s
  101.       retries: 120
  103.   es02:
  104.     depends_on:
  105.       - es01
  106.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  107.     volumes:
  108.       - certs:/usr/share/elasticsearch/config/certs
  109.       - esdata02:/usr/share/elasticsearch/data
  110.     environment:
  111.       - node.name=es02
  112.       - cluster.name=${CLUSTER_NAME}
  113.       - cluster.initial_master_nodes=es01,es02,es03
  114.       - discovery.seed_hosts=es01,es03
  115.       - bootstrap.memory_lock=true
  116.       - xpack.security.transport.ssl.enabled=true
  117.       - xpack.security.transport.ssl.key=certs/es02/es02.key
  118.       - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
  119.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  120.       - xpack.security.transport.ssl.verification_mode=certificate
  121.       - xpack.license.self_generated.type=${LICENSE}
  122.     mem_limit: ${MEM_LIMIT}
  123.     ulimits:
  124.       memlock:
  125.         soft: -1
  126.         hard: -1
  127.     healthcheck:
  128.       test:
  129.         [
  130.           "CMD-SHELL",
  131.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  132.         ]
  133.       interval: 10s
  134.       timeout: 10s
  135.       retries: 120
  137.   es03:
  138.     depends_on:
  139.       - es02
  140.     image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
  141.     volumes:
  142.       - certs:/usr/share/elasticsearch/config/certs
  143.       - esdata03:/usr/share/elasticsearch/data
  144.     environment:
  145.       - node.name=es03
  146.       - cluster.name=${CLUSTER_NAME}
  147.       - cluster.initial_master_nodes=es01,es02,es03
  148.       - discovery.seed_hosts=es01,es02
  149.       - bootstrap.memory_lock=true
  150.       - xpack.security.transport.ssl.enabled=true
  151.       - xpack.security.transport.ssl.key=certs/es03/es03.key
  152.       - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
  153.       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
  154.       - xpack.security.transport.ssl.verification_mode=certificate
  155.       - xpack.license.self_generated.type=${LICENSE}
  156.     mem_limit: ${MEM_LIMIT}
  157.     ulimits:
  158.       memlock:
  159.         soft: -1
  160.         hard: -1
  161.     healthcheck:
  162.       test:
  163.         [
  164.           "CMD-SHELL",
  165.           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
  166.         ]
  167.       interval: 10s
  168.       timeout: 10s
  169.       retries: 120
  171.   kibana:
  172.     depends_on:
  173.       es01:
  174.         condition: service_healthy
  175.       es02:
  176.         condition: service_healthy
  177.       es03:
  178.         condition: service_healthy
  179.     image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
  180.     volumes:
  181.       - certs:/usr/share/kibana/config/certs
  182.       - kibanadata:/usr/share/kibana/data
  183.     ports:
  184.       - ${KIBANA_PORT}:5601
  185.     environment:
  186.       - SERVERNAME=kibana
  187.       - ELASTICSEARCH_HOSTS=http://es01:9200
  188.       - ELASTICSEARCH_USERNAME=kibana_system
  189.       - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
  190.     mem_limit: ${MEM_LIMIT}
  191.     healthcheck:
  192.       test:
  193.         [
  194.           "CMD-SHELL",
  195.           "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
  196.         ]
  197.       interval: 10s
  198.       timeout: 10s
  199.       retries: 120
  201. volumes:
  202.   certs:
  203.     driver: local
  204.   esdata01:
  205.     driver: local
  206.   esdata02:
  207.     driver: local
  208.   esdata03:
  209.     driver: local
  210.   kibanadata:
  211.     driver: local

3 配置vm.max_map_count 最小为262144

  1. vi /etc/sysctl.conf

然后修改vm.max_map_count的值为262144，如果不存在，则直接增加以下内容

  1. vm.max_map_count=262144

这种方式需要重启，一般情况下环境不允许重启的，因此这里可以继续使用如下命令配置

  1. sysctl -w vm.max_map_count=262144

4 启动服务

执行如下命令，启动服务

  1. docker-compose up -d

5 通过docker-compose ps 命令查看状态，如下，标志已经启动成功

  1. [root@master es]# docker-compose ps
  2.    Name                  Command                  State                             Ports
  3. ----------------------------------------------------------------------------------------------------------------
  4. es_es01_1     /bin/tini -- /usr/local/bi ...   Up (healthy)   0.0.0.0:9200->9200/tcp,:::9200->9200/tcp, 9300/tcp
  5. es_es02_1     /bin/tini -- /usr/local/bi ...   Up (healthy)   9200/tcp, 9300/tcp
  6. es_es03_1     /bin/tini -- /usr/local/bi ...   Up (healthy)   9200/tcp, 9300/tcp
  7. es_kibana_1   /bin/tini -- /usr/local/bi ...   Up (healthy)   0.0.0.0:5601->5601/tcp,:::5601->5601/tcp
  8. es_setup_1    /bin/tini -- /usr/local/bi ...   Exit 0
  9. [root@master es]#

6 浏览器访问kibana

然后再浏览器通过ip+端口即可访问了，比如这里 192.168.116.50:5601，然后输入用户名 elastic和配置的密码，如下

7 登录kibana

始终坚持开源开放共享精神，同时感谢您的充电鼓励和支持！
版权所有，转载本站文章请注明出处：redrose2100， http://blog.redrose2100.com/article/315
个人成就
  • 博客总数： 613 
  • 阅读总量： 715110 
  • 2022年 ： 371 篇 
  • 2023年 ： 211 篇 
  • 2024年 ： 31 篇 
  • 2025年 ： 0 篇 
出版书籍
测试开发技术全栈公众号
DevOps技术交流微信群
常用网站链接
文章分类
最新文章
最多阅读
特别推荐

版权所有 © devops-dev.com (2021-2023)

   苏公网安备32011502011999

苏ICP备2021008941号-2

友情链接

CSDN    Github    Gitee   

联系我们

北京★北京市中关村   微信：redrose2100
邮箱：hitredrose@163.com