ChaKanNom's Blog: 8월 2018

2018년 8월 30일 목요일

Docker로 Elasticsearch Cluster 만들기

** 엘라스틱을 사용하기 위해 memlock과 nofile을 변경해야 함
** 참고 : Default ulimit(memlock & nofile) settings (for Elasticsearch)
** 아래 내용은 docker swarm을 기반으로 함
Virtual memory areas 늘리기

$ sudo /etc/sysctl.confi
===== sysctl.conf =====
...
vm.max_map_count=262144
...
==========================
$ sudo reboot now

Master node

$ sudo docker service \
create \
--constraint node.role==manager \
--reserve-cpu 2 \
--reserve-memory 2g \
--limit-cpu 2 \
--limit-memory 2g \
--network net \
--hostname es-master \
--replicas 1 \
--mount type=bind,src=/etc/localtime,dst=/etc/localtime,readonly \
--env "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
--env "cluster.name=es_cluster" \
--env "node.name=es-master" \
--env "node.master=true" \
--env "node.data=false" \
--env "path.data=/usr/share/elasticsearch/data" \
--env "path.logs=/usr/share/elasticsearch/logs" \
--env "bootstrap.memory_lock=true" \
--env "network.host=es-master" \
--env "http.port=9200" \
--env "http.cors.enabled=true" \
--env "http.cors.allow-origin=*" \
--env "transport.tcp.port=9300" \
--env "discovery.zen.minimum_master_nodes=1" \
--env "cluster.routing.allocation.disk.threshold_enabled=true" \
--env "cluster.routing.allocation.disk.watermark.low=1000mb" \
--env "cluster.routing.allocation.disk.watermark.high=500mb" \
--env "xpack.ml.enabled=false" \
--env "xpack.monitoring.enabled=true" \
--env "xpack.security.enabled=false" \
--env "xpack.security.audit.enabled=false" \
--env "xpack.watcher.enabled=false" \
--name es-master \
docker.elastic.co/elasticsearch/elasticsearch:5.5.3

Data node
** {number}에 따라 data node를 여러개 만들 수 있음

$ sudo docker service \
create \
--reserve-cpu 2 \
--reserve-memory 2g \
--limit-cpu 2 \
--limit-memory 2g \
--network net \
--hostname es-data-{number} \
--replicas 1 \
--mount type=bind,src=/etc/localtime,dst=/etc/localtime,readonly \
--env "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
--env "cluster.name=es_cluster" \
--env "node.name=es-data-{number}" \
--env "node.master=false" \
--env "node.data=true" \
--env "path.data=/usr/share/elasticsearch/data" \
--env "path.logs=/usr/share/elasticsearch/logs" \
--env "bootstrap.memory_lock=true" \
--env "network.host=es-data-{number}" \
--env "transport.tcp.port=9300" \
--env "discovery.zen.ping.unicast.hosts=es-master" \
--env "discovery.zen.minimum_master_nodes=1" \
--env "cluster.routing.allocation.disk.threshold_enabled=true" \
--env "cluster.routing.allocation.disk.watermark.low=1000mb" \
--env "cluster.routing.allocation.disk.watermark.high=500mb" \
--env "xpack.ml.enabled=false" \
--env "xpack.monitoring.enabled=true" \
--env "xpack.security.enabled=false" \
--env "xpack.security.audit.enabled=false" \
--env "xpack.watcher.enabled=false" \
--name es-data-{number} \
docker.elastic.co/elasticsearch/elasticsearch:5.5.3

2018년 8월 28일 화요일

Docker Swarm Cluster 만들기

--- In nodes
$ sudo docker network \
create \
--subnet 172.171.0.0/16 \
--gateway 172.171.0.1 \
-o com.docker.network.bridge.enable_icc=false \
-o com.docker.network.bridge.enable_ip_masquerade=true \
-o com.docker.network.bridge.name=docker_gwbridge \
docker_gwbridge

--- In manager node
$ sudo docker swarm init --advertise-addr MANGER_IP_ADDRESS

--- In worker node
$ sudo docker swarm join --token TOKEN_KEY MANGER_IP_ADDRESS:2377

--- Verify
$ sudo docker node ls

--- Set overlay network
$ sudo docker network \
create \
--attachable \
--driver=overlay \
--subnet=172.172.0.0/24 \
--ip-range=172.172.0.0/24 \
--gateway=172.172.0.1 \
NETWORK_NAME

Docker 설정

Enable the remote API for dockerd

$ sudo mkdir -p /etc/systemd/system/docker.service.d
$ sudo vim /etc/systemd/system/docker.service.d/override.conf
===== override.conf =====
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -H tcp://0.0.0.0:2375
...
==========================
$ sudo systemctl daemon-reload
$ sudo service docker restart

Default ulimit(memlock & nofile) settings (for Elasticsearch)

$ sudo vim /etc/security/limits.conf
===== limits.conf =====
...
* - memlock unlimited
* - nofile 65536
...
==========================

$ sudo mkdir -p /etc/systemd/system/docker.service.d
$ sudo vim /etc/systemd/system/docker.service.d/override.conf
===== override.conf =====
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --default-ulimit memlock=-1 --default-ulimit nofile=65536
...
==========================
$ sudo systemctl daemon-reload
$ sudo service docker restart

Change default (/var/lib/docker) to another directory

$ sudo mkdir -p /etc/systemd/system/docker.service.d
$ sudo vim /etc/systemd/system/docker.service.d/override.conf
===== override.conf =====
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -g /another/directory
...
=============================
$ sudo systemctl daemon-reload
$ sudo service docker restart

Docker CE 설치

Docker CE for Ubuntu 설치

** OS 버전(64bit): Bionic 18.04 (LTS) or Artful 17.10 or Xenial 16.04 (LTS) or Trusty 14.04 (LTS)

Install using the repository
----------------------------------------
$ sudo apt-get update
$ sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
----------------------------------------

Install Docker CE
----------------------------------------
$ sudo apt-get update
$ sudo apt-get install docker-ce

--- Starts automatically
--- Verify
$ sudo docker run hello-world
----------------------------------------

Docker CE for CentOS 설치

** OS 버전: CentOS 7이상

Install using the repository
----------------------------------------
$ sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2
$ sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
----------------------------------------

Install Docker CE
----------------------------------------
$ sudo yum update
$ sudo yum install docker-ce

--- Start Docker
$ sudo systemctl start docker

--- Verify
$ sudo docker run hello-world
----------------------------------------

Configure Docker to start on boot

----------------------------------------
--- Enable
$ sudo systemctl enable docker

--- Disable
$ sudo systemctl disable docker
----------------------------------------

참고

페이지