测试开发技术网站

[【原文链接】Skywalking----基于Kubernetes部署Skywalking](http://devops-dev.com/article/663)

[TOC]

![](https://redrose2100.oss-cn-hangzhou.aliyuncs.com/img/54317c6c-4c6e-11ee-a736-0242ac110004.png)

# 一、环境准备
在部署Skywalking之前，最好准备一个全新的Kubernetes集群环境，否则当在对Skywalking做重新部署或回退的时候，很有可能会将之前部署的资源清除掉，切记切记！
## 1.1 部署k8s环境
可以参考 [Kubernetes----基于kubeadm工具在CentOS7.9虚拟机上部署一主两从类型的Kubernetes集群环境](http://devops-dev.com/article/662) 部署一套全新的k8s集群

## 1.2 前提条件
如果想部署全部的Skywalking组件，虚拟机配置至少4核内存至少4G，可通过如下命令查看cpu核数和内存是否满足要求，如下明显满足条件
```bash
[root@master ~]# cat /proc/cpuinfo | grep "processor" | wc -l
8
[root@master ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:            31G        2.1G         26G         18M        2.4G         28G
Swap:            0B          0B          0B
[root@master ~]#
```

此外，必须拥有权限执行 kubectl 命令，可通过如下命令查看是否拥有权限
```bash
kubectl --help
```

## 1.3 Kubernetes环境安装helm工具
在Kubernetes集群环境的master节点执行如下命令
```bash
cd /opt/
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
```
安装完成后，可以通过如下命令确认是否成功安装helm
```bash
helm --help
```

# 二、部署 Skywalking
## 2.1 Kubernetes环境创建命名空间
```bash
kubectl create namespace skywalking
```

## 2.2 配置变量
修改版本号以及namespace
```bash
export SKYWALKING_RELEASE_VERSION=4.3.0
export SKYWALKING_RELEASE_NAME=skywalking
export SKYWALKING_RELEASE_NAMESPACE=skywalking
```

## 2.3 执行安装命令
```bash

helm install "${SKYWALKING_RELEASE_NAME}" \
  oci://registry-1.docker.io/apache/skywalking-helm \
  --version "${SKYWALKING_RELEASE_VERSION}" \
  -n "${SKYWALKING_RELEASE_NAMESPACE}" \
  --set oap.image.tag=9.6.0 \
  --set oap.storageType=elasticsearch \
  --set ui.image.tag=9.6.0 \
  --set ui.service.type=NodePort \
  --set ui.service.nodePort=30080 \
  --set elasticsearch.imageTag=7.5.1 \
  --set oap.replicas=1 \
  --set elasticsearch.minimumMasterNodes=1 \
  --set elasticsearch.replicas=1
```

注意，在执行安装之前，可参考如下参数根据需求进行配置，可配置的参数如下所示

| Parameter | Description | Default |
|----------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| `nameOverride` | Override name | `nil` |
| `serviceAccounts.oap.create` | Create of the OAP service account | `true` |
| `serviceAccounts.oap.name` | Name of the OAP service account to use custom service account when `serviceAccounts.oap.create` is set to false | `` |
| `imagePullSecrets` | Image pull secrets | `[]` |
| `oap.name` | OAP deployment name | `oap` |
| `oap.dynamicConfig.enabled` | Enable oap dynamic configuration through k8s configmap | `false` |
| `oap.dynamicConfig.period` | Sync period in seconds | `60` |
| `oap.dynamicConfig.config` | Oap dynamic configuration [documentation](https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/dynamic-config.md) | `{}` |
| `oap.image.repository` | OAP container image name | `skywalking.docker.scarf.sh/apache/skywalking-oap-server` |
| `oap.image.tag` | OAP container image tag | `6.1.0` |
| `oap.image.pullPolicy` | OAP container image pull policy | `IfNotPresent` |
| `oap.ports.grpc` | OAP grpc port for tracing or metric | `11800` |
| `oap.ports.rest` | OAP http port for Web UI | `12800` |
| `oap.ports.zipkinreceiver` | OAP http port for Zipkin receiver(not exposed by default) | `9411` |
| `oap.ports.zipkinquery` | OAP http port for querying Zipkin traces and UI(not exposed by default) | `9412` |
| `oap.replicas` | OAP k8s deployment replicas | `2` |
| `oap.service.type` | OAP svc type | `ClusterIP` |
| `oap.javaOpts` | Parameters to be added to `JAVA_OPTS`environment variable for OAP | `-Xms2g -Xmx2g` |
| `oap.antiAffinity` | OAP anti-affinity policy | `soft` |
| `oap.nodeAffinity` | OAP node affinity policy | `{}` |
| `oap.nodeSelector` | OAP labels for master pod assignment | `{}` |
| `oap.tolerations` | OAP tolerations | `[]` |
| `oap.resources` | OAP node resources requests & limits | `{} - cpu limit must be an integer` |
| `oap.startupProbe` | Configuration fields for the [startupProbe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) | `tcpSocket.port: 12800` `failureThreshold: 9` `periodSeconds: 10` 
| `oap.livenessProbe` | Configuration fields for the [livenessProbe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) | `tcpSocket.port: 12800` `initialDelaySeconds: 5` `periodSeconds: 10` 
| `oap.readinessProbe` | Configuration fields for the [readinessProbe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) | `tcpSocket.port: 12800` `initialDelaySeconds: 5` `periodSeconds: 10` 
| `oap.env` | OAP environment variables | `[]` |
| `oap.securityContext` | Allows you to set the [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) for the pod | `fsGroup: 1000` `runAsUser: 1000` |
| `ui.name` | Web UI deployment name | `ui` |
| `ui.replicas` | Web UI k8s deployment replicas | `1` |
| `ui.image.repository` | Web UI container image name | `skywalking.docker.scarf.sh/apache/skywalking-ui` |
| `ui.image.tag` | Web UI container image tag | `6.1.0` |
| `ui.image.pullPolicy` | Web UI container image pull policy | `IfNotPresent` |
| `ui.nodeAffinity` | Web UI node affinity policy | `{}` |
| `ui.nodeSelector` | Web UI labels for pod assignment | `{}` |
| `ui.tolerations` | Web UI tolerations | `[]` |
| `ui.ingress.enabled` | Create Ingress for Web UI | `false` |
| `ui.ingress.annotations` | Associate annotations to the Ingress | `{}` |
| `ui.ingress.path` | Associate path with the Ingress | `/` |
| `ui.ingress.hosts` | Associate hosts with the Ingress | `[]` |
| `ui.ingress.tls` | Associate TLS with the Ingress | `[]` |
| `ui.service.type` | Web UI svc type | `ClusterIP` |
| `ui.service.externalPort` | external port for the service | `80` |
| `ui.service.internalPort` | internal port for the service | `8080` |
| `ui.service.externalIPs` | external IP addresses | `nil` |
| `ui.service.loadBalancerIP` | Load Balancer IP address | `nil` |
| `ui.service.annotations` | Kubernetes service annotations | `{}` |
| `ui.service.loadBalancerSourceRanges` | Limit load balancer source IPs to list of CIDRs (where available)) | `[]` |
| `ui.securityContext` | Allows you to set the [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) for the pod | `fsGroup: 1000` `runAsUser: 1000` |
| `oapInit.nodeAffinity` | OAP init job node affinity policy | `{}` |
| `oapInit.nodeSelector` | OAP init job labels for master pod assignment | `{}` |
| `oapInit.tolerations` | OAP init job tolerations | `[]` |
| `oapInit.extraPodLabels` | OAP init job metadata labels | `[]` |
| `elasticsearch.enabled` | Spin up a new elasticsearch cluster for SkyWalking | `true` |
| `elasticsearch.clusterName` | This will be used as the Elasticsearch [cluster.name](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster.name.html) and should be unique per cluster in the namespace | `elasticsearch` |
| `elasticsearch.nodeGroup` | This is the name that will be used for each group of nodes in the cluster. The name will be `clusterName-nodeGroup-X` | `master` |
| `elasticsearch.masterService` | Optional. The service name used to connect to the masters. You only need to set this if your master `nodeGroup` is set to something other than `master`. See [Clustering and Node Discovery](#clustering-and-node-discovery) for more information. | `` |
| `elasticsearch.roles` | A hash map with the [specific roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html) for the node group | `master: true` `data: true` `ingest: true` |
| `elasticsearch.replicas` | Kubernetes replica count for the statefulset (i.e. how many pods) | `3` |
| `elasticsearch.minimumMasterNodes` | The value for [discovery.zen.minimum_master_nodes](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/discovery-settings.html#minimum_master_nodes). Should be set to `(master_eligible_nodes / 2) + 1`. Ignored in Elasticsearch versions >= 7. | `2` |
| `elasticsearch.esMajorVersion` | Used to set major version specific configuration. If you are using a custom image and not running the default Elasticsearch version you will need to set this to the version you are running (e.g. `esMajorVersion: 6`) | `""` |
| `elasticsearch.esConfig` | Allows you to add any config files in `/usr/share/elasticsearch/config/` such as `elasticsearch.yml` and `log4j2.properties`. See [values.yaml](./values.yaml) for an example of the formatting. | `{}` |
| `elasticsearch.extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `[]` |
| `elasticsearch.extraVolumes` | Templatable string of additional volumes to be passed to the `tpl` function | `""` |
| `elasticsearch.extraVolumeMounts` | Templatable string of additional volumeMounts to be passed to the `tpl` function | `""` |
| `elasticsearch.extraInitContainers` | Templatable string of additional init containers to be passed to the `tpl` function | `""` |
| `elasticsearch.secretMounts` | Allows you easily mount a secret as a file inside the statefulset. Useful for mounting certificates and other secrets. See [values.yaml](./values.yaml) for an example | `[]` |
| `elasticsearch.image` | The Elasticsearch docker image | `docker.elastic.co/elasticsearch/elasticsearch` |
| `elasticsearch.imageTag` | The Elasticsearch docker image tag | `7.5.1` |
| `elasticsearch.imagePullPolicy` | The Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value | `IfNotPresent` |
| `elasticsearch.podAnnotations` | Configurable [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) applied to all Elasticsearch pods | `{}` |
| `elasticsearch.labels` | Configurable [label](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) applied to all Elasticsearch pods | `{}` |
| `elasticsearch.esJavaOpts` | [Java options](https://www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html) for Elasticsearch. This is where you should configure the [jvm heap size](https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html) | `-Xmx1g -Xms1g` |
| `elasticsearch.resources` | Allows you to set the [resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the statefulset | `requests.cpu: 100m` `requests.memory: 2Gi` `limits.cpu: 1000m` `limits.memory: 2Gi` |
| `elasticsearch.initResources` | Allows you to set the [resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the initContainer in the statefulset | {} |
| `elasticsearch.sidecarResources` | Allows you to set the [resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the sidecar containers in the statefulset | {} |
| `elasticsearch.networkHost` | Value for the [network.host Elasticsearch setting](https://www.elastic.co/guide/en/elasticsearch/reference/current/network.host.html) | `0.0.0.0` |
| `elasticsearch.volumeClaimTemplate` | Configuration for the [volumeClaimTemplate for statefulsets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-storage). You will want to adjust the storage (default `30Gi`) and the `storageClassName` if you are using a different storage class | `accessModes: [ "ReadWriteOnce" ]` `resources.requests.storage: 30Gi` |
| `elasticsearch.persistence.annotations` | Additional persistence annotations for the `volumeClaimTemplate` | `{}` |
| `elasticsearch.persistence.enabled` | Enables a persistent volume for Elasticsearch data. Can be disabled for nodes that only have [roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html) which don't require persistent data. | `true` |
| `elasticsearch.priorityClassName` | The [name of the PriorityClass](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass). No default is supplied as the PriorityClass must be created first. | `""` |
| `elasticsearch.antiAffinityTopologyKey` | The [anti-affinity topology key](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity). By default this will prevent multiple Elasticsearch nodes from running on the same Kubernetes node | `kubernetes.io/hostname` |
| `elasticsearch.antiAffinity` | Setting this to hard enforces the [anti-affinity rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity). If it is set to soft it will be done "best effort". Other values will be ignored. | `hard` |
| `elasticsearch.nodeAffinity` | Value for the [node affinity settings](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature) | `{}` |
| `elasticsearch.podManagementPolicy` | By default Kubernetes [deploys statefulsets serially](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies). This deploys them in parallel so that they can discover eachother | `Parallel` |
| `elasticsearch.protocol` | The protocol that will be used for the readinessProbe. Change this to `https` if you have `xpack.security.http.ssl.enabled` set | `http` |
| `elasticsearch.httpPort` | The http port that Kubernetes will use for the healthchecks and the service. If you change this you will also need to set [http.port](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#_settings) in `extraEnvs` | `9200` |
| `elasticsearch.transportPort` | The transport port that Kubernetes will use for the service. If you change this you will also need to set [transport port configuration](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html#_transport_settings) in `extraEnvs` | `9300` |
| `elasticsearch.service.labels` | Labels to be added to non-headless service | `{}` |
| `elasticsearch.service.labelsHeadless` | Labels to be added to headless service | `{}` |
| `elasticsearch.service.type` | Type of elasticsearch service. [Service Types](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) | `ClusterIP` |
| `elasticsearch.service.nodePort` | Custom [nodePort](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport) port that can be set if you are using `service.type: nodePort`. | `` |
| `elasticsearch.service.annotations` | Annotations that Kubernetes will use for the service. This will configure load balancer if `service.type` is `LoadBalancer` [Annotations](https://kubernetes.io/docs/concepts/services-networking/service/#ssl-support-on-aws) | `{}` |
| `elasticsearch.service.httpPortName` | The name of the http port within the service | `http` |
| `elasticsearch.service.transportPortName` | The name of the transport port within the service | `transport` |
| `elasticsearch.updateStrategy` | The [updateStrategy](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets) for the statefulset. By default Kubernetes will wait for the cluster to be green after upgrading each pod. Setting this to `OnDelete` will allow you to manually delete each pod during upgrades | `RollingUpdate` |
| `elasticsearch.maxUnavailable` | The [maxUnavailable](https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget) value for the pod disruption budget. By default this will prevent Kubernetes from having more than 1 unhealthy pod in the node group | `1` |
| `elasticsearch.fsGroup (DEPRECATED)` | The Group ID (GID) for [securityContext.fsGroup](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) so that the Elasticsearch user can read from the persistent volume | `` |
| `elasticsearch.podSecurityContext` | Allows you to set the [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) for the pod | `fsGroup: 1000` `runAsUser: 1000` |
| `elasticsearch.securityContext` | Allows you to set the [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the container | `capabilities.drop:[ALL]` `runAsNonRoot: true` `runAsUser: 1000` |
| `elasticsearch.terminationGracePeriod` | The [terminationGracePeriod](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods) in seconds used when trying to stop the pod | `120` |
| `elasticsearch.sysctlInitContainer.enabled` | Allows you to disable the sysctlInitContainer if you are setting vm.max_map_count with another method | `true` |
| `elasticsearch.sysctlVmMaxMapCount` | Sets the [sysctl vm.max_map_count](https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html#vm-max-map-count) needed for Elasticsearch | `262144` |
| `elasticsearch.readinessProbe` | Configuration fields for the [readinessProbe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) | `failureThreshold: 3` `initialDelaySeconds: 10` `periodSeconds: 10` `successThreshold: 3` `timeoutSeconds: 5` |
| `elasticsearch.clusterHealthCheckParams` | The [Elasticsearch cluster health status params](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html#request-params) that will be used by readinessProbe command | `wait_for_status=green&timeout=1s` |
| `elasticsearch.imagePullSecrets` | Configuration for [imagePullSecrets](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-pod-that-uses-your-secret) so that you can use a private registry for your image | `[]` |
| `elasticsearch.nodeSelector` | Configurable [nodeSelector](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) so that you can target specific nodes for your Elasticsearch cluster | `{}` |
| `elasticsearch.tolerations` | Configurable [tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) | `[]` |
| `elasticsearch.ingress` | Configurable [ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) to expose the Elasticsearch service. See [`values.yaml`](./values.yaml) for an example | `enabled: false` |
| `elasticsearch.schedulerName` | Name of the [alternate scheduler](https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/#specify-schedulers-for-pods) | `nil` |
| `elasticsearch.masterTerminationFix` | A workaround needed for Elasticsearch < 7.2 to prevent master status being lost during restarts [#63](https://github.com/elastic/helm-charts/issues/63) | `false` |
| `elasticsearch.lifecycle` | Allows you to add lifecycle configuration. See [values.yaml](./values.yaml) for an example of the formatting. | `{}` |
| `elasticsearch.keystore` | Allows you map Kubernetes secrets into the keystore. See the [config example](/elasticsearch/examples/config/values.yaml) and [how to use the keystore](#how-to-use-the-keystore) | `[]` |
| `elasticsearch.rbac` | Configuration for creating a role, role binding and service account as part of this helm chart with `create: true`. Also can be used to reference an external service account with `serviceAccountName: "externalServiceAccountName"`. | `create: false` `serviceAccountName: ""` |
| `elasticsearch.podSecurityPolicy` | Configuration for create a pod security policy with minimal permissions to run this Helm chart with `create: true`. Also can be used to reference an external pod security policy with `name: "externalPodSecurityPolicy"` | `create: false` `name: ""` |
| `satellite.name` | Satellite deployment name | `satellite` |
| `satellite.replicas` | Satellite k8s deployment replicas | `1` |
| `satellite.enabled` | Is enable Satellite | `false` |
| `satellite.image.repository` | Satellite container image name | `skywalking.docker.scarf.sh/apache/skywalking-satellite` |
| `satellite.image.tag` | Satellite container image tag | `v0.4.0` |
| `satellite.image.pullPolicy` | Satellite container image pull policy | `IfNotPresent` |
| `satellite.antiAffinity` | Satellite anti-affinity policy | `soft` |
| `satellite.nodeAffinity` | Satellite node affinity policy | `{}` |
| `satellite.nodeSelector` | Satellite labels for pod assignment | `{}` |
| `satellite.tolerations` | Satellite tolerations | `[]` |
| `satellite.service.type` | Satellite svc type | `ClusterIP` |
| `satellite.ports.grpc` | Satellite grpc port for tracing, metrics, logs, events | `11800` |
| `satellite.ports.prometheus` | Satellite http port for Prometheus monitoring | `1234` |
| `satellite.resources` | Satellite node resources requests & limits | `{} - cpu limit must be an integer` |
| `satellite.podAnnotations` | Configurable [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) applied to all Satellite pods | `{}` |
| `satellite.env` | Satellite environment variables | `[]` |
| `satellite.securityContext` | Allows you to set the [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) for the pod | `fsGroup: 1000` `runAsUser: 1000` |

## 2.4 检查部署状态
执行如下命令，查看pod状态，如下表示pod都已经启动正常了
```bash
[root@master opt]# kubectl get pod -n skywalking
NAME                              READY   STATUS      RESTARTS   AGE
elasticsearch-master-0            1/1     Running     0          58m
skywalking-es-init-w9h2s          0/1     Completed   0          58m
skywalking-oap-6b84cd585c-qhblr   1/1     Running     0          58m
skywalking-ui-544bfb887b-p6qrh    1/1     Running     0          58m
[root@master opt]#
```

测试开发技术网站

博客