收集 kubernetes 节点应用程序日志

在 k8s-worker01 主机上安装 nginx 并收集其日志

主机安装 nginx 应用

1. 主机安装 nginx

wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
yum -y install nginx

2. nginx 默认内容

echo "worker1 web page" > /usr/share/nginx/html/index.html

3. 启动 nginx

systemctl enable nginx --now

4. 测试访问 nginx

> curl http://192.168.3.60
worker1 web page

编写 filebeat 资源清单文件

filebeat-to-logstash-nginx.yaml

cat > filebeat-to-logstash-nginx.yaml << "EOF"
apiVersion: v1
kind: ConfigMap
metadata:
  name: k8s-filebeat-config-nginx-logs
  namespace: default

data:
  filebeat.yml: |
    filebeat.inputs:
      - type: log
        paths:
          - /var/log/nginx/access.log
        fields:
          app: k8s
          type: module
        fields_under_root: true

      - type: log
        paths:
          - /var/log/nginx/error.log
        fields:
          app: k8s  
          type: module
        fields_under_root: true

    setup.ilm.enabled: false
    setup.template.name: "k8s-module"
    setup.template.pattern: "k8s-module-*"

    output.logstash:
      hosts: ['192.168.1.99:5055']
      enabled: true
      worker: 1
      compression_level: 3
      loadbalance: true

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: k8s-logs
  namespace: default
spec:
  selector:
    matchLabels:
      project: k8s
      app: filebeat
  template:
    metadata:
      labels:
        project: k8s
        app: filebeat
    spec:
      nodeName: k8s-worker01
      containers:
      - name: filebeat
        image: docker.io/elastic/filebeat:7.17.2
        imagePullPolicy: IfNotPresent
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 500Mi
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: filebeat-config
          mountPath: /etc/filebeat.yml
          subPath: filebeat.yml
        - name: nginx-access
          mountPath: /var/log/nginx/access.log
        - name: nginx-error
          mountPath: /var/log/nginx/error.log
      volumes:
      - name: nginx-access
        hostPath:
          path: /var/log/nginx/access.log
      - name: nginx-error
        hostPath:
          path: /var/log/nginx/error.log
      - name: filebeat-config
        configMap:
          name: k8s-filebeat-config-nginx-logs
EOF

编写 logstash 配置文件

1. nginx-logstash-to-elastic.conf

input {
  beats {
    host => "0.0.0.0"
    port => "5055"
  }
}

filter {

}

output {
    elasticsearch {
      index => "nginx-%{+YYYY.MM.dd}"
      hosts => ["http://192.168.1.99:9200"]
      user => "elastic"
      password => "datarc"
    }
}

重启 logstash

应用 filebeat 资源清单文件

kubectl apply -f filebeat-to-logstash-nginx.yaml

验证结果

kubectl get pods -n kube-system -o wide

在 kibana 界面添加索引