服务的相关链接 软件开源连接Git hub
软件官方说明文档
官方软件下载包
docker hub 参考文档
kafka-eagel服务部署完日志会有报错 1 DruidDataSource.Druid-ConnectionPool-Create-2037200746 - ERROR - create connection error, url: jdbc:mysql://mysql:3306/kafkaeagle?useUnicode=true&characterEncoding=UTF8, errorCode 0, state 08001 com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
需要使用MYSQL5.6的版本，以下是mysql的yaml部署文件 service.yaml文件做暴露服务端口，type类型根据需求定义 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 apiVersion: v1 kind: Service metadata: labels: app: my-mysql name: my-mysql spec: type: NodePort ports: - port: 3306 nodePort: 30006 protocol: TCP targetPort: 3306 selector: app: my-mysql
secret.yaml文件做数据库的用户名密码 1 2 3 4 5 6 7 8 apiVersion: v1 kind: Secret metadata: name: my-mysql type: Opaque data: mysql-root-password: eWRhdGE= mysql-password: eWRhdGE=
Pvc.yaml做mysql的动态存储 1 2 3 4 5 6 7 8 9 10 11 12 13 apiVersion: v1 kind: PersistentVolumeClaim metadata: name: my-mysql labels: app: my-mysql spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi storageClassName: course-nfs-storage
deployment.yaml应用部署文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 apiVersion: apps/v1 kind: Deployment metadata: name: my-mysql labels: app: my-mysql spec: selector: matchLabels: app: my-mysql strategy: type: RollingUpdate template: metadata: labels: app: my-mysql spec: containers: - image: mysql:5.6 name: my-mysql args: - "--ignore-db-dir=lost+found" env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: key: mysql-root-password name: my-mysql - name: MYSQL_PASSWORD valueFrom: secretKeyRef: key: mysql-password name: my-mysql - name: MYSQL_USER value: kafka ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysql-persistent-storage mountPath: /var/lib/mysql volumes: - name: mysql-persistent-storage persistentVolumeClaim: claimName: my-mysql
需要创建数据库使用mysql作为源数据存放,需要先创建ke数据库，并赋权，没有kafka认证 1 2 3 4 5 6 # 创建用户和库 create database ke character set 'UTF8'; create user kafka identified by 'kafka'; grant all privileges on ke.* to 'kafka'@'%' identified by 'kafka' with grant option; grant all privileges on ke.* to 'kafka'@'localhost' identified by 'kafka' with grant option; flush privileges;
完成mysql5.6数据库部署，并创建库和授予权限，那就开始部署服务，以下是eagel的yaml文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 --- apiVersion: apps/v1 kind: Deployment metadata: labels: k8s.eip.work/layer: web k8s.eip.work/name: kafka-eagle name: kafka-eagle namespace: qb-kafka spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s.eip.work/layer: web k8s.eip.work/name: kafka-eagle strategy: rollingUpdate: maxSurge: 25 % maxUnavailable: 25 % type: RollingUpdate template: metadata: labels: k8s.eip.work/layer: web k8s.eip.work/name: kafka-eagle spec: containers: - env: - name: ZKSERVER01 value: 'kafka-1589357149-zookeeper-headless:2181' - name: DBTYPE value: mysql - name: DBUSERNAME value: root - name: DBPASSWORD value: ydata - name: DBURL value: 'my-mysql.qb-kafka:3306' image: 'wyfsxs/kafka-eagle:v1.4' imagePullPolicy: Always name: kafka-eagle dnsPolicy: ClusterFirst imagePullSecrets: - name: aliyun restartPolicy: Always schedulerName: default-scheduler terminationGracePeriodSeconds: 30 --- apiVersion: v1 kind: Service metadata: annotations: k8s.eip.work/workload: kafka-eagle labels: k8s.eip.work/layer: web k8s.eip.work/name: kafka-eagle name: kafka-eagle namespace: qb-kafka spec: externalTrafficPolicy: Cluster ports: - name: bz2ppr nodePort: 30048 port: 8048 protocol: TCP targetPort: 8048 selector: k8s.eip.work/layer: web k8s.eip.work/name: kafka-eagle sessionAffinity: None type: NodePort
1 2 3 4 5 6 7 8 9 10 参数说明： ZKSERVER01 kafka集群1的zook ZKSERVER02 kafka集群2的zook DBTYPE 数据库类型 空是sqllite，可以是mysql DBUSERNAME mysql的话需要填，用户名 DBPASSWORD mysql的话需要填，密码 DBURL mysql的话需要填，数据库地址 到端口 127.0.0.1:3306 EAGEPORT 工程的端口，空的话，默认8048 KERBEROSFLAG 是否开启kafka安全，保留字段 KERBEROSDIR kafka keytab的地址，保留字段
一些需要输入的参数环境 Dockerfile也可以自己定制 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 FROM frolvlad/alpine-oraclejdk8:slim ENV ZKSERVER01="" ENV ZKSERVER02="" ENV DBTYPE="" ENV DBUSERNAME="" ENV DBPASSWORD="" ENV DBURL="" ENV EAGEPORT="" ENV KERBEROSFLAG="" ENV KERBEROSDIR="" ADD kafka-eagle /opt/kafka-eagle ENV KE_HOME=/opt/kafka-eagle WORKDIR /opt/kafka-eagle VOLUME /opt/kafka-eagle CMD /bin/sh /opt/kafka-eagle/bin/entrypoint.sh -a "${ZKSERVER01}" -b "${ZKSERVER02}" -c "${DBTYPE}" -d "${DBUSERNAME}" -e "${DBPASSWORD}" -f "${DBURL}" -p "${EAGEPORT}" -s "${KERBEROSFLAG}" -q "${KERBEROSDIR}"
