https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/index.html Install Kafka Broker Display Broker Message Kafka Broker Invoke ISVC Knative Broker 是 Knative Eventing 系统的核心组件，它的主要作用是充当事件路由和分发的中枢，在事件生产者（事件源）和事件消费者（服务）之间提供解耦、可靠的事件传输。 以下是 Knative Broker 的关键作用详解： 事件接收中心： Broker 是事件流汇聚的入口点。各种事件源（如 Kafka 主题、HTTP 源、Cloud Pub/Sub、GitHub Webhooks、定时器、自定义源等）将事件发送到 Broker。 事件生产者只需知道 Broker 的地址，无需关心最终有哪些消费者或消费者在哪里。 事件存储与缓冲： Broker 通常基于持久化的消息系统实现（如 Apache Kafka, Google Cloud Pub/Sub, RabbitMQ, NATS Streaming 或内存实现 InMemoryChannel）。这提供了： 持久化： 确保事件在消费者处理前不会丢失（取决于底层通道实现）。 缓冲： 当消费者暂时不可用或处理速度跟不上事件产生速度时，Broker 可以缓冲事件，避免事件丢失或压垮生产者/消费者。 重试： 如果消费者处理事件失败，Broker 可以重新投递事件（通常需要结合 Trigger 和 Subscription 的重试策略）。 解耦事件源和事件消费者： 这是 Broker 最重要的作用之一。事件源只负责将事件发送到 Broker，完全不知道有哪些服务会消费这些事件。 事件消费者通过创建 Trigger 向 Broker 声明它对哪些事件感兴趣。消费者只需知道 Broker 的存在，无需知道事件是从哪个具体源产生的。 Hugo en Thu, 07 Mar 2024 15:00:59 +0800 https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/install_steps/index.html Thu, 07 Mar 2024 15:00:59 +0800 https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/install_steps/index.html About Source, curl, kafkaSource, Broker Trigger Sink: ksvc, isvc Install a Channel (messaging) layer kubectl apply -f https://github.com/knative-extensions/eventing-kafka-broker/releases/download/knative-v1.18.0/eventing-kafka-controller.yaml Expectd Output configmap/kafka-broker-config created configmap/kafka-channel-config created customresourcedefinition.apiextensions.k8s.io/kafkachannels.messaging.knative.dev created customresourcedefinition.apiextensions.k8s.io/consumers.internal.kafka.eventing.knative.dev created customresourcedefinition.apiextensions.k8s.io/consumergroups.internal.kafka.eventing.knative.dev created customresourcedefinition.apiextensions.k8s.io/kafkasinks.eventing.knative.dev created customresourcedefinition.apiextensions.k8s.io/kafkasources.sources.knative.dev created clusterrole.rbac.authorization.k8s.io/eventing-kafka-source-observer created configmap/config-kafka-source-defaults created configmap/config-kafka-autoscaler created configmap/config-kafka-features created configmap/config-kafka-leader-election created configmap/kafka-config-logging created configmap/config-namespaced-broker-resources created configmap/config-tracing configured clusterrole.rbac.authorization.k8s.io/knative-kafka-addressable-resolver created clusterrole.rbac.authorization.k8s.io/knative-kafka-channelable-manipulator created clusterrole.rbac.authorization.k8s.io/kafka-controller created serviceaccount/kafka-controller created clusterrolebinding.rbac.authorization.k8s.io/kafka-controller created clusterrolebinding.rbac.authorization.k8s.io/kafka-controller-addressable-resolver created deployment.apps/kafka-controller created clusterrole.rbac.authorization.k8s.io/kafka-webhook-eventing created serviceaccount/kafka-webhook-eventing created clusterrolebinding.rbac.authorization.k8s.io/kafka-webhook-eventing created mutatingwebhookconfiguration.admissionregistration.k8s.io/defaulting.webhook.kafka.eventing.knative.dev created mutatingwebhookconfiguration.admissionregistration.k8s.io/pods.defaulting.webhook.kafka.eventing.knative.dev created secret/kafka-webhook-eventing-certs created validatingwebhookconfiguration.admissionregistration.k8s.io/validation.webhook.kafka.eventing.knative.dev created deployment.apps/kafka-webhook-eventing created service/kafka-webhook-eventing created kubectl apply -f https://github.com/knative-extensions/eventing-kafka-broker/releases/download/knative-v1.18.0/eventing-kafka-channel.yaml Expectd Output configmap/config-kafka-channel-data-plane created clusterrole.rbac.authorization.k8s.io/knative-kafka-channel-data-plane created serviceaccount/knative-kafka-channel-data-plane created clusterrolebinding.rbac.authorization.k8s.io/knative-kafka-channel-data-plane created statefulset.apps/kafka-channel-dispatcher created deployment.apps/kafka-channel-receiver created service/kafka-channel-ingress created Install a Broker layer kubectl apply -f https://github.com/knative-extensions/eventing-kafka-broker/releases/download/knative-v1.18.0/eventing-kafka-broker.yaml Expectd Output configmap/config-kafka-broker-data-plane created clusterrole.rbac.authorization.k8s.io/knative-kafka-broker-data-plane created serviceaccount/knative-kafka-broker-data-plane created clusterrolebinding.rbac.authorization.k8s.io/knative-kafka-broker-data-plane created statefulset.apps/kafka-broker-dispatcher created deployment.apps/kafka-broker-receiver created service/kafka-broker-ingress created Reference for more information, you can check 🔗https://knative.dev/docs/eventing/brokers/broker-types/kafka-broker/ https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/display_broker_msg/index.html Thu, 07 Mar 2024 15:00:59 +0800 https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/display_broker_msg/index.html Flow flowchart LR A[Curl] -->|HTTP| B{Broker} B -->|Subscribe| D[Trigger1] B -->|Subscribe| E[Trigger2] B -->|Subscribe| F[Trigger3] E --> G[Display Service] Setps 1. Create Broker Setting kubectl apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: kafka-broker-config namespace: knative-eventing data: default.topic.partitions: "10" default.topic.replication.factor: "1" bootstrap.servers: "kafka.database.svc.cluster.local:9092" #kafka service address default.topic.config.retention.ms: "3600" EOF 2. Create Broker kubectl apply -f - <<EOF apiVersion: eventing.knative.dev/v1 kind: Broker metadata: annotations: eventing.knative.dev/broker.class: Kafka name: first-broker namespace: kserve-test spec: config: apiVersion: v1 kind: ConfigMap name: kafka-broker-config namespace: knative-eventing EOF deadletterSink: https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/kafka_broker_invoke_isvc/index.html Thu, 07 Mar 2024 15:00:59 +0800 https://ops.docs.72602.space/kubernetes/serverless/knative/eventing/broker/kafka_broker_invoke_isvc/index.html 1. Prepare RBAC create cluster role to access CRD isvc kubectl apply -f - <<EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kserve-access-for-knative rules: - apiGroups: ["serving.kserve.io"] resources: ["inferenceservices", "inferenceservices/status"] verbs: ["get", "list", "watch"] EOF create rolebinding and grant privileges kubectl apply -f - <<EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kafka-controller-kserve-access roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kserve-access-for-knative subjects: - kind: ServiceAccount name: kafka-controller namespace: knative-eventing EOF 2. Create Broker Setting kubectl apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: kafka-broker-config namespace: knative-eventing data: default.topic.partitions: "10" default.topic.replication.factor: "1" bootstrap.servers: "kafka.database.svc.cluster.local:9092" #kafka service address default.topic.config.retention.ms: "3600" EOF 3. Create Broker kubectl apply -f - <<EOF apiVersion: eventing.knative.dev/v1 kind: Broker metadata: annotations: eventing.knative.dev/broker.class: Kafka name: isvc-broker namespace: kserve-test spec: config: apiVersion: v1 kind: ConfigMap name: kafka-broker-config namespace: knative-eventing delivery: deadLetterSink: ref: apiVersion: serving.knative.dev/v1 kind: Service name: event-display EOF 4. Create InferenceService Reference you can create isvc first-tourchserve service, by following 🔗link