we have deployed kafka on k8s cluster by using the Kafka helm chart by bitnami. internally it is working fine.
when we are trying to connect kafka which is deployed on EKS cluster from EMR which is running outside of cluster. we are not able to do it. we are getting error “Broker not available”. below is the config which we have:
- name: KAFKA_CFG_LISTENERS value: PLAINTEXT://$(MY_POD_IP):$(KAFKA_PORT_NUMBER) - name: KAFKA_CFG_ADVERTISED_LISTENERS value: PLAINTEXT://$(MY_POD_NAME).indv-v3.services:$(KAFKA_PORT_NUMBER)
here we have private hosted zone indv-v3.services on route 53.
any suggestion or guidance on how we need to configure Kafka cluster on K8s so that clients outside of K8s cluster would also be able to connect, produce and consume messages.