Error while creating topics automatically on Managed Kafka

[md-gd]

Hello,

We deployed managed kafka from GCP and run couple of tests on it.

Currently we stuck into the error  which blocks automatic topic creation

Error creating NewConsumerGroup, kafka client has run out of available brokers to talk unexpected EOF

Manual creation works fine.

What we tried

First assumed that automatic topic creation like is set to false, but its not. Figured out that by using external UI.

Is there anything else that could block it?

[marckevin]

Hi @md-gd,

Welcome to Google Cloud Community!

The “Error creating NewConsumerGroup, kafka client has run out of available brokers to talk unexpected EOF” usually suggest an issue during establishment, configuration or connection to Kafka brokers, which is possibly the reason why automatic topic creation still fails even when enabled and manual creation works fine.

Here are some suggestions that may help resolve the issue:

• Connectivity: Ensure there are no issues with your connectivity, including checking your Firewalls, VPC setting if applicable, Broker Availability, and general network problems.
• Client Configuration: Check client configuration, including ensuring bootstrap servers are properly listed, Security protocols, and ensure that your Kafka client version is compatible with the Kafka brokers you're using.
• Authentication and Access Control: Check if authentication and access control are properly configured with necessary credentials and permissions in Google Cloud Managed Service for Apache Kafka.
• Limitations: Check for any limitation in Google Cloud Managed Service for Apache Kafka including region and zones, broker configurations, resource quotas and limitations on the managed Kafka service, etc.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.