This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
- Google Cloud
- Cloud Forums
- ⚡Community Hub
- ApacheBeamRunJavaPipeline composer/airflow Operato...
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
LinkedIn
Twitter
Post Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Reply posted on
--/--/---- --:-- AM
Post Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am trying to run a dataflow java job that runs perfectly fine on dataflow runner when submitted without composer. The same job when tried from composer using dataflowrunner, some how composer is executing Kafka connection in airflow worker host rather than running from dataflow worker host. we only have firewall open from dataflow worker host and currently getting error fetching topic meta data timeout exception in composer.
java.lang.RuntimeException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
Is there any way to bypass jar execution on the airflow worker and only run the job from dataflow workers?
0
0
322
Topic Labels
- Labels:
-
Apache Beam
-
Cloud Composer
-
Cloud DataFlow
-
Kafka
0 REPLIES 0
Top Labels in this Space
-
0auth2.0
1 -
ABAP SDK
1 -
access
2 -
accesstoken
1 -
Account
4 -
adf
1 -
Administrator
1 -
Advice
1 -
Agent
1 -
AI & Machine Learning
5 -
AI ML General
7 -
Analytics General
5 -
android
1 -
Android Management API
1 -
Announcements
1 -
Apache Beam
1 -
API
30 -
API Gateway
5 -
API Key
4 -
API Security
1 -
API Verification
3 -
Apigee
2 -
Apigee General
2 -
App Dev General
1 -
App Development
1 -
App Engine
3 -
Appeals
1 -
archive
1 -
Artifact Registry
4 -
attemptResponseLog
1 -
attempts
1 -
Authentication
2 -
authorization code
1 -
Automation
2 -
AWS
1 -
basics
1 -
Batch
3 -
beginer
1 -
Best Practices
2 -
Beyond Corp Enterprise
2 -
Bi
1 -
BigQuery
14 -
Bill
1 -
Billing
214 -
billing account
9 -
bot
1 -
buckets
2 -
Bug
2 -
Business Intelligence
2 -
Cant Update Console
1 -
Career Development
2 -
Certification
2 -
Certification Announcements
1 -
CLI
2 -
Cloud
14 -
Cloud Build
5 -
Cloud Code
5 -
Cloud community
1 -
Cloud Composer
1 -
cloud console
2 -
Cloud cost management
1 -
Cloud DataFlow
1 -
Cloud Deploy
6 -
Cloud Developer
2 -
Cloud Digital Leader
2 -
Cloud DLP
1 -
Cloud DNS
2 -
Cloud Endpoint
1 -
Cloud Error Reporting
10 -
Cloud Firewall
2 -
cloud function
1 -
Cloud Functions
1 -
Cloud Identity
2 -
Cloud Logging
5 -
Cloud Monitoring
4 -
Cloud Natural Language API
2 -
Cloud News
1 -
Cloud Profiler
3 -
Cloud PubSub
1 -
Cloud Run
5 -
Cloud SDK
3 -
Cloud Source Repositories
2 -
Cloud SQL for MySQL
3 -
Cloud SQL for Postgres
2 -
Cloud Storage
10 -
cloud tasks
2 -
Cloud Trace
1 -
Cloud Vision ApI
1 -
Cloud VPN
1 -
Cloud Workstations
2 -
Cloud-native
1 -
clusters
1 -
Colab
1 -
Commercial
1 -
commitment used discount
1 -
Community
2 -
Community Challenges
3 -
Comply with domain verification
1 -
Compute
3 -
Compute Engine
9 -
Connectivity
1 -
Console
4 -
Contact Center AI
1 -
Container Registry
3 -
contains active resources
1 -
contract
1 -
copyright
1 -
Cost Optimization
3 -
country
1 -
Credits
2 -
cud
1 -
Customize Email Template
1 -
Dask
1 -
Data
1 -
Data Analytics
4 -
Data Catalog
1 -
Data Engineer
1 -
Data Transfer
1 -
Database
3 -
Database Migration Service
1 -
Dataform
1 -
Datamesh
1 -
Dataplex
1 -
Dataproc
1 -
Datastream
2 -
delete
1 -
deleted
2 -
deleting Google workspace account
1 -
Developer Portal
1 -
Devices
1 -
Dialogflow
3 -
Digital Badging
1 -
display
1 -
DLP
1 -
document creation
1 -
Documentation
37 -
domain
2 -
doubleclickbidmanager
2 -
DR
1 -
Drive
2 -
Duplicate
1 -
dv360
1 -
Email Logs
1 -
emails
1 -
Embedding
1 -
EMM
1 -
error
5 -
etc.
1 -
Events
3 -
expiration
2 -
Feedback
1 -
Filestore
1 -
filter
1 -
firebase
8 -
firebase hosting
2 -
Firestore
4 -
flask
1 -
Free tier
6 -
Fundamental
2 -
GCDS
1 -
gcloud
1 -
GCP
42 -
GCP Billing Account
20 -
GCP Billing Acount
5 -
gcp co
1 -
GCP Console
17 -
GCP Homepage url
1 -
GCP Project
3 -
gcs
1 -
Gemini
1 -
Gen App Builder
1 -
General Discussion
228 -
General Miscellaneous
3 -
Generative AI Studio
2 -
Geocoding
2 -
Geographic
1 -
GKE
6 -
GKE Enterprise
1 -
Gmail
2 -
google actions
1 -
Google AI Studio
2 -
Google API
3 -
Google App Engine
4 -
Google App Script
1 -
Google Chat
1 -
google cloud
2 -
Google Cloud Arcade Swag
1 -
Google Cloud Function
1 -
Google Cloud Next
3 -
google cloud oauth
1 -
Google Cloud Platform
29 -
google cloud project
1 -
Google Cloud signup
2 -
google cloud verifaction
2 -
Google Console
4 -
Google Experts
1 -
Google Forms API
1 -
google home
1 -
Google Identity Platform
4 -
Google Kubernetes Engine (GKE)
1 -
Google Maps
5 -
google sheet
1 -
Google Sheets
2 -
Google Sign-In
1 -
google street view
1 -
Google Translate
1 -
google vision
1 -
google workspace
4 -
Google Workspace Marketplace
3 -
google-app-engine
1 -
google-cloud-platform
1 -
Google_Client
1 -
googlecommunity
4 -
Groups
1 -
grpc
2 -
gsc
1 -
guarantees
1 -
guest attributes
1 -
homegraph
2 -
IAM
4 -
iam access
2 -
iap
1 -
Ideas
1 -
Identify
1 -
Identity & Access Management
9 -
Infrastructure as Code
2 -
Infrastructure General
3 -
Innovators
2 -
Innovators Help
47 -
Integration
1 -
Internet of Things
1 -
Introduction
20 -
Introductions
53 -
invalid
1 -
IoT
3 -
IoTCore
3 -
ip
1 -
issues
1 -
jarvis
1 -
JWT
2 -
Kafka
1 -
labels
3 -
Labs Support & Troubleshooting
3 -
Learn to Earn
3 -
Learning
2 -
Legacy
1 -
licence
1 -
liens
1 -
liens delete
1 -
limitations
1 -
List API
1 -
llo
1 -
load balancer
2 -
Location intelligence
1 -
logging logs
2 -
Login
1 -
Low-code
1 -
Managed Service for Prometheus
1 -
Mapping
1 -
Marketplace
4 -
me-central2
1 -
memorystore
1 -
Memorystore for Memcached
1 -
Memorystore for Redis
1 -
MERN Stack
1 -
metadata
2 -
Migration
4 -
MQTT
2 -
Myself
1 -
need to
2 -
Networking
3 -
New idea
1 -
News & Events
17 -
nginx
1 -
node.js
2 -
oauth
19 -
OAuth API verification
11 -
oauth app verification
1 -
oauth consent screen
14 -
oauth verifaction
1 -
oauth verifaction request
1 -
OCR
1 -
odoo
1 -
Oh
1 -
Open Source
5 -
Organization
5 -
Other
3 -
P1 Bug
1 -
pakdeslot
1 -
partneradvantageportal
1 -
payment
3 -
Payments
1 -
Payments Profile
1 -
PCA
1 -
PCF
1 -
pdf
1 -
people
1 -
Perspective API
1 -
Phishing
1 -
pricing
3 -
Private Service Connection
1 -
Prizes & Swag
1 -
Product Availability
27 -
Professional
1 -
Professional Cloud Network Engineer
1 -
Professional Collaboration Engineer
1 -
project
2 -
project delete
2 -
Projects
9 -
Promotion Codes
1 -
python
6 -
Quotas
3 -
rails
1 -
RDP
1 -
React Js
1 -
Recommendations AI
1 -
recovery
1 -
redirect_uri_mismatch
1 -
Redis
1 -
refresh token
2 -
refreshtoken
1 -
registration error
1 -
Research
1 -
response
1 -
restore billing account
1 -
results
1 -
retail
2 -
Retail API
1 -
retries
1 -
rickroll
1 -
ripple
1 -
roles
1 -
ruby
1 -
rust
1 -
SaaS
1 -
saml
1 -
Scheduler
1 -
Scope
1 -
Scopes Approval
1 -
screenshot
1 -
Search
3 -
Security
2 -
security and guarantees
1 -
Security Command Center
2 -
Security Keys
3 -
sending
1 -
Sensitive scope verification
1 -
Serverless
1 -
Service Account
3 -
Service Directory
1 -
session
1 -
setting
1 -
setup
1 -
Shared VPC
1 -
Signup Error
1 -
Similar items
1 -
SLA
1 -
Speech-to-Text
1 -
sports
1 -
Sportsradar API
1 -
Spring Boot
1 -
static
2 -
static website
3 -
Status
1 -
Storage
3 -
storage class
1 -
Study Jam
1 -
Subject Matter Experts
3 -
subscription
1 -
Success Stories
9 -
Support
2 -
Support Team
1 -
survey
1 -
Suspension
1 -
system-gsuite
1 -
task
2 -
Tasks
1 -
Tensorflow Enterprise
1 -
terminal
2 -
Terraform
1 -
text detection
1 -
Tips & Tricks
56 -
token
1 -
traffic director
1 -
Training
3 -
Translation AI
1 -
translation hub
1 -
translator
1 -
trust and safety team
9 -
UEFI
1 -
unable
1 -
unexisting
1 -
unknown
1 -
unkown
1 -
Unverified
1 -
Urgent
1 -
Use Cases
47 -
USFL
1 -
verification
3 -
Verify App
1 -
Vertex AI Model Registry
1 -
Vertex AI Platform
3 -
Vertex AI Workbench
2 -
Video AI
1 -
Vision AI
2 -
VM
3 -
VPC
1 -
webpage
2 -
WebRisk
1 -
website
6 -
website cannot be reached
1 -
website offline
1 -
wgapp-cto
1 -
wif
1 -
wordpress
4 -
Workflows
2 -
Workload Manager
1 -
Workspace General
1
- « Previous
- Next »
