LinkedIn
Twitter
I have a requirement to transfer XML files over FTP to a file server in Apigee X. So this Api proxy should execute once daily to transfer the files.
Is it possible to do it in Apigee X using some node.js script like setting a crone job?Would something like below work?
What would be your suggestion do it in Apigee X itself or use a cloud scheduler to trigger the Api proxy once daily to do the transfer?
Please let me know a feasible solution having the performance issue in mind as the file sizes are very large(>10 MB)
I have a requirement to transfer XML files over FTP to a file server in Apigee X. So this Api proxy should execute once daily to transfer the files.
Apigee doesn't have a scheduler. It's not the right tool for that requirement. It might play a role, but it's not the solution.
Theres a Google Cloud service called Cloud Scheduler that does the job of "executing something on a schedule". Actually it does not "run a job" in the general sense. The options are: invoke an HTTP endpoint, or publish to aPubSub topic. (docs). So you would still need some hosting system, something that runs your app. The Scheduler merely triggers it.
But you said something beyond "schedule a job" - you said you want to transfer files. I am guessing that might mean (a) pick up the files from some filesystem, (b) do some transform or validation, (c) PUT the files via FTP to some server. And if you are running in an enterprise, then there are a bunch of non-functional concerns, like security: how does the job authenticate to the filesystem and how does the job authenticate to FTP. or monitoring: was the job successful? How many times has it failed over the past 30 days? and etc.
To address those kinds of concerns, you can use an integration platform , that is built for this sort of thing. Google offers Application Integration, which...:
- has a scheduler built in - you can define integrations to run on a schedule.
- includes FTP connectors, as well as connectors to various datastores and other applications
- Can transform files including XML to XML (XSLT) or other formats
- Can run JavaScript logic as part of an execution
- allows for manual approval steps
- has the ability to manage credentials for a variety of enterprise systems
- will log all of its executions
In other words, it's an integration platform as service (iPaaS). Worth a look.
This is the canvas for an integration I built yesterday which.... picks up files from a Google Cloud Storage bucket, validates them, transforms them (splits into batches), then sends one email for each message in the batch.
There's also a Community space dedicated to Integration: here.
