Send a multipart form-data request to the target service

[nouman93]

I have an endpoint which accepts a xlsx file as multipart form-data request to be sent on target server. I have checked the trace and the request looks like below:

----------------------------553612111828308003453474 Content-Disposition: form-data; name="file"; filename="test.xlsx" Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet PK!{��!z�[Content_Types].xml �(����n�0E�����1tQU�E� �&�$�my ߉y��x(�M�؞{�Lf<��:YA@�l&i_$`s��-3�=��=�IY�jg!@1�� g�p��LTD�EJ�+h�΃�…F�Rz�/T ������z�j��� ��)y_�����X��nϵ�L(�k�+b�re�?H���A�|ٰt�>��XPS�>&�)qb(�Qf��AwY��ae<>p�'���vq_�;�ѐLT�O�p�r]�s��y����%Je���~<�2�76���/� �1��y��(s���o]�(z�\�zJܽ�� ��>�Gj�G��ݫ��6��Y8ɱf;y�.;�w�ݕ�/�\s5~+s.�:���PK!^�e�_rels/.rels �(���MK1���!̽;�*"�l/E�Md�1��`7������FAt��z��w�y��f��x�{v�E ��fӻV�K����rF��H"l���3�*���>��⢄.%��uGVł=�\i8X�rZ�J�%\����P�4��H;s�>����67M�i�zoɥ#+��DΐY���B��5�V��$��~����"c'Z����k�RRF%������8���EsܙF|�0�2�Xn/ɢ�1�=c�W�7�޲���PK!�>����xl/_rels/workbook.xml.rels
----------------------------553612111828308003453474--

Now, i have a requirement to create another endpoint which accepts the JSON payload and transform it into the xlsx format and for that i have created a callout as well. I have also took some help from https://github.com/DinoChiesa/Apigee-Java-Simple-MultipartForm to create the multipart form-data request and now the request looks like below:

----------------------VT2NE72E0B9WGP Content-Disposition: form-data; name="test.xlsx"; filename="test.xlsx" Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet This is excel content ----------------------VT2NE72E0B9WGP--

I doubt that the request which i have prepared is not correct as the content inside it is not encoded as mentioned in the first trace. Kindly can someone tell me what i am missing out in this one so that it can work correctly.
Appreciate the help.

[dchiesa1]

---

@nouman93 wrote:

i have a requirement to create another endpoint which accepts the JSON payload and transform it into the xlsx format and for that i have created a callout as well. I have also took some help from https://github.com/DinoChiesa/Apigee-Java-Simple-MultipartForm to create the multipart form-data request and now the request looks like below:

---

What does it mean to transform the JSON into XLSX format?  An XLSX is actually a zip file - it is a compressed thing, with a bunch of constituent files within it.  If you rename a .xlsx file to .zip , you can open it and view those consituent pieces with a zipfile viewer or editor, like in Windows Explorer. 

So how are you producing this .xlsx file wihtin Apigee? 

I think you might have skipped that step, or ... somehow neglected to consider this aspect of the situation . That's why your second payload shows "this is excel content" instead of the gobbledegook you want, like in the first example. (The gobbledegook is just  the way the compressed bytestream can be represented as a string.  It looks like nonsense, but it's not). 

I think if I were doing this I would want to solve the problem in parts. The MultipartForm callout can invoke an endpoint that requires a multipart form request.  But that's just one part.  The other part is, producing the .xlsx content. What are you using for that?  Have you solved that part?  I think maybe I would do this with an External Callout, or an integration, or some other logic that runs externally to Apigee.  I think Excel365 actually has a REST API, and you might even be able to create the .xslx only by making rEST calls. (You would want to make a bunch of them, and for that you'd want Application Integration, or some other way to orchestrate those calls) . 

Stepping back, exactly what is the goal that requires you to "transform JSON to XLSX" ?  What is the real underlying problem you are trying to solve? 

[nouman93]

@dchiesa1 Thankyou for your response.
I have one target server which only accepts .xlsx file as a multipart form-data input request. Now, we thought to transform it in a way that we can create another flow inside the existing apigee proxy and that new flow will have some callouts which can transform the input JSON request into xlsx and i can convert the final output to a bytestream or anything similar then the 2nd callout will create the Multipart form data request and it will be forwarded to the target sever.
Additionally, I have written a code in JAVA using apache POI library which can convert the input JSON to xlsx and further i can fetch the bytestream as well from it so i think it might work if i put the same logic in a JAVA callout and store the bytestream in flow variable. For the 2nd part, i came across that repository of creating the Multipart form data request. So, what if i use that bytestream as an input for the multipart form-data creator callout and it will create a proper multipart form-data request which i can send to the target server. This was my strategy to resolve this problem. Will this work or still i might require some other things.
Best Regards

[dchiesa1]

ok I understand.

It's possible that embedding the XSLX-creation logic into an Apigee callout will work.  It's possible that it will not.

Apigee enforces restrictions  on the things a Java callout may do. For example it may not write to the filesystem, read from the filesystem, read system properties, perform reflection, and many other things .   As a result, you cannot be assured that any particular JAR will be loadable into an Apigee callout.  Apache has lots of  projects that produce JARs that will not load. An example is wss4j, which attempts to read a filesystem file upon initialization. If you try to use that in a callout, it will cause the callout to fail to initialize, and your proxy won't deploy. 

What does the POI jar do?  I don't know.  And of course any JAR that POI loads also is subject to the same restrictions. So what you are describing may work, and it may not. 

On the other hand, whether "it will work" may not be the primary criterion on which you decide whether to actually do this. To my mind, transforming a JSON of some shape, into an XSLX, is a good candidate for a microservice.  That should be contained in its own packaging, should be hosted independently, should run on its own.  Even if your Apigee proxy is the only caller currently, it is a good candidate for separation into a distinct host. If I were doing this I would use Cloud Run, and then set up Apigee to call into Cloud Run via ServiceCallout (or External Callout), using the GCP Authentication capability.  

This is a good pattern to follow - factor out logic like this into external systems and host them independently of Apigee. You may think, "what about PERFORMANCE??!  There's an EXTRA NETWORK HOP 😱 !"

But the GCP network is really fast.  If you're using Apigee X it's not going to be an issue.  

If you're using Apigee hybrid, then.. .you need a knative or similar to be able to host services easily.  And again, your cross-service latency should be completely acceptable.