API Proxies can not be created using the java apigee client

[medbayoudh]

Hey guys, 

We are integrating apigee into our platform and we want to use it programatically. The idea is to import API Proxies based on a predefined template, create the zip bundle, get it in bytes format and send it to the API. Unfortunately I am facing some problems when I am trying to create an API Proxy using the java apigee client like:

"message": "invalid multipart/form-data encoding: failed to read multipart body part: multipart: NextPart: bufio: buffer full"

Here the code which is responsible for the creation

      byte[] zipBytes = byteArrayOutputStream.toByteArray();
      String boundary = UUID.randomUUID().toString();

      MultipartContent multipartContent = new MultipartContent(boundary);
      HttpHeaders headers = new HttpHeaders();
      headers.set("Content-Disposition",
          String.format("form-data; name=\"file\"; filename=\"%s.zip\"",
              apiSpecName.toLowerCase()));
      Part part = new Part(headers,
          new ByteArrayContent("application/octet-stream", zipBytes));
      multipartContent.setParts(Collections.singleton(part));
      part.getHeaders().setAcceptEncoding("none");
      
      ByteArrayOutputStream multiPartContentAsByteArray = new ByteArrayOutputStream();
      multipartContent.writeTo(multiPartContentAsByteArray);

      // Create the GoogleApiHttpBody with standard Base64-encoded data
      GoogleApiHttpBody body = new GoogleApiHttpBody()
          .setContentType("application/octet-stream")
          .encodeData(multiPartContentAsByteArray.toByteArray());

      // Create the API proxy creation request
      Create create = apigee
          .organizations()
          .apis()
          .create(properties.getGoogleOrganization(), body)
          .setAction("import")
          .set("name", apiSpecName.toLowerCase());

      create
          .getRequestHeaders()
          .setContentType("multipart/form-data; boundary=".concat(boundary));

      // Execute the request
      final GoogleCloudApigeeV1ApiProxyRevision apiProxyRevision = create.execute();

Notes:

• In my implementation I relied on those documentations:

  • https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.apis/create

  • https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.environments.keysto...

• Tried first of all to set the zip bytes using GoogleApiHttpBody#encodeData directly but also it was not successful since the boundary is not transmitted within the body(The error message was the following). That's why I was searching for a native way to add the boundary to the body through MultiPartContent.  

  "message": "invalid multipart/form-data encoding: failed to read multipart body part: multipart: NextPart: EOF"​

• In the official documentation is mentioned that the create request should have a ContentType: multipart/form-data which imposes indirectly having a boundary, otherwise the API does not accept the body
• we are using the following apigee java library: 

  com.google.apis:google-api-services-apigee:v1-rev20240919-2.0.0

• A curl command in the terminal was able to create the proxy successfully

curl "https://apigee.googleapis.com/v1/organizations/XXXXXX/apis?name=XX&action=import" \
  --trace-ascii curl_trace.txt -X POST \
  -H "Authorization: Bearer XXXX" \
  -H "Content-type: multipart/form-data" \
  -F "file=@test.zip"
​

---------------------------------
0000: POST /v1/organizations/XXXX/apis?name=XXX&action=import HTTP/2
0059: Host: apigee.googleapis.com
0076: User-Agent: curl/8.7.1
008e: Accept: */*
009b: Authorization: Bearer XXXXX
0198: Content-Length: 3113
01ae: Content-Type: multipart/form-data; boundary=--------------------
01ee: ----J4KcMnAiAbc9Sbb4hyiBV4
020a: 
=> Send data, 3113 bytes (0xc29)
0000: --------------------------J4KcMnAiAbc9Sbb4hyiBV4
0032: Content-Disposition: form-data; name="file"; filename="test.zip"
0074: Content-Type: application/octet-stream
009c: 
009e: PK.........d;Y................apiproxy/targets/default.xml...N.1
00de: ...{.(;.l....P10.N........!....I.....b..g.lYo?N^.AdG........Y..^
011e: ........Z......r;tz6....m .I.9e...Y}.C'.~.^..)....+..z..=...i..J
015e: .'DX.P1.5F...kS.....cJ...:....6..*V....z$Neq.."ecO...'.-9.2.+...
019e: ."..U.ONm.oT..8t_PK....Lq........PK.........d;Y................a
01de: piproxy/policies/FC-generate-access-token.xmlu.AN.0.E.9.5{....v$

When debugging we found that the following MultiPartContent is sent from our side, which is pretty similar to the successful curl trace but unfortunately still not creating the API Proxy 

--ed6b7126-e473-43f9-89b0-dab51228aa30
Accept-Encoding: none
Content-Length: 2901
Content-Type: application/octet-stream
content-disposition: form-data; name="file"; filename="test.zip"
content-transfer-encoding: binary

PK�;Yapiproxy/targets/default.xml���N1��{�(;
l�P10�N��.����!����I����b��g�lYo?N^�AdG�˛͵�Y��^���խ�Z�	����r;tz6��m �I�9e�‹Y}�C'�~^�)�չ�+........
--ed6b7126-e473-43f9-89b0-dab51228aa30--

We will be happy if we can get some hints from you guys how we can fix this.

Thanks in advance

Mohamed Bayoudh, apinity GmbH

[nmarkevich · Accepted solution]

Hi @medbayoudh,

Here's a working example that could help you solve the issue:

package com.example;

import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.ByteArrayContent;
import com.google.api.client.http.HttpHeaders;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.MultipartContent;
import com.google.api.client.http.MultipartContent.Part;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.apigee.v1.Apigee;
import com.google.api.services.apigee.v1.model.GoogleCloudApigeeV1ApiProxyRevision;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;

import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.GeneralSecurityException;
import java.util.Collections;
import java.util.UUID;

public class ApigeeProxyImporter {

    // Path to the service account JSON file
    private static final String SERVICE_ACCOUNT_JSON = "PATH_TO_SERVICE_ACCOUNT_JSON";
    // Your Apigee organization name
    private static final String ORGANIZATION = "ORGANIZATION/PROJECT_NAME"; 

    public static void main(String[] args) throws GeneralSecurityException, IOException {
        // Configure authentication and create Apigee client
        Apigee apigeeService = createApigeeService();

        // Path to the zip file containing the proxy
        String zipFilePath = "PATH_TO_PROXY_ZIP_FILE";
        // Name of the proxy to be created or updated
        String proxyName = "example-proxy";

        // Import the proxy
        importProxy(apigeeService, ORGANIZATION, proxyName, zipFilePath);
    }

    // Method to create an Apigee client using GoogleCredentials and GsonFactory
    private static Apigee createApigeeService() throws GeneralSecurityException, IOException {
        // Read credentials from the service account file
        GoogleCredentials credentials;
        try (FileInputStream serviceAccountStream = new FileInputStream(SERVICE_ACCOUNT_JSON)) {
            credentials = ServiceAccountCredentials.fromStream(serviceAccountStream)
                    .createScoped("https://www.googleapis.com/auth/cloud-platform");
        }

        // Create the transport layer and JSON factory
        HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
        JsonFactory jsonFactory = GsonFactory.getDefaultInstance();

        // Create Apigee client using HttpCredentialsAdapter for credential handling
        return new Apigee.Builder(httpTransport, jsonFactory, new HttpCredentialsAdapter(credentials))
                .setApplicationName("Apigee Proxy Importer")
                .build();
    }

    // Method to import a proxy
    private static void importProxy(Apigee apigeeService, String organization, String proxyName, String zipFilePath) {
        try {
            // Read the zip file into a byte array
            byte[] zipBytes = Files.readAllBytes(Paths.get(zipFilePath));

            // Generate a boundary for multipart/form-data
            String boundary = UUID.randomUUID().toString();

            // Create MultipartContent using the boundary
            MultipartContent multipartContent = new MultipartContent(boundary);

            // Create headers for the content part
            HttpHeaders headers = new HttpHeaders();
            headers.set("Content-Disposition", String.format("form-data; name=\"file\"; filename=\"%s.zip\"", proxyName.toLowerCase()));

            // Create a part with the zip data
            Part part = new Part(headers, new ByteArrayContent("application/octet-stream", zipBytes));
            multipartContent.setParts(Collections.singleton(part));
            part.getHeaders().setAcceptEncoding("none");

            // Write MultipartContent to a byte array for sending the request
            ByteArrayOutputStream multiPartContentAsByteArray = new ByteArrayOutputStream();
            multipartContent.writeTo(multiPartContentAsByteArray);

            // Create an Apigee request using HttpRequestFactory
            HttpRequestFactory requestFactory = apigeeService.getRequestFactory();
            com.google.api.client.http.HttpRequest request = requestFactory.buildPostRequest(
                    new com.google.api.client.http.GenericUrl(String.format(
                            "https://apigee.googleapis.com/v1/organizations/%s/apis?action=import&name=%s",
                            organization, proxyName)),
                    new ByteArrayContent("multipart/form-data; boundary=" + boundary, multiPartContentAsByteArray.toByteArray())
            );

            // Set the parser for response handling
            request.setParser(GsonFactory.getDefaultInstance().createJsonObjectParser());

            // Set the Content-Type header
            request.getHeaders().setContentType("multipart/form-data; boundary=" + boundary);

            // Execute the request and parse the response
            HttpResponse response = request.execute();
            GoogleCloudApigeeV1ApiProxyRevision apiProxyRevision = response.parseAs(GoogleCloudApigeeV1ApiProxyRevision.class);
            System.out.println("Import Operation: " + apiProxyRevision);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Libs I used:

    <dependencies>
        <dependency>
          <groupId>com.google.apis</groupId>
          <artifactId>google-api-services-apigee</artifactId>
          <version>v1-rev20240919-2.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.google.http-client</groupId>
            <artifactId>google-http-client</artifactId>
            <version>1.40.0</version>
        </dependency>
      </dependencies>

[Mohammed-Mahtab]

Can you also please provide the sample working code for OPDK , As we are not using service account there instead Management API authenticated against user id and password

[medbayoudh]

Many thanks @nmarkevich that was very helpful. Do you know why the "native" way didn't work? 

[nmarkevich]

Glad I was able to help!

Idk for sure, probably GoogleApiHttpBody is not suitable for multipart data.

[medbayoudh]

@dchiesa1 maybe you have some ideas why didn't work using the native impl

GoogleApiHttpBody body = 
   new GoogleApiHttpBody()
        .encodeData(multiPartContentAsByteArray.toByteArray());

Create create = 
   apigee
          .organizations()
          .apis()
          .create(properties.getGoogleOrganization(), body)
          .setAction("import")
          .set("name", apiSpecName.toLowerCase());

create
     .getRequestHeaders()
     .setContentType("multipart/form-data; boundary=".concat(boundary));

create.execute();

[dchiesa1]

Sorry, I don't have any ideas. I am not familiar with that library.