Invoke Gemini AI models from SAP applications

[devesh-sapcloud]

SAP businesses today need a faster, cheaper and leaner way to infuse AI into their business processes. To help our customers with this challenge, we recently launched our Vertex AI SDK for ABAP, a powerful toolset to natively utilize the full potential of Google’s Vertex AI capabilities directly from SAP applications. This means that, with minimal efforts, ABAP developers can leverage SDK modules to invoke Gemini AI models, embedding models, create, manage and deploy vector index, perform vector semantic search and much more from their known ABAP environment. They can also start their prototyping with Gemini models in free and paid tier directly from SAP.

In this blog post we would see how customers and developers can start their journey of building Gemini AI infused ABAP applications with a few installation and configuration steps. At the end of this blog you will have an understanding of the below,

• Configure Model Generation parameters for a Gemini Model,
• Invoke the Gemini Model,
• Capture model response,
• Error and exception handling.

Installation of the SDK

Vertex AI SDK for ABAP is built on top of our ABAP SDK for Google Cloud, and is available from Version 1.8 of the ABAP SDK. If you're already using version 1.7 or earlier of the on-premises or any cloud edition of the SDK, we encourage you to download and install the latest version of the SDK to get the latest and greatest of our tool sets and enablements.

If you have restrictions or in lack of SAP system, you can install your own SAP sandbox and try out the SDK by following our blog "Install ABAP Platform Trial 2022 on Google Cloud Platform and Install Vertex AI SDK for ABAP".

Use cases

Using the Gemini models, SAP customers and developers can perform,

• Automatic order processing: Gemini can extract order details from emails, PDFs, or other unstructured data sources and provide structured output to enable automatic creation sales orders in SAP, reducing manual effort and errors.
• Financial Forecasting: Gemini can analyze SAP financial data and reports to generate accurate financial forecasts and identify potential risks and opportunities.
• Supply Chain Optimization: Gemini can analyze supply chain data to predict demand fluctuations, optimize inventory levels, and identify potential disruptions, improving efficiency and resilience for businesses.
• Customer Sentiment Analysis: Gemini can analyze customer feedback from surveys, social media, and other sources to understand customer sentiment and identify areas for improvement in SAP Customer Experience solutions.
• Personalized Product Recommendations: Gemini can analyze customer data and product information to provide personalized product recommendations, increasing sales and customer satisfaction.
• Fraud Detection: Gemini can analyze transaction data to identify patterns and anomalies that may indicate fraudulent activities in business processes.

….and many more.

Using the Vertex AI SDK for ABAP, customers can now think of building these use cases natively from their ABAP environment. It alleviates the technological challenge of lack of enabler SDK for ABAP developers, that other programming languages have (Python, Java, Node.js, etc. have dedicated Vertex AI SDKs) to build AI centric applications.

Okk….enough of sales pitch….let’s get into invoking Gemini models….

Configure Model Generation Parameters

For invoking Gemini AI models, the SDK comes with a central SAP table maintenance based configuration to define model configurations for your scenarios centrally one time against a “Model Key”, and use the key to invoke the model at different placeholders in your application logic. Refer to the link here on how to navigate to the configuration table (table /GOOG/AI_CONFIG) and look at the table fields in detail.

Main information held in this table are,

• A developer defined “Model Key” as a configuration id,
• Model ID of the Gemini model, such as gemini-1.5-flash-001,
• ABAP SDK client key name to connect to the backend Vertex AI API,
• Model generation parameters, such as temperature, top-k and top-p sampling etc.

Model generation parameters are parameters of the model that can be used to control the model responses to get the best responses for your business scenarios. You can provide values for these generation parameters in the configuration table against the model key and these would take into effect when the model is invoked using the SDK from your application logic.

For example, if you want the model to be more creative, define a higher value of temperature (closer to 1) for the configured model key. If you want the model to give a more deterministic and focussed response, define a lower value of temperature (closer to 0) for the configured model key.

Invoke Gemini Models

Gemini Generative Model Invoker component of the SDK can be used to invoke text and multimodal Gemini models from your ABAP based applications. You can invoke the model by,

• Instantiating class /GOOG/CL_GENERATIVE_MODEL with the model key defined in the model configuration table,
• Calling method GENERATE_CONTENT with a prompt to invoke the model id defined against the model key in the configuration table (model generation parameters if defined are taken into effect).
• Chain the invocation with method GET_TEXT to capture the model response.

Invocation in action with a use case

Imagine below use case of a business user who wants to know if there are any risks in their incoming orders.

“As a business user, I want to know if there are any risky clauses in the delivery comments of an incoming purchase order. 

• If yes, the corresponding sales order should be created in blocked status along with an alert generated to review the blocked order.
• If there are no risky clauses, the sales order should be created without any blocks.”

The business would like to implement this requirement using Google’s AI capabilities natively from their SAP environment. This implementation is typically known as business rule engines (BRE) which allows you to define, deploy, and execute business rules. Refer to our reference architecture to implement such rule engines in your SAP applications using Gemini. To implement this, you can have a SAP program running as a background job and,

• ABAP SDK for Google Cloud can be used to invoke Google Cloud Document AI to parse the incoming purchase orders in a GCS bucket and determine the delivery comments from the response.
• Invoke Gemini Generative Model Invoker of embedded Vertex AI SDK to analyze and identify any risks in the delivery comments.

I choose Gemini Flash to implement this (you can choose what works for you from the available model list) and I create a model generation configuration against a model key. To configure,

• Go to SPRO path “Configure Model Generation Parameters” as below.

• Enter Google Cloud key name and model id along with generation configuration (optional) against a Model Key.

Next, I invoke Gemini Flash to determine any risks in the delivery comments using the Model Key configuration as shown in the code snippet below.

DATA:
  lv_prompt            TYPE string,
  lv_delivery_comments TYPE string.

lv_delivery_comments = |{ 'Buyer grants seller a royalty-free, perpetual, irrevocable'} | &&
                       |{ 'license to use any intellectual property developed in'} | &&
                       |{ 'connection with the products'} |.

lv_prompt = |{ 'Identify any risky clause in delivery instruction from a purchase order document.'} | &&
            |{ 'Delivery Instruction -'} | && lv_delivery_comments.

TRY.
    DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'gemini-flash' ).

    DATA(lv_response) = lo_model->generate_content( lv_prompt
                               )->get_text( ).
    IF lv_response IS NOT INITIAL.
      cl_demo_output=>display( lv_response ).

    ENDIF.
  CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
    cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

Once executed, Gemini responds with the risk analysis and ways to mitigate it similar to as shown below.

Hurray….I was able to identify risks and mitigation recommendations using Gemini AI natively in my SAP application. But this response is in a paragraph format and I cannot use this as is to build follow on logic to create a blocked sales order for my rule engine.

So, I additionally instruct the AI model to generate the response in a format that is directly usable by me in the program in order to efficiently create the blocked order. For this, I use the method SET_SYSTEM_INSTRUCTIONS as shown in the code snippet below to set string based specific instructions to the model to generate the response in JSON format having fields issue, issue category, explanation and recommendation.

DATA:
  lv_prompt              TYPE string,
  lv_delivery_comments   TYPE string,
  lv_system_instructions TYPE string.

lv_delivery_comments = |{ 'Buyer grants seller a royalty-free, perpetual, irrevocable'} | &&
                       |{ 'license to use any intellectual property developed in'} | &&
                       |{ 'connection with the products'} |.

lv_system_instructions = |{ 'You are a legal advisor for my company. Analyze the below delivery instruction on a' } | &&
                         |{ 'purchase order from my customer and let me know if it could cause issues to my company' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Respond to me in plain text JSON format with 4 fields.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 1 should be "issue" with possible values Yes or No.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 2 should be "issueCategory" with possible values Indemnification Overreach, Unilateral Termination Clause, Confidentiality Overextension,' } | &&
                         |{ 'Unreasonable Warranty, Ambiguous Governing Law, Unclear Dispute Resolution, Force Majeure Limitation, Unbalanced Liability,' } | &&
                         |{ 'Intellectual Property Ambiguity, Compliance with Sanctions, Others or None.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 3 should be "explanation" - use this field to give a short explanation for your response on Field 1.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 4 should be "recommendation" - use this field to give a short recommendation of how to mitigate such issues.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'DO NOT INCLUDE BACKTICKS IN THE RESPONSE' }|.

lv_prompt = |{ 'Identify any risky clause in delivery instruction from a purchase order document.'} | &&
            |{ 'Delivery Instruction -'} | && lv_delivery_comments.

TRY.
    DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'gemini-flash' ).

    DATA(lv_response) = lo_model->set_system_instructions( lv_system_instructions
                               )->generate_content( lv_prompt
                               )->get_text( ).
    IF lv_response IS NOT INITIAL.
      cl_demo_output=>display( lv_response ).

    ENDIF.
  CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
    cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

With proper instructions, I was able to get the model in JSON format as below.

I can deserialize this into ABAP type having the same fields and write a business rule to look at the issue field.

• If “Yes”, create the sales order in block status, raise an alert (using Cloud Pub/Sub with ABAP SDK),
• If “No”, create the sales order in unblocked status.

I can additionally place the explanation and recommendation for the risks suggested by Gemini in the blocked Sales Order header text for the business user as shown below, to review and take action.

Invocation using multimodal prompts

In the above sections you saw invocation of a Gemini model to look at delivery comments text using a text based prompt and system instructions. The Gemini family of models consist of text as well as multimodal models which can accept other modalities of data other than text and can be used to analyze multimodal data using text based prompts. For example, Gemini Flash is a multimodal model and can accept images, videos, PDFs, etc., to generate response based on text based prompt and instructions.

Vertex AI SDK for ABAP supports invocation models with multimodal inputs. You can input the multimodal data either by,

• Providing the Google Cloud Storage Bucket URI of the file object using SDK method SET_FILE_DATA of class /GOOG/CL_GENERATIVE_MODEL, or
• Providing the raw data of the file object in Base64 encoded format using SDK method SET_INLINE_DATA of class /GOOG/CL_GENERATIVE_MODEL.

Staying close to SAP use cases, let’s see how we can input the purchase order from before with risky delivery comments directly to Gemini through an SAP program, and ask it to parse and analyze any risks. For this I just changed my input prompt slightly, we are not using Document AI parsed delivery instructions any more but asking Gemini to parse, find delivery comments, analyze and suggest risks and mitigation recommendations.

Give GCS URI of the PDF

Here I have used the method SET_FILE_DATA to input GCS URI of the purchase order PDF file.

DATA:
  lv_prompt              TYPE string,
  lv_system_instructions TYPE string.

lv_system_instructions = |{ 'You are a legal advisor for my company. Analyze the delivery instruction on the input' } | &&
                         |{ 'purchase order from my customer and let me know if it could cause issues to my company' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Respond to me in plain text JSON format with 4 fields.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 1 should be "issue" with possible values Yes or No.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 2 should be "issueCategory" with possible values Indemnification Overreach, Unilateral Termination Clause, Confidentiality Overextension,' } | &&
                         |{ 'Unreasonable Warranty, Ambiguous Governing Law, Unclear Dispute Resolution, Force Majeure Limitation, Unbalanced Liability,' } | &&
                         |{ 'Intellectual Property Ambiguity, Compliance with Sanctions, Others or None.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 3 should be "explanation" - use this field to give a short explanation for your response on Field 1.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 4 should be "recommendation" - use this field to give a short recommendation of how to mitigate such issues.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'DO NOT INCLUDE BACKTICKS IN THE RESPONSE' }|.

lv_prompt = 'Identify any risky clause in delivery instruction from the input purchase order PDF document.'.

TRY.
    DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'gemini-flash' ).

    DATA(lv_response) = lo_model->set_system_instructions( lv_system_instructions
                               )->set_file_data( iv_file_uri = 'gs://incoming-purchase-orders/5002000027.pdf'
                               )->generate_content( lv_prompt
                               )->get_text( ).
    IF lv_response IS NOT INITIAL.
      cl_demo_output=>display( lv_response ).

    ENDIF.
  CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
    cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

Output.

Browse file to input raw data

Here I have used the method SET_INLINE_DATA to input Base64 encoded raw data of the browsed purchase order PDF file from my desktop.

DATA:
  lv_file        TYPE string,
  lv_file_length TYPE i,
  lt_bin_data    TYPE STANDARD TABLE OF char1024,
  lv_xfile       TYPE xstring,
  lv_raw_data    TYPE string.

lv_file = 'C:\Users\Guest\Desktop\5002000027.pdf'.

CALL FUNCTION 'GUI_UPLOAD'
  EXPORTING
    filename   = lv_file
    filetype   = 'BIN'
  IMPORTING
    filelength = lv_file_length
  TABLES
    data_tab   = lt_bin_data.

CALL FUNCTION 'SCMS_BINARY_TO_XSTRING'
  EXPORTING
    input_length = lv_file_length
  IMPORTING
    buffer       = lv_xfile
  TABLES
    binary_tab   = lt_bin_data.

CALL FUNCTION 'SCMS_BASE64_ENCODE_STR'
  EXPORTING
    input  = lv_xfile
  IMPORTING
    output = lv_raw_data.

DATA:
  lv_prompt              TYPE string,
  lv_system_instructions TYPE string.

lv_system_instructions = |{ 'You are a legal advisor for my company. Analyze the delivery instruction on the input' } | &&
                         |{ 'purchase order from my customer and let me know if it could cause issues to my company' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Respond to me in plain text JSON format with 4 fields.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 1 should be "issue" with possible values Yes or No.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 2 should be "issueCategory" with possible values Indemnification Overreach, Unilateral Termination Clause, Confidentiality Overextension,' } | &&
                         |{ 'Unreasonable Warranty, Ambiguous Governing Law, Unclear Dispute Resolution, Force Majeure Limitation, Unbalanced Liability,' } | &&
                         |{ 'Intellectual Property Ambiguity, Compliance with Sanctions, Others or None.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 3 should be "explanation" - use this field to give a short explanation for your response on Field 1.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'Field 4 should be "recommendation" - use this field to give a short recommendation of how to mitigate such issues.' } | && cl_abap_char_utilities=>newline &&
                         |{ 'DO NOT INCLUDE BACKTICKS IN THE RESPONSE' }|.

lv_prompt = 'Identify any risky clause in delivery instruction from the input purchase order PDF document.'.

TRY.
    DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'gemini-flash' ).

    DATA(lv_response) = lo_model->set_system_instructions( lv_system_instructions
                               )->set_inline_data( iv_data = lv_raw_data
                               )->generate_content( lv_prompt
                               )->get_text( ).
    IF lv_response IS NOT INITIAL.
      cl_demo_output=>display( lv_response ).

    ENDIF.
  CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
    cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

Following similar patterns as above you can input GCS URIs and raw data for your images, videos and other modalities to the Gemini multimodal models using the SDK, from your SAP applications.

Further control your model generation

With Generative Model Invoker class /GOOG/CL_GENERATIVE_MODEL, you can additionally set more parameters to guide the model to generate responses as per your requirements.

• Utilize method ADD_SAFETY_SETTINGS to set the safety parameters to control the model response based on harm categories, threshold levels and blocking methods (refer to the code sample).
• Utilize method SET_GENERATION_CONFIG to override the generation configurations set against the model key in model generation configuration table /GOOG/AI_CONFIG for a specific invocation using the same model instance.
• Utilize the method SET_RESPONSE_MIME_TYPE to set the Mime Type in which you want the model to generate response in.

Keep track of your billing by counting input characters to the model

Billing for the Gemini models depends on the number of characters or tokens that you invoke it with, which can consist of both text and multimodal input data. The SDK also gives you an out of the method COUNT_TOKENS with Generative Model Invoker class /GOOG/CL_GENERATIVE_MODEL to get the total tokens and total billable characters in your input prompt. You can use this method to keep track of your project’s budget for invoking Gemini models. Here is a code sample to count tokens for your text inputs, below code snippet shows you how to count tokens for your multimodal input.

DATA:
  lv_instruction TYPE string,
  lv_prompt      TYPE string.

lv_instruction = 'You are a Document Parser. Your job is to parse and summarize input documents with different Mime Types.'.
lv_prompt      = 'Give me a summary of the purchase order in JSON format.'.

TRY.
    DATA(lo_model) = NEW /goog/cl_generative_model( iv_model_key = 'gemini-flash' ).
    DATA(lo_model_response) = lo_model->set_file_data( iv_mime_type = 'application/pdf'
                                                       iv_file_uri  = 'gs://PO/po-multi-line.pdf'
                                     )->count_tokens( iv_prompt_text         = lv_prompt
                                                      iv_system_instructions = lv_instruction ).
    DATA(lv_total_tokens) = lo_model_response->get_total_tokens( ).
    DATA(lv_total_billable_chars) = lo_model_response->get_total_billable_characters( ).
    WRITE:
          |{ 'Total tokens in the input -' } | && lv_total_tokens, /
          |{ 'Total billable characters in the input -' } | && lv_total_billable_chars.

  CATCH /goog/cx_sdk INTO DATA(lo_cx_sdk).
    cl_demo_output=>display( lo_cx_sdk->get_text( ) ).

ENDTRY.

Output.

You can also use the method SET_INLINE_DATA in place or alongside method SET_FILE_DATA to input multimodal raw data to count the tokens and billable characters.

Derive the most out of Gemini responses

The GET_TEXT method of the Generative Model Invoker module gets you the response from the Gemini model for an invocation. But you can derive much more from the response instance.

• Utilize the method GET_SAFETY_RATING to get the safety rating of the model response.
• Utilize the method GET_PROMPT_TOKEN_COUNT to get the number of tokens in the input prompt.
• Utilize the method GET_CANDIDATES_TOKEN_COUNT to get the number of tokens in the model response.
• Utilize the method GET_BLOCK_REASON to get the reason for which the model did not generate a response for an input prompt.
• Utilize the method GET_BLOCK_REASON_MESSAGE to the reason in detail for which the model did not generate a response for an input prompt.
• Utilize method GET_CITATIONS to get the citation source of the model response.

Error and exception handling

Any errors returned from the model invocation or internal SDK errors are raised as exceptions and can be caught using ABAP SDK shipped exception class /GOOG/CX_SDK. Remember to encapsulate your code logic (like shown in the code snippets) in a TRY CATCH block with this class.

Also, the SDK has SLG based application logging functionality which you can configure and use to log your errors, warnings and information messages. Refer to the blog on application logging with ABAP SDK to know more.

Get started today

The Vertex AI SDK for ABAP is now available for developers to explore and integrate into their SAP projects. Whether you're an experienced AI practitioner or just starting your AI journey, the SDK provides the tools and resources you need to harness Google AI's potential within your SAP environment.

We're excited to see the innovative solutions that developers will create using this powerful integration. Stay tuned for further updates, tutorials, and success stories as we continue to evolve the Vertex AI SDK for ABAP and empower businesses to achieve more with Google AI within their SAP systems.

Additional resources,

• Vertex AI SDK for ABAP Documentation: [Link]
• Install and configure Vertex AI SDK for ABAP: [Link]
• Run and explore Generative AI demos: [Link]

Ask your questions, tell us about your use cases

The ABAP SDK for Google Cloud Community is now open! This is a place for you to ask questions, share knowledge, and collaborate with other ABAP developers who are using Google Cloud. We encourage you to get involved in the community and help us make the ABAP SDK for Google Cloud even better. We have a lot of exciting things planned for the future, and we want you to be a part of it.

Click on this link to start a conversation under label "ABAP SDK", post your questions and innovate with us.