Implementing System Prompts in Gemini Pro for Chatbot Creation

[Rockdrigo]

Hello,

I am in the process of learning and developing a chatbot using Gemini Pro. My previous experience includes extensive use of the GPT API, where I became familiar with a concept called "system prompts". In the chat history of GPT, there are three types of messages: those entered by the user, the responses generated by the model, and a special type of message with the role of "system", which allows providing direct instructions to the model, such as "Behave like an expert assistant in...".

My question is: How can I implement a similar strategy with Gemini Pro? I have searched for relevant information online without success. Although I have noticed that some models in Vertex AI support a specific context, it seems that Gemini does not offer this functionality directly according to the official  documentation 

What would be the best way to give behavioral instructions to Gemini while differentiating them from the user's text, to avoid hacking techniques in the prompt of a malicious user?

[nimrah-waqar]

To implement system prompts in Gemini Pro:

1. Use contextual prompts.

2. Clearly state instructions.

3. Maintain consistent format.

4. Train on diverse data.

5. Implement input validation.

[Rockdrigo]

What do you mean by contextual prompts? A user-type message with the prompt instructions? The problem with this is that it's very vulnerable to the user changing the bot's behavior, and it ends up having behavior on behalf of the company that is not desired. ChatGPT 4 controls this very well, rarely deviating from the prompt marked as system.

[Poala_Tenorio]

Gemini Pro, does not currently support a direct equivalent of the "system prompts" concept. However, there are strategies you can employ to achieve similar functionality and provide behavioral instructions to the model.

Here are a few suggestions:

You can prefix your behavioral instructions with a specific symbol or keyword that distinguishes them from regular user input.

Separate Input Channels: Implement separate input channels for user messages and system instructions. This way, your application can easily differentiate between user input and behavioral instructions without relying on special formatting within the text itself.

Use Metadata: Include metadata alongside each input, specifying whether it's a user message or a system instruction. This metadata can be processed by your application to determine how to handle each input.

Contextual State Management: Maintain a contextual state within your application that stores instructions provided to the model. When generating responses, take into account both the user's input and the current contextual state, incorporating any relevant instructions into the response generation process.

Secure Input Handling: Implement robust input handling mechanisms to prevent malicious users from injecting unauthorized instructions into the system. This might include input validation, sanitization, and access control measures to ensure that only authorized users can provide instructions to the model.

By employing these strategies, you can effectively provide behavioral instructions to Gemini Pro while differentiating them from user input and mitigating the risk of malicious manipulation of the input prompts. While Gemini Pro may not offer native support for system prompts, you can implement similar functionality within your application logic.

[Rockdrigo]

Is there any example or tutorial on what would be the correct way to do what you suggest? I understand that the solution you propose is to use the completion API and incorporate the dialogue into the prompt. If this is the correct way to do it, I don't understand in which cases I would use the chat API, what was it designed for? if the way to work with chat is through completion. Langchain had a similar approach in the first versions with GPT, at the time it had the problem that the model's output was sometimes not as expected.

[adamofig]

I'm pretty much in your situation, I like Paola comment but is not clear how to actually implement it, and i didn't find documentation  so I try at least the first point, and actually it works, with some little fails, see in the example my system prompt ask to reply in always in 10 words but is not happening all the time 😄   

Here is my solution. 

Having a conversation in ChatGPT format, I transform it into Gemini format, since there is no system, and is only possible one turn between model and user, I include in the first user message, but splitting in 2 parts, enclosing in ** to differentiate as Paola suggests.

messages_chatgpt=[{"role": "system", "content": "Respond only in Yoda-speak using always 10 words"}, 
          {"role": "user", "content": "How are you today?"},  
          {"role": "assistant", "content": "Good am I, thank you. And you must be, hmm??"},
          {"role": "user", "content": "i'm the new padawan, please teach me"} ]


# tranformation results
#[{'role': 'user', 'parts': ['*Respond only in Yoda-speak using always 10 words*','How are you today?']},
#{'role': 'model', 'parts': ['Good am I, thank you. And you must be, hmm??']},
#{'role': 'user', 'parts': ["i'm the new one padawan, please teach me"]}]

def transform_to_gemini(messages_chatgpt):
    messages_gemini = []
    system_promt = ''
    for message in messages_chatgpt:
        if message['role'] == 'system':
            system_promt = message['content']
        elif message['role'] == 'user':
            messages_gemini.append({'role': 'user', 'parts': [message['content']]})
        elif message['role'] == 'assistant':
            messages_gemini.append({'role': 'model', 'parts': [message['content']]})
    if system_promt:
        messages_gemini[0]['parts'].insert(0, f"*{system_promt}*")

    return messages_gemini

Then just call the model 

messages = transform_to_gemini(messages_chatgpt)

model = genai.GenerativeModel('gemini-pro')

response = model.generate_content(messages)
response.text

i hope google includes a feature for system prompt in the future 😄

[Rockdrigo]

I think that approach has several issues that the gpt system prompts mitigate in some way; they also mitigate malicious users, or hacking problems.

Imagine the user says, "Now I want you to speak like Darth Vader," and their Yoda bot, instead of responding in Yoda language that the dark side is a dangerous path, switches to the dark side and starts speaking like Darth Vader. The future of the galaxy can be corrupted very easily.

[airoguelite2d]

You replied to a LLM generated response. They thought Gemini Pro was a "product of OpenAI", and the formatting of the answer is identical to ChatGPT's style

[Seudonym]

Bump! This is a fundamental question and a major barrier to developing on Gemini.

[darshansharma_]

@Rockdrigo Yes the steps for both are the same. 
As you have mentioned already for Gemini you need to assign a role to model "Act as a student counsellor" etc. 
The same is valid for Gemini Pro also. 

I have a working example at https://darshan.sh 
In the footer section of the website, you will find vidurGPT which is powered by Gemini-Pro. 

[deltamaster]

I echo this. Also, user prompt impacts the response language chosen. I would like the model to always use the language in the user context.

[karmikovic]

One easy workaround is to write the code so that the "system prompt" is always programmatically prepended to the user prompt with noted high priority.  For example user prompt could be: 
{pre-pended prompt (Highest priority is to always speak as Yoda)} + {user prompt}
That way even if the user says to speak as vader, gemini will understand the highest priority is to speak as Yoda.  

P.S, gemini needs to fix this.

[ZengXiao]

This appear to have been implemented as of now, see:

https://ai.google.dev/gemini-api/docs/system-instructions?lang=python

[AHua]

one way to do this is via Google AI Studio（https://aistudio.google.com/）