Integrate SUVA Agents with Large Language Models

Under the Integrations tab, you will see two types of LLMs:

• SUVA Partner Provisioned LLMs such as Claude and OpenAI Service

• Bring your own LLM such as OpenAI and Bard.

SUVA Partner Provisoned LLMs, like Claude by AWS and OpenAI Service by Azure, ensure data privacy. Unlike some third-party LLMs, Claude and OpenAI Service neither store customer data nor use customer data to train their models.

Integration with LLMs such as Claude and OpenAI is available immediately while work on integration with Bard and OpenAI Service is in development. SUVA's in-house LLM will be announced in upcoming releases.

Setting Up LLM Integration

1. From the SUVA admin panel, navigate to Large Language Model > Integrations.

2. Under the Integrations tab, you will see two set of LLMs:
   2.1) SUVA Partner Provisioned LLM, including Claude and OpenAI Service

   2.2) Bring your own LLM includes the popular third-party LLMS, such as OpenAI and Bard.

3. To use Claude by AWS, start a 14-day free trial. To continue with Claude after the trial period, reach out to your CSM or drop an email at email at llmrequestinfo@searchunify.com.

   Note: Save the settings after you have activated the Claude free trial.

   

4. Alternatively, use a third-party LLM by entering its authentication key. OpenAI is the only third-party LLM currently supported..

   

   Refer to the following doc to learn how to get the OpenAI authentication key:

   Get the Authentication Key for Setting Up LLM Integration

5. If the integration is successful, toggle on to activate the OpenAI integration.

   

Complete the integration by configuring agents. The instructions are in Agent Configuration: Select Agent(s) and LLM Model, and Set Temperature Settings .

Replacing the Authentication Key

Use the edit icon to replace the authentication key. After confirming that you want to replace the key, the current key is removed.

Note. The current integration will be disabled if the authentication key changes. Reconnect with the new authentication key and Activate the integration. The selected Virtual Agent(s) and the temperature setting remains unaffected.

While changing keys, ensure that the new key is correct. If the authentication key is invalid and the configuration tab is closed or the settings with an invalid authentication key are saved, then the app continues to function based on the previous key.

After enabling the LLM integration, complete the process by configuring an agent or agents with activated LLM(s).

1. Switch to the Agent configuration tab. You will see a list of Virtual Agents added in your SUVA instance.

   

2. Expand an agent and you will see three options:

   i) Select Model. Pick a suitable LLM. Three options are available in OpenAI: None, OpenAI - 4K context BYOL, or OpenAI - 16k context BYOL..

   ii) Set Temperature in the range 0-1.0. You can set the temperature to other values, such as 0.2 and 0.4. The recommended temperature, however, is 0.2.

   Note. 0.2 may not be the ideal temperature for your LLM integration. The ideal temperature depends on your use case. However, 0.2 is a good place to start. You can then adjust the temperature up or down to get the results.

   iii) OPTIONAL. Prompt is the text that you give to the LLM to start the response generation process. It should be clear and user-friendly.

   NOTE. An activated LLM can be linked to more than one Virtual Agents.

3. Save your settings. If you skip this step, then the integration will not work.

   

Once you have saved your settings, the process is complete. You can now see the LLM integration configured with a virtual agent.

The setup is done and the usage is covered in LLM Integration Usage.

How Temperature Setting Impacts Chatbot Responses

As an AI language model, LLMs can sometimes hallucinate and give inaccurate responses. Temperature Setting is there to help you minimize the hallucination. Temperature setting is a hyperparameter that can be used to control the randomness of responses. A higher temperature produces more unpredictable and creative results, while a lower temperature produces more common and conservative output.

The scale for setting the temperature is between 0 and 1.0; with 0 causing the chatbot to produce only conservative responses and 1.0 nudging the chatbot to come up with more human-like responses. If you adjust the temperature to 0.4, then the model generates more predictable text and less creative text than if you were to set the temperature to 1.0. The temperature scale can only be set to even values like 0.2, 0.4. 0.6. 0.8, and 1.0.

The best temperature depends on the desired output. Temperature, however, is one of the many factors that influence LLM output. The other factors include:

• The prompt is the text that you give to the LLM to start the response generation process. The prompt can have a big impact on the output, so it is important to choose a prompt that is clear and concise.

• The training data is the set of text that the LLM was trained on. The training data can affect the output in a number of ways, including the diversity of the output, the accuracy of the output, and the fluency of the output.

• The model architecture is the design of the LLM. The model architecture can affect the output in a number of ways, including the speed of the generation process, the complexity of the output, and the accuracy of the output.

The recommend temperature is 0.2. Note. 0.2 isn't the ideal temperature for all LLM. The ideal temperature setting will vary depending on the specific task and the desired output. To find out the ideal temperature for your LLM, experiment a little.

Following are some of the example responses generated with different temperature setting scales:

Temperature Setting @ 0.2

Temperature Setting @ 1.0