OCI GenAI Cohere Chat

OCI GenAI Service offers generative AI chat with on-demand models, or dedicated AI clusters.

The OCI Chat Models Page and OCI Generative AI Playground provide detailed information about using and hosting chat models on OCI.

Prerequisites

You will need an active Oracle Cloud Infrastructure (OCI) account to use the OCI GenAI Cohere Chat client. The client offers four different ways to connect, including simple authentication with a user and private key, workload identity, instance principal, or OCI configuration file authentication.

Add Repositories and BOM

Spring AI artifacts are published in Maven Central and Spring Snapshot repositories. Refer to the Artifact Repositories section to add these repositories to your build system.

To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the Dependency Management section to add the Spring AI BOM to your build system.

Auto-configuration

Spring AI provides Spring Boot auto-configuration for the OCI GenAI Cohere Chat Client. To enable it add the following dependency to your project’s Maven pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-oci-genai</artifactId>
</dependency>

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-model-oci-genai'
}

Chat Properties

Connection Properties

The prefix spring.ai.oci.genai is the property prefix to configure the connection to OCI GenAI.

Configuration Properties

The prefix spring.ai.oci.genai.cohere.chat is the property prefix that configures the ChatModel implementation for OCI GenAI Cohere Chat.

Runtime Options

The OCICohereChatOptions.java provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.

On start-up, the default options can be configured with the OCICohereChatModel(api, options) constructor or the spring.ai.oci.genai.cohere.chat.options.* properties.

At run-time you can override the default options by adding new, request specific, options to the Prompt call. For example to override the default model and temperature for a specific request:

ChatResponse response = chatModel.call(
    new Prompt(
        "Generate the names of 5 famous pirates.",
        OCICohereChatOptions.builder()
            .model("my-model-ocid")
            .compartment("my-compartment-ocid")
            .temperature(0.5)
        .build()
    ));

Sample Controller

Create a new Spring Boot project and add the spring-ai-starter-model-oci-genai to your pom (or gradle) dependencies.

Add a application.properties file, under the src/main/resources directory, to enable and configure the OCI GenAI Cohere chat model:

spring.ai.oci.genai.authenticationType=file
spring.ai.oci.genai.file=/path/to/oci/config/file
spring.ai.oci.genai.cohere.chat.options.compartment=my-compartment-ocid
spring.ai.oci.genai.cohere.chat.options.servingMode=on-demand
spring.ai.oci.genai.cohere.chat.options.model=my-chat-model-ocid

This will create a OCICohereChatModel implementation that you can inject into your class. Here is an example of a simple @Controller class that uses the chat model for text generations.

@RestController
public class ChatController {

    private final OCICohereChatModel chatModel;

    @Autowired
    public ChatController(OCICohereChatModel chatModel) {
        this.chatModel = chatModel;
    }

    @GetMapping("/ai/generate")
    public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return Map.of("generation", chatModel.call(message));
    }

    @GetMapping("/ai/generateStream")
    public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        var prompt = new Prompt(new UserMessage(message));
        return chatModel.stream(prompt);
    }
}

Manual Configuration

The OCICohereChatModel implements the ChatModel and uses the OCI Java SDK to connect to the OCI GenAI service.

Add the spring-ai-oci-genai dependency to your project’s Maven pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-oci-genai</artifactId>
</dependency>

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-oci-genai'
}

Next, create a OCICohereChatModel and use it for text generations:

var CONFIG_FILE = Paths.get(System.getProperty("user.home"), ".oci", "config").toString();
var COMPARTMENT_ID = System.getenv("OCI_COMPARTMENT_ID");
var MODEL_ID = System.getenv("OCI_CHAT_MODEL_ID");

ConfigFileAuthenticationDetailsProvider authProvider = new ConfigFileAuthenticationDetailsProvider(
        CONFIG_FILE,
        "DEFAULT"
);
var genAi = GenerativeAiInferenceClient.builder()
        .region(Region.valueOf("us-chicago-1"))
        .build(authProvider);

var chatModel = new OCICohereChatModel(genAi, OCICohereChatOptions.builder()
        .model(MODEL_ID)
        .compartment(COMPARTMENT_ID)
        .servingMode("on-demand")
        .build());

ChatResponse response = chatModel.call(
        new Prompt("Generate the names of 5 famous pirates."));

The OCICohereChatOptions provides the configuration information for the chat requests. The OCICohereChatOptions.Builder is fluent options builder.