Coding and deploying AI services with Cloud Pak for Data Command-Line Interface (CPDCTL)

Last updated: Feb 06, 2025

CPDCTL is a command-line tool for deploying and managing AI services on the IBM Cloud Pak for Data (CPD) platform. It provides a simple and streamlined way to deploy AI services, eliminating the need for manual configuration and reducing the risk of errors. You can use the Cloud Pak for Data (CPDCTL) command-line interface to deploy your generative AI solutions that use AI services programmatically.

Process overview

The process of deploying an AI service with CPDCTL involves preparing the environment by installing CPDCTL and configuring environment variables, creating an AI service instance, uploading the code for the AI service, and deploying the AI service to make it available for use. The deployment process is initiated by running a series of CPDCTL commands, including creating an AI service instance, uploading the code, and deploying the AI service. The deployed AI service can then be accessed through a REST API endpoint, enabling real-time predictions and decision-making. Throughout the process, CPDCTL provides a simple and efficient way to deploy, manage, and scale AI services.

Prerequisites

Before deploying AI services with CPDCTL, you must meet the following prerequisites:

Install and configure CPDCTL. For more information, see IBM Cloud Pak for Data Command Line Interface.
Create a deployment space with access to watsonx.ai.
Save the deployment space ID, service URL, and API key as environment variables: SPACE_ID, WML_URL, and API_KEY.
Prepare the software specification runtime-24.1-py3.11 with access to langgraph: SOFT_SPEC.

Preparing the code

Download the AI service code: deployable_code.py.

wget -O deployable_code.py https://raw.githubusercontent.com/IBM/watson-machine-learning-samples/refs/heads/ai-service-cli-sample/cloud/definitions/ai_service/deployable_function_ai_service_cli.py

You must provide the space_id and url to the AI service.

Prepare a gzip file with the code file.
```
gzip -k deployable_code.py
```

Creating an AI service instance

Create an AI Service instance by using CPDCTL:

cpdctl wx-ai ai-service create \
  --space-id $SPACE_ID \
  --name ai-service-app \ 
  --description "Sample AI Service" \
  --tags langgraph,example \
  --code-type python \
  --software-spec-id $SOFT_SPEC

This returns basic information about the created AI Service instance. Use the --output JSON flag to get more detailed data.

Save the obtained ID as AI_SERVICE_ID.

Uploading code to AI service instance

Upload the code to the AI Service instance by using CPDCTL:

cpdctl wx-ai ai-service upload-code \
  --space-id $SPACE_ID \
  --ai-service-id $AI_SERVICE_ID \
  --upload-code deployable_code.py.gz

Deploy AI service

Deploy the AI Service by using CPDCTL:

cpdctl wx-ai deployment create \
  --space-id $SPACE_ID \
  --asset-id $AI_SERVICE_ID \
  --name ai-service-deployment \
  --description "Deployed AI Service" \
  --tags ai_service,example,langgraph \
  --online '{}' \
  --output json

Save the URL found in the response responsible for accessing the generate_stream method as URL_STREAM.

Providing custom parameters object

When creating a deployment, you can provide a custom parameters object by using the --custom flag. This flag accepts a JSON object with the key-value pairs.

For example, if you want to provide the following custom object:

{
  "avatar_icon": "Bot",
  "avatar_color": "background",
  "placeholder_image": "image4.png",
  "sample_questions": [
    "Question 1",
    "Question 2"
  ]
}

You can use the following command:

cpdctl wx-ai deployment create \
  --space-id $SPACE_ID \
  --asset-id $AI_SERVICE_ID \
  --name ai-service-deployment \
  --description "Deployed AI Service" \
  --tags ai_service,example,langgraph \
  --online '{}' \
  --output json \
  --custom '{"avatar_icon": "Bot", "avatar_color": "background", "placeholder_image": "image4.png", "sample_questions": ["Question 1", "Question 2"]}'

Alternatively, you can prepare a JSON file with the custom parameters and pass the path to this file using the --custom flag with the @ symbol. For example:

cpdctl wx-ai deployment create \
  --space-id $SPACE_ID \
  --asset-id $AI_SERVICE_ID \
  --name ai-service-deployment \
  --description "Deployed AI Service" \
  --tags ai_service,example,langgraph \
  --online '{}' \
  --output json \
  --custom @path_to_file_with_params.json

Make sure to replace path_to_file_with_params.json with the actual path to your JSON file.

Generating Cloud Identity and Access Management token

To call the deployment endpoint, you must provide an IAM token. Generate an IAM token by using curl:

curl -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -H "Accept: application/json" \
  --data-urlencode "grant_type=urn:ibm:params:oauth:grant-type:apikey" \
  --data-urlencode "apikey=$API_KEY" \
  "https://iam.test.cloud.ibm.com/identity/token"

Save the obtained token as IAM_TOKEN.

Inference the endpoint

Send a request to the endpoint by using curl:

curl -X POST \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $IAM_TOKEN" \
  -d "{\"space_id\": \"$SPACE_ID\", \"url\": \"$WML_URL\", \"question\": \"What is the total sum of the numbers 11,   13, and 20? Remember to always return the final result using the last tool message.\"}" \
  $URL_STREAM

This returns a stream of outputs from a model.

Parent topic: Deploying AI services with direct coding