You can use the manual setup to configure machine learning model evaluations. With manual setup, you can use existing assets, such as databases and deployment spaces. You can also choose the environment type (pre-production or production) for your deployment. Unlike the auto setup, the manual setup does not install sample assets to demonstrate model evaluations.
Running the manual setup
Follow these steps to start the manual setup:
- Start Watson OpenScale from your account.
- Choose the Manual setup option.
The System setup page opens. To finish the manual setup, you must complete the steps that are described in the following sections.
Adding a database connection
Connect to a database to store model transactions and model evaluation results. You can use a Free lite plan database to get started. Alternatively, if you have an existing EDB Postgres or Db2 database, you can use it to evaluate models. You can also purchase a new database.
Follow these steps to add a database connection for model evaluations:
-
Ensure that the Database tab is open in the System setup page. Click the Edit
icon. -
Choose the database type:
-
To use the database at no cost, select the Free lite plan database from the list.
-
To use an existing database, choose one of the following options:
a. Databases for EDB
b. Db2
c. Db2 Warehouse -
To purchase a new database, click Purchase a database.
-
-
Complete the details for your database connection and click Save.
Limitations
- The database and IBM watsonx.ai Runtime instance must be deployed in the same account.
- You can use a EDB Postgres or Db2 database to store model-related data (feedback data, scoring payload) and calculated metrics. Lite Db2 plans are not currently supported.
- The free Lite plan database is not GDPR-compliant. If your model processes personally identifiable information (PII), you must purchase a new database or use an existing database that does conform to GDPR rules.
Setting up a machine learning provider
You can connect to deployed models stored in a machine learning environment, including pre-production and production environments. The following machine learning service providers are available for model evaluations:
- watsonx.ai Runtime
- Amazon SageMaker
- Microsoft Azure ML Studio
- Microsoft Azure ML Service
You can also use a custom service environment.
Follow these steps to connect to a machine learning provider for model evaluations:
- In the Machine learning providers section, click Add machine learning provider.
- Optional: To change the default name, click the Edit icon beside Machine learning providers.
- Optional: To enter a description, click the Edit icon beside Description.
- To enter connection information, click the Edit icon beside Connection.
- Choose a Service provider and specify the connection details.
- Click Save.
Managing users and roles
You must add the users that you want to have access to your model evaluations and assign roles to determine which tasks they can complete.
Learn more
Parent topic: Setup options for model evaluations