Model and Orchestration Storage

How to manage model and orchestration storage in Wallaroo

Table of Contents

Organizations manage their model and orchestration storage through Model Admin and Workload Admin interfaces by deleting existing models and workloads to free space.

The following details how to expand the storage capacity for these services.

Increase Model and Workload Storage

Storage capacity is managed by the Wallaroo MinIO service. For full details on using the MinIO service, see the MinIO Documentation site.

Storage capacity increases for MinIO requires that AllowVolumeExpansion: True is set in your cluster. Check with your cloud service provider to verify this setting can be enabled.

The following procedures are based on the method for installing Wallaroo:

  • kots
  • helm

Increase Model and Workload Storage for Kots

For installation of Wallaroo via kots:

  1. Access the Kots Administrative Dashboard with the kubectl kots command, replacing $NAMESPACE with the Kubernetes namespace used to install Wallaroo. By default this is wallaroo.

    kubectl kots admin-console --namespace $NAMESPACE

  2. A URL will be provided to access the dashboard in a browser. By default, this is http://localhost:8800.

  3. Select Config, then the field Wallaroo ML Engine->Size of model data store. In units of Gi, which must be provided.

  4. Update the storage capacity in terms of Gi. By default, this is 10Gi.

  5. Once updated, scroll to the bottom and select Save config, then deploy the new configuration.

Increase Model and Workload for Helm

To increase the model storage size for helm based installations of Wallaroo:

  1. Edit your local values.yaml with the following fragment:

    minio:
  persistence:
    size: 25Gi # Minio model storage disk size. Smaller than 10Gi is not recommended.

  2. Apply the update with helm upgrade. For example, to upgrade the registration wallaroo from the EE channel the command would be:

    helm upgrade wallaroo oci://registry.replicated.com/wallaroo/2024-3/wallaroo --version 2024.3 --values local-values.yaml --timeout 10m

Troubleshooting

If the PersistentVolumeClaim (PVC) status still shows Waiting, check with the PersistentVolume (PV) settings and verify that the storage settings match the PVC.