Manage Minio Storage for Models Storage

How to manage model storage in Wallaroo

Table of Contents

Targeted Role
Dev Ops

Organizations can manage their ML Model storage in their Wallaroo instances through the MinIO interface included in the standard Wallaroo installation.

The following details how to access and the Wallaroo MinIO service. For full details on using the MinIO service, see the MinIO Documentation site.

All of the steps below require administrative access to the Kubernetes service hosting the Wallaroo instance.

Wallaroo MinIO Model Storage

Wallaroo ML Models are stored in the MinIO bucket model-bucket.

Retrieving the Wallaroo MinIO Password

Access to the Wallaroo MinIO service is password protected. DevOps with administrative access to the Kubernetes cluster hosting the Wallaroo instance can retrieve this password with the following:

  • The kubectl command.
  • The namespace the Wallaroo instance is installed to.

This command takes the following format:

kubectl -n {Wallaroo Namespace} get secret minio -o 'jsonpath={.data.rootPassword}' | base64 -d

For example, if the Wallaroo instance is installed into the namespace wallaroo this command would be:

kubectl -n wallaroo get secret minio -o 'jsonpath={.data.rootPassword}' | base64 -d

Accessing the MinIO Service

Access to the MinIO service in a Wallaroo instance is performed either with the Command Line Interface (CLI), or through a browser based User Interface (UI).

Accessing the MinIO Service Through CLI

Access to the MinIO service included with the Wallaroo instance can be performed with the command line tool mc. For more details, see the MinIO Client documentation.

Installing Minio CLI

The following demonstrates installing the mc command for MacOS and Linux.

Installing for MacOS with Brew

MacOS users who have installed Homebrew can install mc with the following:

brew install minio/stable/mc

Installing for Linux

Linux users can install the MinIO CLI tool mc with the following:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/mc

Port Forward for MinIO CLI Access

To access the Wallaroo MinIO service, use Kubernetes port-forward to connect. By default this is on port 9000. This command requires the following:

  • The kubectl command.
  • The namespace the Wallaroo instance is installed to.

This command is in the following format:

kubectl port-forward services/minio 9000:9000 -n {Wallaroo Namespace}

For example, if the Wallaroo instance is installed to the default namespace wallaroo, this command is:

kubectl port-forward services/minio 9000:9000 -n wallaroo

Show MinIO Disk Usage through CLI

To view the Wallaroo MinIO service through the CLI, the following is required:

  • The kubectl command.
  • The namespace the Wallaroo instance is installed to.
  • The MinIO CLI tool mc.
  • The MinIO password.

Here’s an example showing MinIO disk usage via the CLI for the namespace wallaroo. When prompted for the Secret Key, press Enter for none.

The following script displays the space used for a default installation of Wallaroo in the namespace wallaroo.

#!/bin/bash
if kubectl -n wallaroo get secret minio >& /dev/null; then
    u="$(kubectl -n wallaroo get secret minio -o jsonpath='{ .data.rootUser }' | base64 -d)"
    p="$(kubectl -n wallaroo get secret minio -o jsonpath='{ .data.rootPassword }' | base64 -d)"
    creds="$u $p"
fi

mc alias set --insecure wallaroo http://localhost:9000 $creds; mc du --recursive wallaroo

The output:

Added `wallaroo` successfully.
2.2GiB	22 objects	model-bucket
2.2GiB	22 objects

Accessing the MinIO Service Through the UI

The MinIO service included with the Wallaroo instance can be access through the MinIO user interface. By default this is port 9001.

Port Forward for MinIO UI Access

The MinIO UI port 9001 can be access through the kubectl port-forward command.

This command requires the following:

  • The kubectl command.
  • The namespace the Wallaroo instance is installed to.

This command is in the following format:

kubectl port-forward services/minio-console 9001:9001 -n {Wallaroo Namespace}

For example, to port forward through the default installation namespace wallaroo:

kubectl port-forward services/minio-console 9001:9001 -n wallaroo

Accessing the Wallaroo MinIO UI

Once the port forward command is running, the MinIO UI is access through a browser on port 9001 with the user minio and the MinIO administrative password retrieved through the step Retrieving the Minio Administrative Password.

Wallaroo MinIO Login

Viewing General Storage

General disk usage is displayed through Monitoring->Metrics.

Wallaroo MinIO *Monitoring->Metrics

Viewing ML Model Storage

ML Models stored for Wallaroo are accessed through the bucket model-bucket.

model-bucket

Select Browse to view the contents of the model-bucket. To determine the specific file name, access the Name of the object and view the Tags. The file name is access via the Tag file-name.

Objects can be deleted from this bucket with the Delete option.

Delete Model