Wallaroo pipelines can be published to a Edge Open Container Initiative (OCI) Registry Service, known here as the Edge Registry Service, as a container images. This allows the Wallaroo pipelines to be deployed in other environments, such as Docker or Kubernetes with all of the pipeline model. When deployed, these pipelines can perform inferences from the ML models exactly as if they were deployed as part of a Wallaroo instance.

When a pipeline is updated with new model steps or deployment configurations, the updated pipeline is republished to the Edge Registry as a new repo and version. This allows DevOps engineers to update an Wallaroo pipeline in any container supporting environment with the new versions of the pipeline.

Pipeline Publishing Flow

A typical ML Model and Pipeline deployment to Wallaroo Ops and to remote locations as a Wallaroo Inference server is as follows:

• Components:
  • Wallaroo Ops: The Wallaroo Ops provides the backbone services for ML Model deployment. This is where ML models are uploaded, pipelines created and deployed for inferencing, pipelines published to OCI compliant registries, and other functions.
  • Wallaroo Inference Server: A remote deployment of a published Wallaroo pipeline with the Wallaroo Inference Engine outside the Wallaroo Ops instance. When the edge name is added to a Wallaroo publish, the Wallaroo Inference Server’s inference logs are submitted to the Wallaroo Ops instance. These inference logs are stored as part of the Wallaroo pipeline the remote deployment is published from.

1. DevOps:
   1. Add Edge Publishing and Edge Observability to the Wallaroo Ops center. See Edge Deployment Registry Guide for details on updating the Wallaroo instance with Edge Publishing and Edge Observability.
2. Data Scientists:
   1. Develop and train models.
   2. Test their deployments in Wallaroo Ops Center as Pipelines with:
      1. Pipeline Steps: The models part of the inference flow.
      2. Pipeline Deployment Configurations: CPUs, RAM, GPU, and Architecture settings to run the pipeline.
   3. Publish the Pipeline from the Wallaroo Ops to an OCI Registry: Store a image version of the Pipeline with models and pipeline configuration into the OCI Registry set by the DevOps engineers as the Wallaroo Edge Registry Service.
3. DevOps:
   1. Retrieve the new or updated Wallaroo published pipeline from the Wallaroo Edge Registry Service.
   2. (Optional): Add an edge to the Wallaroo publish. This provides the EDGE_BUNDLE with the credentials for the Wallaroo Inference Server to transmit its inference result logs back to the Wallaroo Ops instance. These inference logs are added to the originating Wallaroo pipeline, labeled with the metadata.partition being the name of the edge deployed Wallaroo Inference server. For more details, see Wallaroo SDK Essentials Guide: Pipeline Edge Publication: Edge Observability
   3. Deploy the Pipeline as a Wallaroo Inference Server as a Docker or Kubernetes container, updating the resource allocations as part of the Helm chart, Docker Compose file, etc.

Enable Wallaroo Edge Registry

Set Edge Registry Service

Wallaroo Pipeline Publishes aka Wallaroo Servers are automatically routed to the Edge Open Container Initiative (OCI) Registry Service registered in the Wallaroo instance. This is enabled through either the Kots Administrative Dashboard through kots, or by enabling it through a helm chart setting. From here on out, we will refer to it as the Edge Registry Service.

Set Edge Registry Service through Kots

To set the Edge Registry Settings through the Kots Administrative Dashboard:

1. Launch the Kots Administrative Dashboard using the following command, replacing the --namespace parameter with the Kubernetes namespace for the Wallaroo instance:

   kubectl kots admin-console --namespace wallaroo
   

2. Open a browser at the URL detailed in the step above and authenticate using the console password set as described in the as detailed in the Wallaroo Install Guides.

3. From the top menu, select Config then scroll to Edge Deployment.

4. Enable Provide OCI registry credentials for pipelines.

5. Enter the following:

   

   1. Registry URL: The address of the registry service. For example: us-west1-docker.pkg.dev.
   2. email: The email address of the user account used to authenticate to the service.
   3. username: The account used to authenticate to the registry service.
   4. password: The password or token used to authenticate to the registry service.
   5. To enable edge observability, enable Enable pipelines deployed on the edge to send data back to the OpsCenter.

6. Save the updated configuration, then deploy it. Once complete, the edge registry settings will be available.

Set Edge Registry Service through Helm

The helm settings for adding the Edge Server configuration details are set through the ociRegistry element, with the following settings.

• ociRegistry: Sets the Edge Server registry information.
  • enabled: true enables the Edge Server registry information, false disables it.
  • registry: The registry url. For example: reg.big.corp:3579.
  • repository: The repository within the registry. This may include the cloud account, or the full path where the Wallaroo published pipelines should be kept. For example: account123/wallaroo/pipelines.
  • email: Optional field to track the email address of the registry credential.
  • username: The username to the registry. This may vary based on the provider. For example, GCP Artifact Registry with service accounts uses the username _json_key_base64 with the password as a base64 processed token of the credential information.
  • password: The password or token for the registry service.

Set Edge Observability Service

Edge Observability allows published Wallaroo Servers to community with the Wallaroo Ops center to update their associated Wallaroo Pipeline with inference results, visible in the Pipeline logs.

The edge sites will report their inference results to the address $WALLAROODOMAIN/edge. For example: If the Wallaroo Domain Name is wallaroo.example.com, then the Edge Observability Service address is wallaroo.example.com/edge.

Set Edge Observability Service through Kots

To enable Edge Observability using the Kots Administrative Dashboard for kots installed instances of Wallaroo Ops:

1. Launch the Kots Administrative Dashboard using the following command, replacing the --namespace parameter with the Kubernetes namespace for the Wallaroo instance:

   kubectl kots admin-console --namespace wallaroo
   

2. Open a browser at the URL detailed in the step above and authenticate using the console password set as described in the as detailed in the Wallaroo Install Guides.

3. Access Config and scroll to Edge Deployment and enable Enable pipelines deployed on the edge to send data back to the OpsCenter.

4. Set the following:

   

   1. Enable Enable pipelines deployed on the edge to send data back to the OpsCenter.

5. Save the updated configuration, then deploy it. Once complete, the edge observability service is available.

Set Edge Observability Service through Helm

To enable the Edge Observability Service for Wallaroo Ops Helm based installation, include the following variables for the helm settings. For these instructions they are stored in local-values.yaml:

edgelb:
    enabled: true

pipelines:
    enabled: true

Update the Wallaroo Helm installation with the same version as the Wallaroo ops and the channel. For example, if updating Wallaroo Enterprise server, use the following:

helm upgrade wallaroo oci://registry.replicated.com/wallaroo/2025-1/wallaroo --version 2025.1.0-6290 --values local-values.yaml --timeout 10m --wait --wait-for-jobs

This process will take 5-15 minutes depending on other configuration options.

Registry Setup Guides

The following are short guides for setting up the credentials for different registry services. Refer to the registry documentation for full details.

• Environments:
• GitHub Container Registry
• GCP Artifact Registry

docker login -u {Github Username} --password {Your Token} ghcr.io/{Your Github Username or Organization}

REPOSITORY_NAME="YOUR NAME"
LOCATION="us-west1"
DESCRIPTION="My amazing registry."

gcloud artifacts repositories create REPOSITORY \
    --repository-format=docker \
    --location=LOCATION \
    --description="$DESCRIPTION" \
    --async

KEY_FILE = ~/.gcp-sa-registry-keyfile.json
KEYFILEBASE64 = ~/.gcp-sa-registry-keyfile-b64.json
base64 -i $KEY_FILE -o $KEYFILEBASE64

token=$(cat $KEYFILEBASE64)
cat $tok | docker login -u _json_key_base64 --password-stdin https://{GCP artifact registry region}.pkg.dev

Pipeline Publish Endpoints

Publish a Pipeline

Pipelines are published as images to the edge registry set in the Enable Wallaroo Edge Registry through the following endpoint.

• Endpoint:
  • /v1/api/pipelines/publish
• Parameters
  • pipeline_id (Integer Required): The numerical id of the pipeline to publish to the edge registry.
  • pipeline_version_id (Integer Required): The numerical id of the pipeline’s version to publish to the edge registry.
  • model_config_ids (List Optional): The list of model config ids to include.
• Returns
  • id (Integer): Numerical Wallaroo id of the published pipeline.
  • pipeline_version_id (Integer): Numerical Wallaroo id of the pipeline version published.
  • status: The status of the pipeline publish. Values include:
    • PendingPublish: The pipeline publication is about to be uploaded or is in the process of being uploaded.
    • Published: The pipeline is published and ready for use.
  • created_at (String): When the published pipeline was created.
  • updated_at (String): When the published pipeline was updated.
  • created_by (String): The email address of the Wallaroo user that created the pipeline publish.
  • pipeline_url (String): The URL of the published pipeline in the edge registry. May be null until the status is Published)
  • engine_url (String): The URL of the published pipeline engine in the edge registry. May be null until the status is Published.
  • helm: The Helm chart information including the following fields:
    • reference (String): The Helm reference.
    • chart (String): The Helm chart URL.
    • version (String): The Helm chart version.
  • engine_config (*wallaroo.deployment_config.DeploymentConfig) | The pipeline configuration included with the published pipeline.

Publish a Pipeline Example

The following example shows how to publish a pipeline to the edge registry service associated with the Wallaroo instance.

# get the pipeline version
pipeline_version_id = pipeline.versions()[-1].id()

pipeline_id = pipeline.id()

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}

url = f"{wl.api_endpoint}/v1/api/pipelines/publish"
response = requests.post(
    url,
    headers=headers,
    json={"pipeline_id": 1, 
          "pipeline_version_id": pipeline_version, 
          "model_config_ids": []
          }
    )
display(response.json())

{'id': 10,
 'pipeline_version_id': 30,
 'pipeline_version_name': 'ab7ab48f-9aab-4c3d-9f66-097477bd34bf',
 'status': 'PendingPublish',
 'created_at': '2023-08-28T14:54:54.14098+00:00',
 'updated_at': '2023-08-28T14:54:54.14098+00:00',
 'created_by': 'db364f8c-b866-4865-96b7-0b65662cb384',
 'pipeline_url': None,
 'engine_url': None,
 'helm': None,
 'engine_config': {'engine': {'resources': {'limits': {'cpu': 4.0,
     'memory': '3Gi'},
    'requests': {'cpu': 4.0, 'memory': '3Gi'}}},
  'enginelb': {'resources': {'limits': {'cpu': 1.0, 'memory': '512Mi'},
    'requests': {'cpu': 0.2, 'memory': '512Mi'}}},
  'engineAux': {}}}

Get Publish Status

The status of a created Wallaroo pipeline publish is available through the following endpoint.

• Endpoint:
  • /v1/api/pipelines/get_publish_status
• Parameters
  • publish_id (Integer Required): The numerical id of the pipeline publish to retrieve.
• Returns
  • id (Integer): Numerical Wallaroo id of the published pipeline.
  • pipeline_version_id (Integer): Numerical Wallaroo id of the pipeline version published.
  • status: The status of the pipeline publish. Values include:
    • PendingPublish: The pipeline publication is about to be uploaded or is in the process of being uploaded.
    • Published: The pipeline is published and ready for use.
  • created_at (String): When the published pipeline was created.
  • updated_at (String): When the published pipeline was updated.
  • created_by (String): The email address of the Wallaroo user that created the pipeline publish.
  • pipeline_url (String): The URL of the published pipeline in the edge registry. May be null until the status is Published)
  • engine_url (String): The URL of the published pipeline engine in the edge registry. May be null until the status is Published.
  • helm: The Helm chart information including the following fields:
    • reference (String): The Helm reference.
    • chart (String): The Helm chart URL.
    • version (String): The Helm chart version.
  • engine_config (*wallaroo.deployment_config.DeploymentConfig) | The pipeline configuration included with the published pipeline.

Get Publish Status Example

The following example shows retrieving the status of a recently created pipeline publish.

# Publish Status

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}
url = f"{wl.api_endpoint}/v1/api/pipelines/get_publish_status"
response = requests.post(
    url,
    headers=headers,
    json={
        "id": publish_id
        },
)
display(response.json())

{'id': 10,
 'pipeline_version_id': 30,
 'pipeline_version_name': 'ab7ab48f-9aab-4c3d-9f66-097477bd34bf',
 'status': 'Published',
 'created_at': '2023-08-28T14:54:54.14098+00:00',
 'updated_at': '2023-08-28T14:54:54.14098+00:00',
 'created_by': 'db364f8c-b866-4865-96b7-0b65662cb384',
 'pipeline_url': 'ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:ab7ab48f-9aab-4c3d-9f66-097477bd34bf',
 'engine_url': 'ghcr.io/wallaroolabs/doc-samples/engine:v2023.3.0-main-3731',
 'helm': {'reference': 'ghcr.io/wallaroolabs/doc-samples/charts@sha256:dbb46f807dbd75840cf1433daee51f4cc272791da5836366bfb8b2b863abf63b',
  'chart': 'ghcr.io/wallaroolabs/doc-samples/charts/edge-pipeline',
  'version': '0.0.1-ab7ab48f-9aab-4c3d-9f66-097477bd34bf'},
 'engine_config': {'engine': {'resources': {'limits': {'cpu': 4.0,
     'memory': '3Gi'},
    'requests': {'cpu': 4.0, 'memory': '3Gi'}}},
  'enginelb': {'resources': {'limits': {'cpu': 1.0, 'memory': '512Mi'},
    'requests': {'cpu': 0.2, 'memory': '512Mi'}}},
  'engineAux': {}}}

List Publishes for a Pipeline

A list of publishes created for a specific pipeline is retrieved through the following endpoint.

• Endpoint
  • /v1/api/pipelines/list_publishes_for_pipeline
• Parameters
  • publish_id (Integer Required): The numerical id of the pipeline to retrieve all publishes for.
• Returns a List of pipeline publishes with the following fields:
  • id (Integer): Numerical Wallaroo id of the published pipeline.
  • pipeline_version_id (Integer): Numerical Wallaroo id of the pipeline version published.
  • status: The status of the pipeline publish. Values include:
    • PendingPublish: The pipeline publication is about to be uploaded or is in the process of being uploaded.
    • Published: The pipeline is published and ready for use.
  • created_at (String): When the published pipeline was created.
  • updated_at (String): When the published pipeline was updated.
  • created_by (String): The email address of the Wallaroo user that created the pipeline publish.
  • pipeline_url (String): The URL of the published pipeline in the edge registry. May be null until the status is Published)
  • engine_url (String): The URL of the published pipeline engine in the edge registry. May be null until the status is Published.
  • helm: The Helm chart information including the following fields:
    • reference (String): The Helm reference.
    • chart (String): The Helm chart URL.
    • version (String): The Helm chart version.
  • engine_config (*wallaroo.deployment_config.DeploymentConfig) | The pipeline configuration included with the published pipeline.

List Publishes for a Pipeline Example

The following retrieves a list of all publishes for a specific pipeline.

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}

url = f"{wl.api_endpoint}/v1/api/pipelines/list_publishes_for_pipeline"

response = requests.post(
    url,
    headers=headers,
    json={"pipeline_id": pipeline_id},
)
display(response.json())

{'pipelines': [{'id': 10,
   'pipeline_version_id': 30,
   'pipeline_version_name': 'ab7ab48f-9aab-4c3d-9f66-097477bd34bf',
   'status': 'Published',
   'created_at': '2023-08-28T14:54:54.14098+00:00',
   'updated_at': '2023-08-28T14:54:54.14098+00:00',
   'created_by': 'db364f8c-b866-4865-96b7-0b65662cb384',
   'pipeline_url': 'ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:ab7ab48f-9aab-4c3d-9f66-097477bd34bf',
   'engine_url': 'ghcr.io/wallaroolabs/doc-samples/engine:v2023.3.0-main-3731',
   'helm': {'reference': 'ghcr.io/wallaroolabs/doc-samples/charts@sha256:dbb46f807dbd75840cf1433daee51f4cc272791da5836366bfb8b2b863abf63b',
    'chart': 'ghcr.io/wallaroolabs/doc-samples/charts/edge-pipeline',
    'version': '0.0.1-ab7ab48f-9aab-4c3d-9f66-097477bd34bf'},
   'engine_config': {'engine': {'resources': {'limits': {'cpu': 4.0,
       'memory': '3Gi'},
      'requests': {'cpu': 4.0, 'memory': '3Gi'}}},
    'enginelb': {'resources': {'limits': {'cpu': 1.0, 'memory': '512Mi'},
      'requests': {'cpu': 0.2, 'memory': '512Mi'}}},
    'engineAux': {}}}]}

Edge Observability

Edge Observability allows edge deployments of Wallaroo Server to transmit inference results back to the Wallaroo Ops center and become part of the pipeline’s logs. This is valuable for data scientists and MLOps engineers to retrieve edge deployment logs for use in model observability, drift, and other use cases.

Before starting, the Edge Observability Service must be enabled in the Wallaroo Ops center. See the Edge Deployment Registry Guide for details on enabling the Wallaroo Edge Deployment service.

Wallaroo Server edge observability is enabled when a new edge location is added to the pipeline publish. Each location has its own EDGE_BUNDLE settings, a Base64 encoded set of instructions informing the edge deployed Wallaroo Server on how to communicate with Wallaroo Ops center.

Add Publish Edge

Edges are added to an existing pipeline publish with the following endpoint.

• Endpoint
  • /v1/api/pipelines/add_edge_to_publish
• Parameters
  • name (String Required): The name of the edge. This must be a unique value across all edges in the Wallaroo instance.
  • pipeline_publish_id (Integer Required): The numerical identifier of the pipeline publish to add this edge to.
  • tags (List[String] Optional): A list of optional tags.
• Returns
  • id (Integer): The integer ID of the pipeline publish.
  • created_at: (String): The DateTime of the pipeline publish.
  • docker_run_variables (String) The Docker variables in base64 encoded format that include the following: The BUNDLE_VERSION, EDGE_NAME, JOIN_TOKEN_, OPSCENTER_HOST, PIPELINE_URL, and WORKSPACE_ID.
  • engine_config (String): The Wallaroo wallaroo.deployment_config.DeploymentConfig for the pipeline.
  • pipeline_version_id (Integer): The integer identifier of the pipeline version published.
  • status (String): The status of the publish. Published is a successful publish.
  • updated_at (DateTime): The DateTime when the pipeline publish was updated.
  • user_images (List[String]): User images used in the pipeline publish.
  • created_by (String): The UUID of the Wallaroo user that created the pipeline publish.
  • engine_url (String): The URL for the published pipeline’s Wallaroo engine in the OCI registry.
  • error (String): Any errors logged.
  • helm (String): The helm chart, helm reference and helm version.
  • pipeline_url (String): The URL for the published pipeline’s container in the OCI registry.
  • pipeline_version_name (String): The UUID identifier of the pipeline version published.
  • additional_properties (String): Any additional properties.

Add Publish Edge Example

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}

edge_name = f"ccfraud-observability-api{suffix_random}"

url = f"{wl.api_endpoint}/v1/api/pipelines/add_edge_to_publish"

response = requests.post(
    url,
    headers=headers,
    json={
        "pipeline_publish_id": publish_id,
        "name": edge_name,
        "tags": []
    }
)
display(response.json())

{'id': 9,
 'pipeline_version_id': 17,
 'pipeline_version_name': 'cd70ed84-2db7-4fc5-a766-482686ca3fcc',
 'status': 'Published',
 'created_at': '2023-10-30T19:04:16.758117+00:00',
 'updated_at': '2023-10-30T19:04:16.758117+00:00',
 'created_by': '1b26fc10-d0f9-4f92-a1a2-ab342b3d069e',
 'pipeline_url': 'ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:cd70ed84-2db7-4fc5-a766-482686ca3fcc',
 'engine_url': 'ghcr.io/wallaroolabs/doc-samples/engines/proxy/wallaroo/ghcr.io/wallaroolabs/standalone-mini:v2023.4.0-main-4079',
 'helm': {'reference': 'ghcr.io/wallaroolabs/doc-samples/charts@sha256:6f557685fb6e7509dcee4e09af1ece94df1dcc9886ef19a9adc6c015e9a7279b',
  'chart': 'ghcr.io/wallaroolabs/doc-samples/charts/edge-pipeline',
  'version': '0.0.1-cd70ed84-2db7-4fc5-a766-482686ca3fcc',
  'values': {'edgeBundle': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT1jY2ZyYXVkLW9ic2VydmFiaWxpdHktYXBpZnpkdgpleHBvcnQgSk9JTl9UT0tFTj1kYTM2Njk3Zi1iYWIyLTQyYTktYmNmNy03MTk5YWU0YWVkZjkKZXhwb3J0IE9QU0NFTlRFUl9IT1NUPWRvYy10ZXN0LmVkZ2Uud2FsbGFyb29jb21tdW5pdHkubmluamEKZXhwb3J0IFBJUEVMSU5FX1VSTD1naGNyLmlvL3dhbGxhcm9vbGFicy9kb2Mtc2FtcGxlcy9waXBlbGluZXMvZWRnZS1waXBlbGluZTpjZDcwZWQ4NC0yZGI3LTRmYzUtYTc2Ni00ODI2ODZjYTNmY2MKZXhwb3J0IFdPUktTUEFDRV9JRD05'}},
 'engine_config': {'engine': {'resources': {'limits': {'cpu': 4.0,
     'memory': '3Gi'},
    'requests': {'cpu': 4.0, 'memory': '3Gi'}}},
  'enginelb': {'resources': {'limits': {'cpu': 1.0, 'memory': '512Mi'},
    'requests': {'cpu': 0.2, 'memory': '512Mi'}}},
  'engineAux': {}},
 'error': None,
 'user_images': [],
 'docker_run_variables': {'EDGE_BUNDLE': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT1jY2ZyYXVkLW9ic2VydmFiaWxpdHktYXBpZnpkdgpleHBvcnQgSk9JTl9UT0tFTj1kYTM2Njk3Zi1iYWIyLTQyYTktYmNmNy03MTk5YWU0YWVkZjkKZXhwb3J0IE9QU0NFTlRFUl9IT1NUPWRvYy10ZXN0LmVkZ2Uud2FsbGFyb29jb21tdW5pdHkubmluamEKZXhwb3J0IFBJUEVMSU5FX1VSTD1naGNyLmlvL3dhbGxhcm9vbGFicy9kb2Mtc2FtcGxlcy9waXBlbGluZXMvZWRnZS1waXBlbGluZTpjZDcwZWQ4NC0yZGI3LTRmYzUtYTc2Ni00ODI2ODZjYTNmY2MKZXhwb3J0IFdPUktTUEFDRV9JRD05'}}

Remove Publish Edge

Edges are removed from an existing pipeline publish with the following endpoint.

• Endpoint
  • /v1/api/pipelines/remove_edge_from_publish
• Parameters
  • name (String Required): The name of the edge. This must be a unique value across all edges in the Wallaroo instance.
• Returns
  • null

Remove Publish Edge Example

The following example will add an edge to a pipeline publish, then remove the same edge.

# adding another edge to remove

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}

edge_name2 = f"ccfraud-observability-api2{suffix_random}"

url = f"{wl.api_endpoint}/v1/api/pipelines/add_edge_to_publish"

response = requests.post(
    url,
    headers=headers,
    json={
        "pipeline_publish_id": publish_id,
        "name": edge_name2,
        "tags": []
    }
)
display(response.json())

{'id': 10,
 'pipeline_version_id': 17,
 'pipeline_version_name': 'cd70ed84-2db7-4fc5-a766-482686ca3fcc',
 'status': 'Published',
 'created_at': '2023-10-30T19:50:32.164154+00:00',
 'updated_at': '2023-10-30T19:50:32.164154+00:00',
 'created_by': '1b26fc10-d0f9-4f92-a1a2-ab342b3d069e',
 'pipeline_url': 'ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:cd70ed84-2db7-4fc5-a766-482686ca3fcc',
 'engine_url': 'ghcr.io/wallaroolabs/doc-samples/engines/proxy/wallaroo/ghcr.io/wallaroolabs/standalone-mini:v2023.4.0-main-4079',
 'helm': {'reference': 'ghcr.io/wallaroolabs/doc-samples/charts@sha256:cfd0e77347ec46b6d3dec2c2cc83733c353c5b583428b5c6f4fca11277a2c43e',
  'chart': 'ghcr.io/wallaroolabs/doc-samples/charts/edge-pipeline',
  'version': '0.0.1-cd70ed84-2db7-4fc5-a766-482686ca3fcc',
  'values': {'edgeBundle': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT1jY2ZyYXVkLW9ic2VydmFiaWxpdHktYXBpMmtwd2wKZXhwb3J0IEpPSU5fVE9LRU49NWVmZmIyYzYtZTNmNi00MmY1LWFhYjAtMWE5YTNiN2YwMTIyCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6Y2Q3MGVkODQtMmRiNy00ZmM1LWE3NjYtNDgyNjg2Y2EzZmNjCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9OQ=='}},
 'engine_config': {'engine': {'resources': {'limits': {'cpu': 4.0,
     'memory': '3Gi'},
    'requests': {'cpu': 4.0, 'memory': '3Gi'}}},
  'enginelb': {'resources': {'limits': {'cpu': 1.0, 'memory': '512Mi'},
    'requests': {'cpu': 0.2, 'memory': '512Mi'}}},
  'engineAux': {}},
 'error': None,
 'user_images': [],
 'docker_run_variables': {'EDGE_BUNDLE': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT1jY2ZyYXVkLW9ic2VydmFiaWxpdHktYXBpMmtwd2wKZXhwb3J0IEpPSU5fVE9LRU49NWVmZmIyYzYtZTNmNi00MmY1LWFhYjAtMWE5YTNiN2YwMTIyCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6Y2Q3MGVkODQtMmRiNy00ZmM1LWE3NjYtNDgyNjg2Y2EzZmNjCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9OQ=='}}

# remove the edge

headers = {
    "Authorization": wl.auth._bearer_token_str(),
    "Content-Type": "application/json",
}

edge_name2 = f"ccfraud-observability-api2{suffix_random}"

url = f"{wl.api_endpoint}/v1/api/pipelines/remove_edge_from_publish"

response = requests.post(
    url,
    headers=headers,
    json={
        "name": edge_name2,
    }
)
display(response.json())

null

Edge Bundle Token TTL

When an edge is added to a pipeline publish, the field docker_run_variables contains a JSON value for edge devices to connect to the Wallaroo Ops instance. The settings are stored in the key EDGE_BUNDLE as a base64 encoded value that include the following:

• BUNDLE_VERSION: The current version of the bundled Wallaroo pipeline.
• EDGE_NAME: The edge name as defined when created and added to the pipeline publish.
• JOIN_TOKEN_: The one time authentication token for authenticating to the Wallaroo Ops instance.
• OPSCENTER_HOST: The hostname of the Wallaroo Ops edge service. See Edge Deployment Registry Guide for full details on enabling pipeline publishing and edge observability to Wallaroo.
• PIPELINE_URL
• WORKSPACE_ID.

For example:

{'edgeBundle': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT14Z2ItY2NmcmF1ZC1lZGdlLXRlc3QKZXhwb3J0IEpPSU5fVE9LRU49MzE0OGFkYTUtMjg1YS00ZmNhLWIzYjgtYjUwYTQ4ZDc1MTFiCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6ZjM4OGMxMDktOGQ1Ny00ZWQyLTk4MDYtYWExM2Y4NTQ1NzZiCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9NQ=='}

base64 -D
ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT14Z2ItY2NmcmF1ZC1lZGdlLXRlc3QKZXhwb3J0IEpPSU5fVE9LRU49MzE0OGFkYTUtMjg1YS00ZmNhLWIzYjgtYjUwYTQ4ZDc1MTFiCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6ZjM4OGMxMDktOGQ1Ny00ZWQyLTk4MDYtYWExM2Y4NTQ1NzZiCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9NQ==^D
export BUNDLE_VERSION=1
export EDGE_NAME=xgb-ccfraud-edge-test
export JOIN_TOKEN=3148ada5-285a-4fca-b3b8-b50a48d7511b
export OPSCENTER_HOST=doc-test.wallaroocommunity.ninja/edge
export PIPELINE_URL=ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:f388c109-8d57-4ed2-9806-aa13f854576b
export WORKSPACE_ID=5

The JOIN_TOKEN is a one time access token. Once used, a JOIN_TOKEN expires. The authentication session data is stored in persistent volumes. Persistent volumes must be specified for docker and docker compose based deployments of Wallaroo pipelines; helm based deployments automatically provide persistent volumes to store authentication credentials.

The JOIN_TOKEN has the following time to live (TTL) parameters.

• Once created, the JOIN_TOKEN is valid for 24 hours. After it expires the edge will not be allowed to contact the OpsCenter the first time and a new edge bundle will have to be created.
• After an Edge joins to Wallaroo Ops for the first time with persistent storage, the edge must contact the Wallaroo Ops instance at least once every 7 days.
  • If this period is exceeded, the authentication credentials will expire and a new edge bundle must be created with a new and valid JOIN_TOKEN.

Wallaroo edges require unique names. To create a new edge bundle with the same name:

• Use the Remove Edge to remove the edge by name.
• Use Add Edge to add the edge with the same name. A new EDGE_BUNDLE is generated with a new JOIN_TOKEN.

Pipeline Edge Deployment

Once a pipeline is deployed to the Edge Registry service, it can be deployed in environments such as Docker, Kubernetes, or similar container running services by a DevOps engineer.

docker run \
    -p $EDGE_PORT:8080 \
    -e OCI_USERNAME=$OCI_USERNAME \
    -e OCI_PASSWORD=$OCI_PASSWORD \
    -e PIPELINE_URL=sample-pipeline-url \
    -e CONFIG_CPUS=1.0 --gpus all --cpus=1.25 --memory=1536m \
    sample-engine-url

services:
  engine:
    image: {Your Engine URL}
    ports:
      - 8080:8080
    environment:
      PIPELINE_URL: {Your Pipeline URL}
      OCI_REGISTRY: {Your Edge Registry URL}
      OCI_USERNAME:  {Your Registry Username}
      OCI_PASSWORD: {Your Token or Password}
      CONFIG_CPUS: 4

docker compose up
[+] Running 1/1
 ✔ Container cv_data-engine-1  Recreated                                                                                                                                                                 0.5s
Attaching to cv_data-engine-1
cv_data-engine-1  | Wallaroo Engine - Standalone mode
cv_data-engine-1  | Login Succeeded
cv_data-engine-1  | Fetching manifest and config for pipeline: sample-registry.com/pipelines/edge-cv-retail:bf70eaf7-8c11-4b46-b751-916a43b1a555
cv_data-engine-1  | Fetching model layers
cv_data-engine-1  | digest: sha256:c6c8869645962e7711132a7e17aced2ac0f60dcdc2c7faa79b2de73847a87984
cv_data-engine-1  |   filename: c6c8869645962e7711132a7e17aced2ac0f60dcdc2c7faa79b2de73847a87984
cv_data-engine-1  |   name: resnet-50
cv_data-engine-1  |   type: model
cv_data-engine-1  |   runtime: onnx
cv_data-engine-1  |   version: 693e19b5-0dc7-4afb-9922-e3f7feefe66d
cv_data-engine-1  |
cv_data-engine-1  | Fetched
cv_data-engine-1  | Starting engine
cv_data-engine-1  | Looking for preexisting `yaml` files in //modelconfigs
cv_data-engine-1  | Looking for preexisting `yaml` files in //pipelines