Model Endpoints Guide

Enable external deployment URLs to perform inferences through API calls.

Wallaroo provides the ability to perform inferences through deployed pipelines via both internal and external inferences URLs. These URLs allow inferences to be performed by submitting data to the internal or external inferences URL, with the inference results returned in the same format as the InferenceResult Object.

Internal URLs are available only through the internal Kubernetes environment hosting the Wallaroo instance.
External URLs are available outside of the Kubernetes environment, such as the public internet. Authentication will be required to connect to these external deployment URLs.

The following process will enable external inference URLs

  1. Enable external URl inference endpoints through the Kots Administrative Dashboard or through helm setup. This can be accessed through the kots or helm as detailed in the Wallaroo Install Guides and the How to Install Wallaroo Enterprise via Helm guides.

    helm users can update the configuration and enable endpoints by setting the apilb\external_inference_endpoints_enabled to true as follows:

        # Required to perform remote inferences either through the SDK or the API
        external_inference_endpoints_enabled: true

    For kots users: To access the Kots Administrative Dashboard:

    1. From a terminal shell connected to the Kubernetes environment hosting the Wallaroo instance, run the following kots command:
    kubectl kots admin-console --namespace wallaroo

    This provides the following standard output:

      • Press Ctrl+C to exit
      • Go to http://localhost:8800 to access the Admin Console

    This will host a http connection to the Kots Administrative Dashboard, by default at http://localhost:8800.

    1. Open a browser at the URL detailed in the step above and authenticate using the console password set as described in the as detailed in the Wallaroo Install Guides.

    2. From the top menu, select Config then verify that Networking Configuration -> Ingress Mode for Wallaroo interactive services -> Enable external URL inference endpoints is enabled.

      Endpoint enabled
    3. Save the updated configuration, then deploy it. Once complete, the external URL inference endpoints will be enabled.