Edge Deployment Endpoints

The following endpoints are available for API calls to the edge deployed pipeline.

List Pipelines

The endpoint GET /pipelines returns:

• id (String): The name of the pipeline.
• status (String): The status as either Running, or Error if there are any issues.

List Pipelines Example

curl localhost:8080/pipelines
{"pipelines":[{"id":"edge-cv-retail","status":"Running"}]}

List Models

The endpoint GET /models returns a List of models with the following fields:

• name (String): The model name.
• sha (String): The sha hash value of the ML model.
• status (String): The status of either Running or Error if there are any issues.
• version (String): The model version. This matches the version designation used by Wallaroo to track model versions in UUID format.

List Models Example

curl localhost:8080/models
{"models":[{"name":"resnet-50","sha":"c6c8869645962e7711132a7e17aced2ac0f60dcdc2c7faa79b2de73847a87984","status":"Running","version":"693e19b5-0dc7-4afb-9922-e3f7feefe66d"}]}

Edge Inference Endpoints

The inference endpoint takes the following patterns:

• POST /infer: The static inference endpoint. If a model deployment is updated or a new pipeline publish replaces a previous one, the /infer endpoint always points to the current deployed pipeline. For more information, see Run Anywhere: In-Line Model Updates on Edge Devices.
• POST /pipelines/{pipeline-name}: The pipeline-name is the same as returned from the /pipelines endpoint as id. This endpoint changes based on the pipeline publish deployed.

Organizations are encouraged to use the /infer endpoint for consistency.

Wallaroo inference endpoint URLs accept the following data inputs through the Content-Type header:

• Content-Type: application/vnd.apache.arrow.file: For Apache Arrow tables.
• Content-Type: application/json; format=pandas-records: For pandas DataFrame in record format.

Once deployed, we can perform an inference through the deployment URL.

The endpoint returns Content-Type: application/json; format=pandas-records by default with the following fields:

• check_failures (List[Integer]): Whether any validation checks were triggered. For more information, see Wallaroo SDK Essentials Guide: Pipeline Management: Anomaly Testing.
• elapsed (List[Integer]): A list of time in nanoseconds for:
  • [0] The time to serialize the input.
  • [1…n] How long each step took.
• model_name (String): The name of the model used.
• model_version (String): The version of the model in UUID format.
• original_data: The original input data. Returns null if the input may be too long for a proper return.
• outputs (List): The outputs of the inference result separated by data type, where each data type includes:
  • data: The returned values.
  • dim (List[Integer]): The dimension shape returned.
  • v (Integer): The vector shape of the data.
• pipeline_name (String): The name of the pipeline.
• shadow_data: Any shadow deployed data inferences in the same format as outputs.
• time (Integer): The time since UNIX epoch.

Edge Inference Endpoint Example

The following example demonstrates sending an Apache Arrow table to the Edge deployed pipeline, requesting the inference results back in a pandas DataFrame records format.

curl -X POST localhost:8080/infer -H "Content-Type: application/vnd.apache.arrow.file" -H 'Accept: application/json; format=pandas-records'  --data-binary @./data/image_224x224.arrow

Returns:

[{"check_failures":[],"elapsed":[1067541,21209776],"model_name":"resnet-50","model_version":"2e05e1d0-fcb3-4213-bba8-4bac13f53e8d","original_data":null,"outputs":[{"Int64":{"data":[535],"dim":[1],"v":1}},{"Float":{"data":[0.00009498586587142199,0.00009141524787992239,0.0004606838047038764,0.00007667174941161647,0.00008047101437114179,...],"dim":[1,1001],"v":1}}],"pipeline_name":"edge-cv-demo","shadow_data":{},"time":1694205578428}]

Log Retrieval from Edge Locations

Inference logs are retrieved from edge location deployments through the /logs endpoint.

• Endpoint: /logs
• Type: POST
• Headers:
  • Content-Type: application/json: Submissions to the /logs endpoint in JSON text format.
  • Accept: application/json; format=pandas-records: The /logs endpoint returns JSON in pandas Record format.
• Parameters:
  • {}: An empty set.

Inference logs are returned as JSON in pandas Record Format with the following fields:

Log Retrieval from Edge Locations Example

The following shows retrieving logs from a model deployment on an edge location.

We will store the logs to a JSON file in pandas Record format, then display the edge logs as a DataFrame.

!curl -X POST http://localhost:8080/logs \
    -H "Content-Type: Content-Type: application/json; format=pandas-records" \
    --data {} > ./edge-logs.df.json

df_logs = pd.read_json("./edge-logs.df.json", orient="records")
df_logs

1000 rows × 5 columns

Edge Bundle One Time Token

When an edge is added to a pipeline publish, the field docker_run_variables contains a JSON value for edge devices to connect to the Wallaroo Ops instance.

The settings are stored in the key EDGE_BUNDLE as a base64 encoded value that include the following:

• BUNDLE_VERSION: The current version of the bundled Wallaroo pipeline.
• EDGE_NAME: The edge name as defined when created and added to the pipeline publish.
• JOIN_TOKEN_: The one time authentication token for authenticating to the Wallaroo Ops instance.
• OPSCENTER_HOST: The hostname of the Wallaroo Ops edge service. See Edge Deployment Registry Guide for full details on enabling pipeline publishing and edge observability to Wallaroo.
• PIPELINE_URL: The OCI registry URL to the containerized pipeline.
• WORKSPACE_ID: The numerical ID of the workspace.

For example:

{'edgeBundle': 'ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT14Z2ItY2NmcmF1ZC1lZGdlLXRlc3QKZXhwb3J0IEpPSU5fVE9LRU49MzE0OGFkYTUtMjg1YS00ZmNhLWIzYjgtYjUwYTQ4ZDc1MTFiCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6ZjM4OGMxMDktOGQ1Ny00ZWQyLTk4MDYtYWExM2Y4NTQ1NzZiCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9NQ=='}

base64 -D
ZXhwb3J0IEJVTkRMRV9WRVJTSU9OPTEKZXhwb3J0IEVER0VfTkFNRT14Z2ItY2NmcmF1ZC1lZGdlLXRlc3QKZXhwb3J0IEpPSU5fVE9LRU49MzE0OGFkYTUtMjg1YS00ZmNhLWIzYjgtYjUwYTQ4ZDc1MTFiCmV4cG9ydCBPUFNDRU5URVJfSE9TVD1kb2MtdGVzdC5lZGdlLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphCmV4cG9ydCBQSVBFTElORV9VUkw9Z2hjci5pby93YWxsYXJvb2xhYnMvZG9jLXNhbXBsZXMvcGlwZWxpbmVzL2VkZ2UtcGlwZWxpbmU6ZjM4OGMxMDktOGQ1Ny00ZWQyLTk4MDYtYWExM2Y4NTQ1NzZiCmV4cG9ydCBXT1JLU1BBQ0VfSUQ9NQ==^D
export BUNDLE_VERSION=1
export EDGE_NAME=xgb-ccfraud-edge-test
export JOIN_TOKEN=3148ada5-285a-4fca-b3b8-b50a48d7511b
export OPSCENTER_HOST=doc-test.wallaroocommunity.ninja/edge
export PIPELINE_URL=ghcr.io/wallaroolabs/doc-samples/pipelines/edge-pipeline:f388c109-8d57-4ed2-9806-aa13f854576b
export WORKSPACE_ID=5

The JOIN_TOKEN is a one time access token. Once used, a JOIN_TOKEN expires. The authentication session data is stored in persistent volumes. Persistent volumes must be specified for docker and docker compose based deployments of Wallaroo pipelines; helm based deployments automatically provide persistent volumes to store authentication credentials.

The JOIN_TOKEN has the following time to live (TTL) parameters.

• Once created, the JOIN_TOKEN is valid for 24 hours. After it expires the edge will not be allowed to contact the OpsCenter the first time and a new edge bundle will have to be created.
• After an Edge joins to Wallaroo Ops for the first time with persistent storage, the edge must contact the Wallaroo Ops instance at least once every 7 days.
  • If this period is exceeded, the authentication credentials will expire and a new edge bundle must be created with a new and valid JOIN_TOKEN.

Wallaroo edges require unique names. To create a new edge bundle with the same name:

• Use the Remove Edge to remove the edge by name.
• Use Add Edge to add the edge with the same name. A new EDGE_BUNDLE is generated with a new JOIN_TOKEN.