Deployed pipelines have their own Inference URL that accepts HTTP POST submissions.
For connections that are external to the Kubernetes cluster hosting the Wallaroo instance, model endpoints must be enabled.
HTTP Headers
The following headers are required for connecting the the Pipeline Deployment URL:
-
Authorization: This requires the JWT token in the format
'Bearer ' + token
. For example:Authorization: Bearer abcdefg==
-
Content-Type:
-
For DataFrame formatted JSON:
Content-Type:application/json; format=pandas-records
-
For Arrow binary files, the
Content-Type
isapplication/vnd.apache.arrow.file
.Content-Type:application/vnd.apache.arrow.file
-
IMPORTANT NOTE: Verify that the pipeline deployed has status
Running
before attempting an inference.
# Retrieve the token
headers = wl.auth.auth_header()
# set Content-Type type
headers['Content-Type']='application/json; format=pandas-records'
## Inference through external URL using dataframe
# retrieve the json data to submit
data = [
{
"tensor":[
1.0678324729,
0.2177810266,
-1.7115145262,
0.682285721,
1.0138553067,
-0.4335000013,
0.7395859437,
-0.2882839595,
-0.447262688,
0.5146124988,
0.3791316964,
0.5190619748,
-0.4904593222,
1.1656456469,
-0.9776307444,
-0.6322198963,
-0.6891477694,
0.1783317857,
0.1397992467,
-0.3554220649,
0.4394217877,
1.4588397512,
-0.3886829615,
0.4353492889,
1.7420053483,
-0.4434654615,
-0.1515747891,
-0.2668451725,
-1.4549617756
]
}
]
# submit the request via POST, import as pandas DataFrame
response = pd.DataFrame.from_records(
requests.post(
deployurl,
json=data,
headers=headers)
.json()
)
display(response)
time | in | out | check_failures | metadata | |
---|---|---|---|---|---|
0 | 1684356836285 | {'tensor': [1.0678324729, 0.2177810266, -1.7115145262, 0.682285721, 1.0138553067, -0.4335000013, 0.7395859437, -0.2882839595, -0.447262688, 0.5146124988, 0.3791316964, 0.5190619748, -0.4904593222, 1.1656456469, -0.9776307444, -0.6322198963, -0.6891477694, 0.1783317857, 0.1397992467, -0.3554220649, 0.4394217877, 1.4588397512, -0.3886829615, 0.4353492889, 1.7420053483, -0.4434654615, -0.1515747891, -0.2668451725, -1.4549617756]} | {'dense_1': [0.0014974177]} | [] | {'last_model': '{"model_name":"apimodel","model_sha":"bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507"}', 'pipeline_version': '', 'elapsed': [163502, 309804]} |