Wallaroo MLOps API Essentials Guide: Connections Management

How to use the Wallaroo API for Connections Management

Wallaroo Data Connections are provided to establish connections to external data stores for requesting or submitting information. They provide a source of truth for data source connection information to enable repeatability and access control within your ecosystem. The actual implementation of data connections are managed through other means, such as Wallaroo Pipeline Orchestrators and Tasks, where the libraries and other tools used for the data connection can be stored.

Create Connection

Lists the enablement features for the Wallaroo instance.

REQUEST PATH
- POST /v1/api/connections/create
PARAMETERS
- name (String Required): The name of the connection.
- type (String Required): The user defined type of connection.
- details (String Required): User defined configuration details for the data connection. These can be {'username':'dataperson', 'password':'datapassword', 'port': 3339}, or {'token':'abcde123==', 'host':'example.com', 'port:1234'}, or other user defined combinations.
RETURNS
- id (String): The id of the connection id in UUID format.

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/create"

# input connection
data = {
    'name': bigquery_connection_input_name,
    'type' : bigquery_connection_input_type,
    'details': bigquery_connection_input_argument
}

response=requests.post(url, headers=headers, json=data).json()
display(response)

{'id': '02e587e9-e564-4af4-b3c4-0800994febfd'}

Add Connection to Workspace

Adds a connection to a Wallaroo workspace, accessible by workspace members.

REQUEST PATH
- POST /v1/api/connections/create
PARAMETERS
- workspace_id (String Required): The name of the connection.
- connection_id (String Required): The UUID connection ID
RETURNS
- id (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/add_to_workspace"

data = {
    'workspace_id': workspace_id,
    'connection_id': connection_input_id
}

response=requests.post(url, headers=headers, json=data)
display(response.json())

{'id': '0c470074-d864-4467-bcd4-f6e8de7794e1'}

List Connections in Workspace

Lists the connections set to the specified workspace.

Adds a connection to a Wallaroo workspace, accessible by workspace members.

REQUEST PATH
- POST /v1/api/connections/create
PARAMETERS
- workspace_id (Integer Required): The numerical id of the workspace.
RETURNS
- List[workspaces]: List of the connections in the workspace.
  - name (String): The id of the workspace’s connection id in UUID format. Note that the connection’s workspace connection id is separate from the Wallaroo connection id.
  - type (String): The user defined type of connection.
  - details (dict): The user defined details of the connection.
  - created_at (String): Timestamp of the connection creation date.
  - workspace_names (List[String]): List of workspaces the connection is attached to.

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/list"

# output connection
data = {
    'workspace_id': workspace_id
}

response=requests.post(url, headers=headers, json=data).json()
display(response)

{'connections': [{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}
    ]
}

Get Connection by Name

Retrieves the connection by the unique connection name.

REQUEST PATH
- POST /v1/api/connections/get
PARAMETERS
- name (String Required): The name of the connection.
RETURNS
- name (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.
- type (String): The user defined type of connection.
- details (dict): The user defined details of the connection.
- created_at (String): Timestamp of the connection creation date.
- workspace_names (List[String]): List of workspaces the connection is attached to.

# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/get"

data = {
    'name': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data).json())

{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}

Get Connection by Id

Retrieves the connection by the unique connection id.

REQUEST PATH
- POST /v1/api/connections/get_by_id
PARAMETERS
- id (String Required): The id of the connection.
RETURNS
- name (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.
- type (String): The user defined type of connection.
- details (dict): The user defined details of the connection.
- created_at (String): Timestamp of the connection creation date.
- workspace_names (List[String]): List of workspaces the connection is attached to.

# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/get_by_id"

data = {
    'id': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data).json())

{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}

Remove Connection from Workspace

Removes the specified connection from the workspace.

REQUEST PATH
- POST /v1/api/connections/remove_from_workspace
PARAMETERS
- workspace_id (Integer Required): The numerical id of the workspace.
- connection_id (String Required): The connection id.
RETURNS
- HTTP 204 response.

# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/remove_from_workspace"

data = {
    'workspace_id': workspace_id,
    'connection_id': connection_input_id
}

display(requests.post(url, headers=headers, json=data))

<Response [204]>

Delete Connection

Deletes the connection from the Wallaroo instance. The connection must be removed from all workspaces before it can be deleted.

REQUEST PATH
- POST /v1/api/connections/delete
PARAMETERS
- name (String Required): The connection name.
RETURNS
- HTTP 204 response.

# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/delete"

data = {
    'name': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data))

<Response [204]>

Previous: Wallaroo MLOps API Essentials Guide: Assays Management

Next: Wallaroo MLOps API Essentials Guide: ML Workload Orchestration Management