Wallaroo MLOps API Essentials Guide: Connections Management

How to use the Wallaroo API for Connections Management

Table of Contents

Wallaroo Data Connections are provided to establish connections to external data stores for requesting or submitting information. They provide a source of truth for data source connection information to enable repeatability and access control within your ecosystem. The actual implementation of data connections are managed through other means, such as Wallaroo Pipeline Orchestrators and Tasks, where the libraries and other tools used for the data connection can be stored.

Create Connection

Lists the enablement features for the Wallaroo instance.

  • REQUEST PATH
    • POST /v1/api/connections/create
  • PARAMETERS
    • name (String Required): The name of the connection.
    • type (String Required): The user defined type of connection.
    • details (String Required): User defined configuration details for the data connection. These can be {'username':'dataperson', 'password':'datapassword', 'port': 3339}, or {'token':'abcde123==', 'host':'example.com', 'port:1234'}, or other user defined combinations.
  • RETURNS
    • id (String): The id of the connection id in UUID format.
# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/create"

# input connection
data = {
    'name': bigquery_connection_input_name,
    'type' : bigquery_connection_input_type,
    'details': bigquery_connection_input_argument
}

response=requests.post(url, headers=headers, json=data).json()
display(response)

{'id': '02e587e9-e564-4af4-b3c4-0800994febfd'}

Add Connection to Workspace

Adds a connection to a Wallaroo workspace, accessible by workspace members.

  • REQUEST PATH
    • POST /v1/api/connections/create
  • PARAMETERS
    • workspace_id (String Required): The name of the connection.
    • connection_id (String Required): The UUID connection ID
  • RETURNS
    • id (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.
# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/add_to_workspace"

data = {
    'workspace_id': workspace_id,
    'connection_id': connection_input_id
}

response=requests.post(url, headers=headers, json=data)
display(response.json())

{'id': '0c470074-d864-4467-bcd4-f6e8de7794e1'}

List Connections in Workspace

Lists the connections set to the specified workspace.

Adds a connection to a Wallaroo workspace, accessible by workspace members.

  • REQUEST PATH
    • POST /v1/api/connections/create
  • PARAMETERS
    • workspace_id (Integer Required): The numerical id of the workspace.
  • RETURNS
    • List[workspaces]: List of the connections in the workspace.
      • name (String): The id of the workspace’s connection id in UUID format. Note that the connection’s workspace connection id is separate from the Wallaroo connection id.
      • type (String): The user defined type of connection.
      • details (dict): The user defined details of the connection.
      • created_at (String): Timestamp of the connection creation date.
      • workspace_names (List[String]): List of workspaces the connection is attached to.
# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/list"

# output connection
data = {
    'workspace_id': workspace_id
}

response=requests.post(url, headers=headers, json=data).json()
display(response)

{'connections': [{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}
    ]
}

Get Connection by Name

Retrieves the connection by the unique connection name.

  • REQUEST PATH
    • POST /v1/api/connections/get
  • PARAMETERS
    • name (String Required): The name of the connection.
  • RETURNS
    • name (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.
    • type (String): The user defined type of connection.
    • details (dict): The user defined details of the connection.
    • created_at (String): Timestamp of the connection creation date.
    • workspace_names (List[String]): List of workspaces the connection is attached to.
# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/get"

data = {
    'name': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data).json())

{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}

Get Connection by Id

Retrieves the connection by the unique connection id.

  • REQUEST PATH
    • POST /v1/api/connections/get_by_id
  • PARAMETERS
    • id (String Required): The id of the connection.
  • RETURNS
    • name (String): The id of the workspace’s connection id in UUID format. Note that the workspace’s connection id is separate from the Wallaroo connection id.
    • type (String): The user defined type of connection.
    • details (dict): The user defined details of the connection.
    • created_at (String): Timestamp of the connection creation date.
    • workspace_names (List[String]): List of workspaces the connection is attached to.
# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/get_by_id"

data = {
    'id': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data).json())

{'id': '2274dd22-2a18-454a-8cde-383cc013928b',
   'name': 'bigqueryhouseapiinput',
   'type': 'BIGQUERY',
   'details': {'type': 'service_account',
    'table': 'housepricesaga_inputs',
    'dataset': 'release_testing_2023_2',
   'created_at': '2023-05-16T16:43:05.170914Z',
   'workspace_names': ['bigqueryapiworkspace']}

Remove Connection from Workspace

Removes the specified connection from the workspace.

  • REQUEST PATH
    • POST /v1/api/connections/remove_from_workspace
  • PARAMETERS
    • workspace_id (Integer Required): The numerical id of the workspace.
    • connection_id (String Required): The connection id.
  • RETURNS
    • HTTP 204 response.
# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/remove_from_workspace"

data = {
    'workspace_id': workspace_id,
    'connection_id': connection_input_id
}

display(requests.post(url, headers=headers, json=data))

<Response [204]>

Delete Connection

Deletes the connection from the Wallaroo instance. The connection must be removed from all workspaces before it can be deleted.

  • REQUEST PATH
    • POST /v1/api/connections/delete
  • PARAMETERS
    • name (String Required): The connection name.
  • RETURNS
    • HTTP 204 response.
# get the connection input details

# retrieve the authorization token
headers = wl.auth.auth_header()

url = f"{APIURL}/v1/api/connections/delete"

data = {
    'name': bigquery_connection_input_name
}

display(requests.post(url, headers=headers, json=data))

<Response [204]>