Requests API (.requests)

Requests API (.requests)

Estimated reading time: 5’

What you’ll learn

This guide’s objective is to explain the usage of requests in the collaboration between the data owners and data scientists.

Introduction

A data owner’s role is to decide how access to their server and assets (such as datasets or models) takes place. However, instead of pre-emptively restricting everything a data scientist can do to a set of allowed actions, PySyft enables more flexibility by allowing the data scientist to request very specific changes in what they are permissioned to do. These are called requests.

How are requests structured?

Request Components

A request has two main functional components:

  • a sy.Change, which is the subject of the request (request.changes). This is explained in more detail below.

  • a sy.RequestStatus, which is whether the change was or not applied. It can be Approved, Pending or Denied. (request.status)

Request Workflow

In general, the workflow of a requests within PySyft consists of:

  • (1) submission: a request is created on the appropriate API (e.g. client.code, client.images), stating the desired output.

  • (2) review & approval: the request must be appropriately reviewed by the data owner to understand the implications of approving the request from a security and privacy standpoint. A data owner could decide to approve or deny. If approved, the changes proposed as part of the request are applied to the indended object.

  • (3) changes are aplied: if approved, the changes applied are visible back to the requesting data scientist.

An example is given below for the code requests. All requests follow the same pattern of execution.

Types of requests & changes

Requests can be initiated by a user of the server and depending on the type of the request, they contain a pre-defined “change”. A sy.Change links an existing Syft Object (change.linked_obj) and encapsulates a specific type of alteration of that object. Whilst the above is sightly abstract, a few types of requests are currently available in PySyft, with the help of changes:

  • Code Requests: these requests used the most common type of change, called UserCodeStatusChange, which allows data scientists to request the permission to execute a specific Syft Function (available via request.code). Concretely, when a UserCode is paired with specific Syft Policies, the user can request the permission to execute that code within the bounds permitted by the Input and Output policies attached. Such policies must state, at minimum, the asset on which the computation intended for and how many times it can be executed.

Execution Permissions on a Low-side server

Such permissions for execution are useful only when a data scientist has direct access to the server in which the private data is stored. In the case of a low-side server, the data scientist will be interfacing with a SyncedUserCodeStatusChange that only allows to retrieve execution output, rather than executing the code themselves, due to the airgap implied by the system and the lack of the linked obj on the server.

  • Custom Worker Pool Requests: these requests rely on CreateCustomWorkerPoolChange change, which allows data scientists to request the creation of a new resource on the server, namely a worker pool, from an existing custom image on the server, so that data scientists can execute their code in a container running the desired Python packages and possibly allow them to scale even further depending on the worker pool’s size proposed. Note that here, requests are also able to launch resources. More about this coming soon in the images API guide.

  • Custom Image & Worker Pool Requests: these requests imply two changes: CreateCustomImageChange and CreateCustomWorkerPoolChange. This allows data scientists not only to create a custom worker pool from an image, like the request above, but also to specify a new image definition alongside with their desired software, in case the existing images do not fullfill their needs.

  • Association Requests: these requests imply a AssociationRequestChange change, which allows a data owner to submit a request on an external Datasite server to create an unidirectional route to communicate with that server. This requires approval from the external data owner that owns that server. More about this coming soon in the networking guide.

  • Action Object Permission Request: these requests imply a ActionStoreChange change, which allows data scientists to request READ permissions on a specific ActionObject available on the server, which is the wrapper object for data that is being stored in PySyft or generated out of computations. More about this coming soon in the action API guide.

Let’s see now how one can create a code request. We will shortly document how to create the other types of requests, as the rest of the requests are currently under beta.

Create a code request

We will first run some setup code to enable the code request creation.

Hide code cell source 
import numpy as np
import syft as sy

server = sy.orchestra.launch(name="my_server", port="auto", dev_mode=False, reset=True)
do_client = server.login(email="[email protected]", password="changethis")
do_client.register(
    email="[email protected]", name="John Doe", password="pw", password_verify="pw"
)
ds_client = server.login(email="[email protected]", password="pw")

dataset = sy.Dataset(
    name="Dataset name",
    description="**Placehoder Dataset description**",
    asset_list=[sy.Asset(
        name="asset_name",
        data=[1,2,3], # real data
        mock=[4,5,6], # mock data
    )],
)

do_client.upload_dataset(dataset)
Hide code cell output 
Starting my_server server on 0.0.0.0:36497

Waiting for server to start Done.
SyftInfo: 
You have launched a development server at http://0.0.0.0:36497.It is intended only for local use.

Logged into <my_server: High side Datasite> as <[email protected]>
SyftWarning: 
You are using a default password. Please change the password using `[your_client].account.set_password([new_password])`.

Logged into <my_server: High side Datasite> as <[email protected]>

Uploading:   0%|          | 0/1 [00:00<?, ?it/s]

Uploading: asset_name:   0%|          | 0/1 [00:00<?, ?it/s]

Uploading: asset_name: 100%|██████████| 1/1 [00:00<00:00, 20.45it/s]
SyftSuccess: 
Dataset uploaded to 'my_server'. To see the datasets uploaded by a client on this server, use command `[your_client].datasets`

By inspecting the code API guide, we can construct a dummy Syft Function to include in our request:

@sy.syft_function_single_use(x=ds_client.datasets[0].assets[0])
def my_dummy_func(x):
    return x+1
SyftSuccess: 
Syft function 'my_dummy_func' successfully created. To add a code request, please create a project using `project = syft.Project(...)`, then use command `project.create_code_request`.

Ways to create a code request

There are currently two ways to create such code execution requests in PySyft:

  1. Directly via the Code API

  2. Within a Project’s Scope

Directly via the Code API

ds_client.code.request_code_execution(my_dummy_func)

Request

Id: 45615a8c80ba4e1c8c5dbf93e681e887

Request time: 2024-08-02 13:02:43

Status: RequestStatus.PENDING

Requested on: My_server of type Datasite

Requested by: John Doe ([email protected])

Changes: Request to change my_dummy_func (Pool Id: default-pool) to permission RequestStatus.APPROVED. No nested requests.

Within a project’s scope

# Create a project
project = sy.Project(
    name="Dummy project",
    description="I am doing a dummy project for illustrating code requests.",
    members=[ds_client],
).send()

project

Dummy project

I am doing a dummy project for illustrating code requests.

Created by: John Doe ([email protected])

[]

To see a list of projects, use command `<your_client>.projects`

project.create_code_request(my_dummy_func)
SyftError: 
Request 45615a8c80ba4e1c8c5dbf93e681e887 already exists for this UserCode. Please use the existing request, or submit a new UserCode to create a new request.

View requests

All requests, regarding of their type and purpose, can be viewed under the Requests API exposed by PySyft.

As with other APIs, an admin or data owner can see all requests submitted to their server, whilst a data scientist can only see the ones that they submitted:

ds_client.requests

Request List

Total: 0

We can index into each individual request as such:

request = ds_client.requests[0]

request

Request

Id: 45615a8c80ba4e1c8c5dbf93e681e887

Request time: 2024-08-02 13:02:43

Status: RequestStatus.PENDING

Requested on: My_server of type Datasite

Requested by: John Doe ([email protected])

Changes: Request to change my_dummy_func (Pool Id: default-pool) to permission RequestStatus.APPROVED. No nested requests.

If curious, you can also see the exact change type and the linked object to the change:

request.changes[0]

class UserCodeStatusChange:
  id: 15b9fc7b129247d087fd8ff4bba4fcfb
  function: my_dummy_func
  input_policy_type: ExactMatch
  output_policy_type: OutputPolicyExecuteOnce
  approved: False

request.changes[0].linked_obj

class LinkedObject:
  id: str = b58910b0a08c4b1fbca9fde2bd7c779e

Request Metadata

Further metadata is available on a sy.Request object, such as:

  • who created the request:

    • requesting_user_name: the user that created the request

    • requesting_user_email: the email of the requesting user

    • requesting_user_institution: the institution of affiliation of the requesting user

    • requesting_user_verify_key: the system generated verify key of the requesting user

  • when it was modified:

    • request_time: the time when the request was created

    • updated_at: the time when the request was updated last (e.g. its status)

    • history: contains a log of the request’s status changes

  • where it was created:

    • server_uid: the UID of the server on which the request was created

  • who approved the request:

    • request.approving_user_verifying_key: the system generated verify key of the approving user

ds_client.requests[0].changes[0].linked_obj

class LinkedObject:
  id: str = b58910b0a08c4b1fbca9fde2bd7c779e

Responding to a request

Only an admin or data owner have the permission of approving or denying a request.

It is their responsaibility to make sure that approval is not imposing privacy or security concerns and that it is aligned with their internal rules for releasing data. In case they are not, the data owner are able to deny the requests.

Review step

do_client.requests[0]

Request

Id: 45615a8c80ba4e1c8c5dbf93e681e887

Request time: 2024-08-02 13:02:43

Status: RequestStatus.PENDING

Requested on: My_server of type Datasite

Requested by: John Doe ([email protected])

Changes: Request to change my_dummy_func (Pool Id: default-pool) to permission RequestStatus.APPROVED. No nested requests. 

request = do_client.requests[0]
request.code

UserCode

id: UID = 4ff128c78ce84e8f86f30f816bd4b2c1

service_func_name: str = my_dummy_func

shareholders: list = ['my_server']

status: list = ['Server: my_server, Status: pending']

inputs: dict = 

{
  "assets": {
    "x": {
      "action_id": "13cfe6b9e4304667adaf91b1576c36d4",
      "source_asset": "asset_name",
      "source_dataset": "Dataset name",
      "source_server": "12fdcea1fdd048008ba9c2ec5755b945"
    }
  }
}

code:

@sy.syft_function_single_use(x=ds_client.datasets[0].assets[0])
def my_dummy_func(x):
    return x+1

Approval step

This is possible via the .approve and .deny APIs. However, please note that code requests might behave differently depending on the deployment setup. If the setup consists only of one server containing both the mock and private data (a high-side node), the data owner can directly proceed with:

request.approve()

Approving request on change my_dummy_func for datasite my_server
SyftSuccess: 
Request 45615a8c80ba4e1c8c5dbf93e681e887 changes applied

Or deny:

request.deny(reason="This imposes privacy risks")
SyftSuccess: 
Request 45615a8c80ba4e1c8c5dbf93e681e887 successfully denied !

However, when the request is created on an air-gapped system, the approval is implicit part of the syncing flow, whilst the denial is taking place similarily, where the request was created (on the low-side). You can read more about the whole flow in the syncing API.

On This Page