Deploy locally using K3d

Deploy locally using K3d

This is supported only on a local machine/cloud VM running Linux or MacOS. In this setup, we will deploy Syft server locally, using K3d. Even if you need to run the deployment in a remote or production environment, it is highly recommended that you first test-out the deployment locally.

Prerequisites

  1. Install K3d: K3d installation guide as:

  2. Install/upgrade helm to latest version:

    sudo apt-get update
sudo apt-get install helm

    • Test as:

    helm

  3. Install kubectl:

    kubectl version --client

Deploy Syft Server

Now that we have installed all the required packages, we will move forward with the deployment of a PySyft server.

There are two ways to deploy PySyft server:

  1. Using a one liner convenience script

  2. Running the commands manually

Using convenience script

curl -o {{file_name}}.sh https://raw.githubusercontent.com/OpenMined/PySyft/syft-setup-script/scripts/setup_syft.sh  && bash {{file_name}}.sh  --cluster-name {{cluster_name}} --namespace {{namespce_name}} --version {{version}}

Example:

curl -o setup.sh https://raw.githubusercontent.com/OpenMined/PySyft/syft-setup-script/scripts/k3d-setup.sh  && bash syft.sh  --cluster-name syft-test --namespace syft --version 0.8.3

–versionis the version of PySyft server you want to deploy.

For k3d deployment version is the only required argument, if you choose not to pass cluster_name and namespace, the defaults of syft-test and syft will be used respectively

Using Helm Charts

A Helm chart is a collection of files that describe a set of Kubernetes resources. Helm uses these charts to deploy applications or services on Kubernetes. Each chart contains the following essential components:

  • Chart.yaml: This file contains metadata about the chart, such as the name, version, and description.

  • Values.yaml: This file includes default values for the chart’s variables, which can be overridden by users during deployment.

Step 1: Create a cluster with K3d

To setup a k3d deployment, you need a running Kubernetes cluster. We will cover here the setup of a local registry using k3d as such:

```bash

k3d cluster create {{cluster_name}} -p 8080:80@loadbalancer
```

Example is provided below:
```bash
k3d cluster create syft-test -p 8080:80@loadbalancer
```

Step 2. Use Helm to install syft

A Helm repository is a collection of packaged charts that can be shared and accessed publicly or privately. These repositories are similar to package repositories in other ecosystems, like npm for Node.js or PyPI for Python. Helm repositories make it easy to distribute charts and manage their versions.

To use a Helm repository, you need to add it to your local Helm client. Here we are adding PySyft’s helm repository:

```bash
helm repo add openmined https://openmined.github.io/PySyft/helm

helm repo update openmined
```

Step 3. Search for available Syft versions

You can search for available charts in the added repositories using the helm search command. For instance, to search for various versions of the Syft charts, you can use:

```bash
helm search repo openmined/syft --versions --devel
```

Adding --devel to the search will include the beta versions as well.

Step 4. Set your preferred Syft version an deploy

To install a chart from a repository, use the helm install command along with the repository name and the chart name.

```bash
SYFT_VERSION="<paste the chart version number>"

helm install my-server openmined/syft --version $SYFT_VERSION --namespace syft --create-namespace
```

To pass a local values.yaml file, you can add the following to the above:

 -f $PATH_TO_VALUES \
 # Or directly override values from the yaml file as:
 --value_name $VALUE

Test your deployment

  1. Check ingress run

K3d comes packaged with Traefik as a load balancer, thus after the helm command, a Traefik ingress controller will be provisioned.

To confirm the health of the ingress run:

kubectl describe ingress -n syft

  1. Check running pods

kubectl get pods -n syft

Additionally, visiting localhost:8080 should show the Syft Server login page.

  1. Launch your Jupyter Notebook Instance

In this step, you need a client server in order to interact with the PySyft server you just deployed. Here, the client consists of a Jupyter notebook running the same version of PySyft as the server served as a container.

We need to start the client as follows:

docker run --rm -it --network=host openmined/syft-client:${VERSION}

where ${VERSION} is the version of PySyft Client you’d like to run. It must be the same as you passed before at deployment step.

Keeping the notebook running

Often, the jupyter notebook instance dies once you exit the terminal. To make it long running even after you close the terminal for easy access, consider running the command in a screen or tmux session. See this link for more information on how to use screen: Screen Command or tmux: Tmux Command.

If this step is successful, you will be taken to a Jupyter notebook interface in the browser. Please create a new notebook and run the following command to test your full setup is ready to be used:

  1. Connect to your launched server

import syft as sy

sy.login(url="localhost:80", email="[email protected]", password="changethis")

 # Please pass the URL, the email and password used to create the server. The above are the default ones used during server creation.

If everything works well, you should be able to login to your server. You can now start running PySyft commands on the client server to interact with the Syft server you just deployed.

On This Page