Provision a Jupyter Notebook Workspace for Data Scientist or Developer Use

Contributors mboglesby kevin-hoke Download PDF of this page

Kubeflow is capable of rapidly provisioning new Jupyter Notebook servers to act as data scientist workspaces. To provision a new Jupyter Notebook server with Kubeflow, perform the following tasks. For more information about Jupyter Notebooks within the Kubeflow context, see the official Kubeflow documentation.

  1. From the Kubeflow central dashboard, click Notebook Servers in the main menu to navigate to the Jupyter Notebook server administration page.

    Error: Missing Graphic Image

  2. Click New Server to provision a new Jupyter Notebook server.

    Error: Missing Graphic Image

  3. Give your new server a name, choose the Docker image that you want your server to be based on, and specify the amount of CPU and RAM to be reserved by your server. If the Namespace field is blank, use the Select Namespace menu in the page header to choose a namespace. The Namespace field is then auto-populated with the chosen namespace.

    In the following example, the kubeflow-anonymous namespace is chosen. In addition, the default values for Docker image, CPU, and RAM are accepted.

    Error: Missing Graphic Image

  4. Specify the workspace volume details. If you choose to create a new volume, then that volume or PVC is provisioned using the default StorageClass. Because a StorageClass utilizing Trident was designated as the default StorageClass in the section Kubeflow Deployment, the volume or PVC is provisioned with Trident. This volume is automatically mounted as the default workspace within the Jupyter Notebook Server container. Any notebooks that a user creates on the server that are not saved to a separate data volume are automatically saved to this workspace volume. Therefore, the notebooks are persistent across reboots.

    Error: Missing Graphic Image

  5. Add data volumes. The following example specifies an existing PVC named 'pb-fg-all' and accepts the default mount point.

    Error: Missing Graphic Image

  6. Optional: Request that the desired number of GPUs be allocated to your notebook server. In the following example, one GPU is requested.

    Error: Missing Graphic Image

  7. Click Launch to provision your new notebook server.

  8. Wait for your notebook server to be fully provisioned. This can take several minutes if you have never provisioned a server using the Docker image that you specified because the image needs to be downloaded. When your server has been fully provisioned, you see a green check mark in the Status column on the Jupyter Notebook server administration page.

    Error: Missing Graphic Image

  9. Click Connect to connect to your new server web interface.

  10. Confirm that the dataset volume that was specified in step 6 is mounted on the server. Note that this volume is mounted within the default workspace by default. From the perspective of the user, this is just another folder within the workspace. The user, who is likely a data scientist and not an infrastructure expert, does not need to possess any storage expertise in order to use this volume.

    Error: Missing Graphic Image

    Error: Missing Graphic Image

  11. Open a Terminal and, assuming that a new volume was requested in step 5, execute df -h to confirm that a new Trident-provisioned persistent volume is mounted as the default workspace.

    The default workspace directory is the base directory that you are presented with when you first access the server’s web interface. Therefore, any artifacts that you create by using the web interface are stored on this Trident-provisioned persistent volume.

    Error: Missing Graphic Image

    Error: Missing Graphic Image

  12. Using the terminal, run nvidia-smi to confirm that the correct number of GPUs were allocated to the notebook server. In the following example, one GPU has been allocated to the notebook server as requested in step 7.

Error: Missing Graphic Image