Welcome to Scinco’s Documentation!
What is SCINCO?
The Scinco project at TACC provides a hosted interactive computing platform that augments and complements batch computing capabilities available today within science gateways. SCINCO combines container technology and state-of-the-art open source technologies such as Kubernetes and JupyterHub to deliver:
A hardened, production-grade platform that can support hundreds of projects running across thousands of nodes geographically distributed across dozens of academic clusters.
Deep integration with advanced storage and computing resources available in existing science gateways.
Sharing and publishing features allowing analyses to be packaged with data and shared privately with collaborators or disseminated to the community at large. Diverse areas of research, including Astronomy, Biology, Climate Science, Neuroscience, and various fields of Engineering will leverage the SCINCO platform to analyze big data, implement computational models, disseminate results and train the next generation of researchers.
Capabilities
Provides users with a customized Jupyter notebook server with access to TACC’s storage and compute resources. Projects currently using JupyterHub at TACC are Designsafe-CI , Hobby Eberly Dark Energy Experiment (HETDEX) and DARPA SHADE.
Components
At its core, the Scinco architecture utilizes a customized JupyterHub version, an open-source, cloud-based Jupyter project that allows users to access notebook servers running on remote machines. Scinco incorporates multitenancy or logically separated instances of the platform so that individual projects can configure various aspects of the notebook servers and data made available to their users based on their individual project needs. Basic components of Scinco are:
OAuth-based Authenticator Plugin
JupyterHub
Customized notebook server images
Administrative Portal
Scinco Publications
Jamthe, A.; Stubbs, J.; Packard, M.; Chuah, J.; Looney, J.; Curbelo & Gilbert, C. (2021). Enriching scientific and interactive computing with project SCINCO: JupyterHub on Kubernetes. In Gateways 2021
Stubbs, J., Looney, J., Poindexter, M., Chalhoub, E., Zynda, G., Ferlanti, E., Vaughn, M., Fonner, J., & Dahan, M. (2020). Integrating Jupyter into Research Computing Ecosystems: Challenges and Successes in Architecting JupyterHub for Collaborative Research Computing Ecosystems. In Practice and Experience in Advanced Research Computing (pp. 91–98). Association for Computing Machinery.
Quick Start Guide
To accesss JupyterHub hosted at TACC, you will need to perform following steps:
Request for TACC login credentials from Tacc User Portal (TUP) This request is usually completes within 1-2 business days.
Please email cicsupport@tacc.utexas.edu in case you did not hear back.
Once your receive TACC credentials, login to TACC JupyterHub
Depending on whether you have access to multiple notebook images, you may see a spawner screen with list of dockerhub images to choose from.
You may select any of the images and launch your notebook server.
Packages can be installed by the users in the notebook as needed.
If you need more memory for performing your scientific experiments, please contact our staff at cicsupport@tacc.utexas.edu and we can discuss it further with you.
Depending on your scientific use-case, evalulate if a separate instance of JupyterHub is needed for your project. You may follow the steps below:
Email to cicsupport@tacc.utexas.edu with a breif description of your use-case and request for a meeting with our staff.
During the meeting, explain your use-case and mention the memory and CPU requirements for running experiments.
Mention any specific packages that should be loaded in the JupyterHub for running your experiments.
Evaluate whether your workflow can be containerized into a docker image.
Login to JupyterHub
Users can login to Scinco JupyterHub using their TACC credentials. Once login is successful, users can spawn their notebook servers.
Selecting an Image
When launching a server, the JupyterHub you are using may ask you to select an image. If there is only one image, it will be selected by default. If there are multiple images availble for you, you can choose one from the list.
Starting a Notebook
Once you have launched your server, you can run start one or more notebooks.
Administration Portal
SCINCO JupyterHubs include an administrative portal. This interface allows Hub administrators to perform tasks such as:
Starting and stopping servers on behalf of users
Adding volume mounts to a Hub
Adding custom images to a Hub
See the list of users on the Hub
Configure user groups for the Hub
If you are a PI and would like to control administrative functions on your SCINCO JupyterHub, please file a ticket for TACC staff to add you as an administrator.
Accessing the Administration Portal
TACC staff can add a user to the Administration Portal instance that is running in Kubernetes.
Typically, Administration Portals are available at https://<portal_name>-admin.io.jupyter.tacc.cloud.
You will be prompted for your TACC username and password. If you have been granted access to
the Administration Portal, you can log in and make alterations to the JupyterHub
Using the Administration Portal
Once you have successfully logged in, you will see the main page:
From here, you may choose the following options:
Warning
The JupyterHub must be restarted by TACC staff before any configuration changes will appear.
User Groups
User Groups provide a way for JupyterHub administrators to control user access to specific server images and volume mounts based on group membership. When accessing this part of the Administration Portal, you should see a list of groups for JupyterHub.
You can create a new User Group by clicking the + New User Group button. You will be prompted for a name for this group:
Alternatively, to edit an existing group you can click on the group name in the table.
Editing a User Group
When editing a user group, you may control who has access to this group, what images the group has in addition to global JupyterHub images, and what volume mounts will be available in addition to global JupyterHub volume mounts.
You may add a username to this group by clicking the + Add User button. Alternatively, you may remove a user by clicking on the username in the group and clicking the Delete button.
Adding images and volume mounts follow a process that are similar to how global JupyterHub images and mounts are added. Please see the following:
Hub Images
JupyterHub images are Docker images that are used to spawn a user’s JupyterHub notebook server. Users may select any image that is available on your JupyterHub to spawn their notebook server. Images can be built to offer different Python packages or other customizations.
When a user logs in to a JupyterHub, if there is more than one image available they will be prompted to select an image to be used to start their notebook server. It may appear like this:
Adding or Editing Images
To add a new image option, click the + New Image button. Alternatively, to edit or delete an existing image option, click on its name in the table.
When editing an image option, the Display Name is the name that is shown in the Server Options prompt in the server spawner. The Image Name is the Docker image name used to for this server option.
Hub Volume Mounts
Volume mounts allow a user to access file system data from within their JupyterHub server. Volume mounts can either be data that is locally available on the Kubernetes node running the server or an external NFS mount.
Adding or Editing Images
To add a new mount, click the + New Mount button. Alternatively, to edit or delete an existing mount, click on its name in the table. When creating a mount, select a Mount Type of either Host Path or NFS.
Configuring a Host Path Mount
A Host Path mount is a Notebook Server mount to a file system location on the node that is hosting the server. For host paths to be available as mount options, they must be configured by TACC staff. If you would like a new host path mount, you must file a ticket requesting the host path before making configuration changes in the Administration Portal.
The Remote Server option is disabled for Host Path mounts. The Path option is the file system path on the host node that will be mounted. The Mount Path option is the path as it appears in a user’s Notebook server. The Read Only option restricts a user’s Notebook server write access to that path.
Configuring an NFS Mount
An NFS mount is a Notebook Server mount to a remote file system that has been exported via NFS. If you would like to configure an NFS mount, you must file a ticket to verify that an NFS export has been configured before making configuration changes in the Administration Portal.
The configuration options here are similar to those available when configuring a Host Path
mount, except that you may now specify an address for the Remote Server. It does not
need to be prefixed with nfs://.
Hub Admin
When a user logs in to a JupyterHub the first time, they are registered as a user on the JupyterHub. At this point, their activity can be viewed through Hub Admin and you may perform administrative actions on their Notebook server.
You may sort the list of users by clicking the arrows at the top of each column on the table. Users that are listed as Inactive do not have a running Notebook server. Users that have running Notebook servers will show timestamps for their activity. To view a user, click on their name. When viewing an active user, you will see a screen that may look like this:
Viewing a User’s Server
By clicking the View User Server button, you will be taken to the user’s active Notebook server. You are logged in to their server as that user using a token for authorization. From here, you may perform any action normally available to that user on their behalf, such as opening terminals, starting notebooks or shutting down the server.
Stopping a Server
Sometimes it is necessary to stop a Notebook server on behalf of the user. By clicking the Stop Server button, the user’s Notebook server will shut down. It may take some time for the operation to complete.