Search papers, labs, and topics across Lattice.
JuPyLive is a novel migration mechanism that facilitates the seamless transition of Jupyter notebooks between local workstations and high-performance computing (HPC) environments, enhancing user experience by eliminating migration complexities. By utilizing ElasticNotebook for in-memory state migration and automating resource allocation and communication, JuPyLive allows users to initiate and monitor migrations with a single click, without requiring any code changes or new syntax. This innovation democratizes access to HPC resources, enabling interdisciplinary researchers to efficiently scale their resource-intensive workflows.
JuPyLive transforms Jupyter notebooks into a bridge for effortless migration between local and HPC resources, enabling researchers to scale their workflows with just one click.
This work introduces JuPyLive, a migration mechanism that enables seamless transition of Jupyter notebooks between local resources of user's workstation and remote resources of high-performance computing~(HPC) environments, while preserving the user experience. JuPyLive eliminates the underlying complexities of migration process, enabling users to freely choose among available local and remote resources, directly within the familiar Jupyter notebook environment via a single click. JuPyLive leverages ElasticNotebook to manage in-memory state migration, it automates resource allocation on HPC cluster and orchestrates required remote communication channels between the source and destination to enable a bidirectional migration. Furthermore, HPC status monitor of JuPyLive provides a live overview of available remote resources, allowing users to make informed decisions on choosing the relevant resources before initiating a migration process. The proposed fully automatic mechanism requires no code changes or configurations by the end user, nor does it demand users to learn a new syntax, instead the migration process can be intuitively initiated and monitored using visual elements from within the Jupyter notebook. By bridging the gap between local workspace and remote resources, JuPyLive offers a seamless experience for scaling local resource-intensive workflows with minimal user intervention, thus further democratizing the usage of HPC clusters among the interdisciplinary researchers.
