Anup Khanal's Blog

Run Streamlit App on a HPC System

Anup Khanal's photo
Anup Khanal
·

7 min read

Streamlit is a popular open-source framework for data scientists and AI/ML engineers to create interactive frontends using Python. Hosting a streamlit app on a local machine is a straightforward process for smaller datasets or models. But there are cases where the resources of a local machine might not be sufficient enough to host or process the underlying data; for e.g., you are testing a LangChain app to create a chatbot using Llama 3.1 70b model or any other model for that matter that too much for your local machine resources and you have access to HPC system which is capable. To test your app, you can use the following process to start a streamlit app on the remote HPC and interact with it from your local browser.

The following script examples are for an HPC system with SLURM as a workload manager; if your HPC uses any other work manager, the overall process would still be similar.

Requirements

  • conda (or any other package manager) or python virtual env with streamlit installed

WARNINGS

Before you read further. As you might have been explained to you by your HPC admins

  • DO NOT run jobs (in this case, the streamlit app) on your login node; this might get you BANNED.

  • In most configurations, all the network traffic to and from the compute node will pass through the login nodes, so the public IP provided by the streamlit server to access your app won’t most likely work (you should disable them anyway).

TLDR

start streamlit app on the host machine → create an SSH tunnel with a port specified → view the app on the local machine browser

#!/bin/bash
ENVIRONMENT=general
LOGIN_HOST=$(hostname -s)
LOGIN_HOST_DOMAIN='<your HPC host domain>'
NODE_TYPE=<your HPC node_type>
source ~/anaconda3/etc/profile.d/conda.sh
srun -p $NODE_TYPE --pty bash -c '
source ~/anaconda3/etc/profile.d/conda.sh
conda activate '$ENVIRONMENT'
LOCAL_JN_PORT=$(expr 50000 + ${SLURM_JOBID: -4})
SSH_CTL=$TMPDIR/.ssh-tunnel-control
ssh -f -g -N -M -S $SSH_CTL \
    -R *:$LOCAL_JN_PORT:localhost:$LOCAL_JN_PORT \
     '$LOGIN_HOST'
echo
echo "========================================================"
echo
echo "Your View your Streamlit App:"
echo "1) Mac/Linux/MobaXterm users: run the following command FROM A NEW LOCAL TERMINAL WINDOW (not this one)"
echo
echo "ssh -L${LOCAL_JN_PORT}:localhost:${LOCAL_JN_PORT} $USER@'$LOGIN_HOST'.'$LOGIN_HOST_DOMAIN'"
echo
echo "For other users (PuTTY, etc) create a new SSH session and tunnel port $LOCAL_JN_PORT to localhost:$LOCAL_JN_PORT"
echo
echo "========================================================"
streamlit hello  --server.port=$LOCAL_JN_PORT --browser.serverAddress localhost
ssh -S $SSH_CTL -O exit localhost 2>&1 > /dev/null
exit'

Replace the ENVIRONMENT, LOGIN_HOST_DOMAIN, NODE_TYPE, and ‘hello’ with the Python file, and you are good to go

With some Bells & Whistles

  1. Set up some variables that can be used later to configure the type of compute nodes to be used (here, we will use GPU); for more node info, please refer to SLURM documentation here or your HPC user guide.
ENVIRONMENT=<your conda environment>
TIMELIMIT="15:59:59"
CPUS_PER_TASK=24
MEM_PER_CPU=0
JOB_NAME="<job name>"
LOGIN_HOST=$(hostname -s)
GPU_TYPE="<gpu type>"
NODE_TYPE="<your HPC node_type>"
LOGIN_HOST_DOMAIN="<your HPC host domain>"
  1. Steup Conda and add a countdown (this is totally optional)
# Source conda so we can use it
source ~/anaconda3/etc/profile.d/conda.sh

# Countdown function to delay the streamlit server startup but still show output to user 
# this is optional and can be skipped
countdown()
(
  IFS=:
  set -- $*
  secs=$(( ${1#0} * 3600 + ${2#0} * 60 + ${3#0} ))
  while [ $secs -gt 0 ]
  do
    sleep 1 &
    printf "\r%02d:%02d:%02d" $((secs/3600)) $(( (secs/60)%60)) $((secs%60))
    secs=$(( $secs - 1 ))
    wait
  done
  echo
)

# export function to make sure it can be used in subshells
export -f countdown
  1. Setup so the script can take some arguments
# Get all the passed in options
while getopts "dhn:t:c:m:" opt; do
  case ${opt} in
    n )
      ENVIRONMENT=$OPTARG
      ;;
    h )
      echo "Usage: $(basename $0) [-h] [-d] [-n ENVIRONMENT] [-c CPUS] [-t TIMELIMIT] [-m MEM]"
      echo "    -h            Display this message"
      echo "    -d            Don't check to see if a password is set (default checks for password)"
      echo "    -n            Name of conda environment (default=${ENVIRONMENT}) that jupyter is"
      echo "                   installed into.  If you pass -n base it will use the base environment"
      echo "    -t            Timelimit of job to run (default=${TIMELIMIT})"
      echo "    -c            Number of cpus per task to pass to srun (default=$CPUS_PER_TASK)"
      echo "    -m            Memory per cpu (default=${MEM_PER_CPU} megs)"
      exit 0
      ;;
    t )
      TIMELIMIT=$OPTARG
      ;;
    c )
      CPUS_PER_TASK=$OPTARG
      ;;
    m )
      MEM_PER_CPU=$OPTARG
      ;;
    \? )
      echo "Invalid option: $OPTARG" 1>&2
      exit -1
      ;;
    : )
      echo "Invalid option: $OPTARG requires an argument" 1>&2
      exit -1
      ;;
  esac
done
shift $((OPTIND -1))

# We should be able to active the environment now
# Check to make sure it exists first
if [[ $($CONDA_EXE env list | awk '{print $1}') != *$ENVIRONMENT* ]]; then
    echo "Cannont find environment: $ENVIRONMENT"
    exit 1
fi
conda activate $ENVIRONMENT
  1. Start the job with the defined configurations
# Start up an interactive job to with the streamlit server with the user-asked configuration of CPUs/GPUs and stuffs 
srun -c $CPUS_PER_TASK -t $TIMELIMIT --mem-per-cpu $MEM_PER_CPU -J $JOB_NAME -p $NODE_TYPE --gres=gpu:$GPU_TYPE:1 --pty bash -c '
source ~/anaconda3/etc/profile.d/conda.sh
conda activate '$ENVIRONMENT'
LOCAL_JN_PORT=$(expr 50000 + ${SLURM_JOBID: -4})
SSH_CTL=$TMPDIR/.ssh-tunnel-control
ssh -f -g -N -M -S $SSH_CTL \
    -R *:$LOCAL_JN_PORT:localhost:$LOCAL_JN_PORT \
     '$LOGIN_HOST'
echo
echo "========================================================"
echo
echo "Your Streamlit Server is now running"
echo "To Connect:"
echo "1) Mac/Linux/MobaXterm users: run the following command FROM A NEW LOCAL TERMINAL WINDOW (not this one)"
echo
echo "ssh -L${LOCAL_JN_PORT}:localhost:${LOCAL_JN_PORT} $USER@'$LOGIN_HOST'.'$LOGIN_HOST_DOMAIN'"
echo
echo "For other users (PuTTY, etc) create a new SSH session and tunnel port $LOCAL_JN_PORT to localhost:$LOCAL_JN_PORT"
echo
echo "========================================================"
echo "Starting Streamlit Server in..."
countdown "00:00:05"
echo "========================================================"
echo
echo "Connect to your Streamlit App with the following links."
echo "  http://localhost:$LOCAL_JN_PORT"
echo
echo "========================================================"
echo
# here goes the streamlit command
streamlit hello  --server.port=$LOCAL_JN_PORT --browser.serverAddress localhost
sleep 3
ssh -S $SSH_CTL -O exit localhost 2>&1 > /dev/null
sleep 3
exit'

If your script has some of its own arguments, the double dash (--) will be for the arguments of the Python script rather than the streamlit itself; streamlit arguments will be then passed by using a single dash (-) instead.

For more information, please refer to Streamlit documentation here.

Putting all together:

#!/bin/bash
ENVIRONMENT=<your conda environment>
TIMELIMIT="15:59:59"
CPUS_PER_TASK=24
MEM_PER_CPU=0
JOB_NAME="<job name>"
LOGIN_HOST=$(hostname -s)
GPU_TYPE="<gpu type>"
NODE_TYPE="<your HPC node_type>"
LOGIN_HOST_DOMAIN="<your HPC host domain>"

# Source conda so we can use it
source ~/anaconda3/etc/profile.d/conda.sh

# Countdown function to delay the streamlit server startup but still show output to user 
# this is optional and can be skipped
countdown()
(
  IFS=:
  set -- $*
  secs=$(( ${1#0} * 3600 + ${2#0} * 60 + ${3#0} ))
  while [ $secs -gt 0 ]
  do
    sleep 1 &
    printf "\r%02d:%02d:%02d" $((secs/3600)) $(( (secs/60)%60)) $((secs%60))
    secs=$(( $secs - 1 ))
    wait
  done
  echo
)

# export function to make sure it can be used in subshells
export -f countdown

# Get all the passed in options
while getopts "dhn:t:c:m:" opt; do
  case ${opt} in
    n )
      ENVIRONMENT=$OPTARG
      ;;
    h )
      echo "Usage: $(basename $0) [-h] [-d] [-n ENVIRONMENT] [-c CPUS] [-t TIMELIMIT] [-m MEM]"
      echo "    -h            Display this message"
      echo "    -d            Don't check to see if a password is set (default checks for password)"
      echo "    -n            Name of conda environment (default=${ENVIRONMENT}) that jupyter is"
      echo "                   installed into.  If you pass -n base it will use the base environment"
      echo "    -t            Timelimit of job to run (default=${TIMELIMIT})"
      echo "    -c            Number of cpus per task to pass to srun (default=$CPUS_PER_TASK)"
      echo "    -m            Memory per cpu (default=${MEM_PER_CPU} megs)"
      exit 0
      ;;
    t )
      TIMELIMIT=$OPTARG
      ;;
    c )
      CPUS_PER_TASK=$OPTARG
      ;;
    m )
      MEM_PER_CPU=$OPTARG
      ;;
    \? )
      echo "Invalid option: $OPTARG" 1>&2
      exit -1
      ;;
    : )
      echo "Invalid option: $OPTARG requires an argument" 1>&2
      exit -1
      ;;
  esac
done
shift $((OPTIND -1))

# We should be able to active the environment now
# Check to make sure it exists first
if [[ $($CONDA_EXE env list | awk '{print $1}') != *$ENVIRONMENT* ]]; then
    echo "Cannont find environment: $ENVIRONMENT"
    exit 1
fi
conda activate $ENVIRONMENT


# Start up an interactive job to with the streamlit server with the user-asked configuration of CPUs/GPUs and stuffs 
srun -c $CPUS_PER_TASK -t $TIMELIMIT --mem-per-cpu $MEM_PER_CPU -J $JOB_NAME -p $NODE_TYPE --gres=gpu:$GPU_TYPE:1 --pty bash -c '
source ~/anaconda3/etc/profile.d/conda.sh
conda activate '$ENVIRONMENT'
LOCAL_JN_PORT=$(expr 50000 + ${SLURM_JOBID: -4})
SSH_CTL=$TMPDIR/.ssh-tunnel-control
ssh -f -g -N -M -S $SSH_CTL \
    -R *:$LOCAL_JN_PORT:localhost:$LOCAL_JN_PORT \
     '$LOGIN_HOST'
echo
echo "========================================================"
echo
echo "Your Streamlit Server is now running"
echo "To Connect:"
echo "1) Mac/Linux/MobaXterm users: run the following command FROM A NEW LOCAL TERMINAL WINDOW (not this one)"
echo
echo "ssh -L${LOCAL_JN_PORT}:localhost:${LOCAL_JN_PORT} $USER@'$LOGIN_HOST'.'$LOGIN_HOST_DOMAIN'"
echo
echo "For other users (PuTTY, etc) create a new SSH session and tunnel port $LOCAL_JN_PORT to localhost:$LOCAL_JN_PORT"
echo
echo "========================================================"
echo "Starting Streamlit Server in..."
countdown "00:00:05"
echo "========================================================"
echo
echo "Connect to your Streamlit App with the following links."
echo "  http://localhost:$LOCAL_JN_PORT"
echo
echo "========================================================"
echo
# here goes the streamlit command
streamlit hello  --server.port=$LOCAL_JN_PORT --browser.serverAddress localhost
sleep 3
ssh -S $SSH_CTL -O exit localhost 2>&1 > /dev/null
sleep 3
exit'
hpcstreamlitAIMLTestingllm