Notebook Setup for Data Science
Remote setup to run machine learning workflows
The following are the steps to set up a remote jupyter notebook server that one can connect locally to run their machine learning.
The steps are as follows
CREATE EXEC SCRIPT
- CREATE a file jupyter-startup.sh
source ~/miniconda3/etc/profile.d/conda.sh
conda activate YOUR_CONDA_VIRTUAL_ENVIRONMENT
jupyter notebook
CREATE JUPYTER SERVICE
CREATE a following jupyter service in your home directory
COPY the file to /etc/systemd/system/jupyter.service
[Unit]
Description=Jupyter Notebook Service Description
[Service]
Type=simple
PIDFile=/run/jupyter.pid
ExecStart=/home/YOUR_USER_NAME/jupyter-startup.sh
User=YOUR_USER_NAME
Group=YOUR_USER_NAME
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
- sudo systemctl enable jupyter.service
- sudo systemctl daemon-reload
- sudo systemctl restart jupyter.service
Now you have 2 options
SSH Tunnel (option 1)
- Setup a password to the notebook server so that we don't have to provide token every single time. On the remote server, do the following
jupyter notebook password
#Enter password: ****
#Verify password: ****
#[NotebookPasswordApp] Wrote hashed password to /Users/<<you>>/.jupyter/jupyter_notebook_config.json
- Run SSH tunnel to setup a link on the local host
Pick any port number for LOCAL_PORT. You may need to enter a passphrase.
ssh -N -f -L localhost:<LOCAL_PORT>:localhost:8080 username@remotemachine -p <<PORT ID>> -v
- Type localhost:8888/tree in your local browser to access the jupyter notebook setup remotely.
PUBLIC URL (option 2)
CONFIG JUPYTER NOTEBOOK
- Create Jupyter Notebook Configuration
jupyter notebook –generate-config
CREATE PASSWORD and CERTIFICATES
- First generate a new password to login
from notebook.auth import passwd
import IPython
passwd()
- Generate the certificate file. It will be used in the configuration step
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mykey.key -out mycert.pem
EDIT NOTEBOOK CONFIGURATION FILES
cd .jupyter
nano jupyter_notebook_config.py
Open the configuration file and edit as follows
c.NotebookApp.allow_origin = '*'
c.NotebookApp.allow_remote_access = True
c.NotebookApp.certfile = 'YOUR_CERTIFICATE_FILE'
c.NotebookApp.ip = '0.0.0.0'
c.NotebookApp.open_browser = False
c.NotebookApp.password = u''
c.NotebookApp.port = ANY_PORT_YOU_LIKE
EDIT FIREWALL RULES (if needed)
sudo ufw allow from <<IP ADDRESS>>