UMIACS Servers: Difference between revisions

From David's Wiki
Line 102: Line 102:


==git==
==git==
The MBRC cluster has an older version of git available in the modules.<br>
The MBRC cluster has an git available in the modules.<br>
If you need <code>git lfs</code>, you can download the tarball from their [https://mirrors.edge.kernel.org/pub/software/scm/git/ repo].<br>
Here is a [https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.25.1.tar.gz link to 2.25.1]. Install with <code>make</code> and <code>make install</code>.<br>
Then you can download [https://github.com/git-lfs/git-lfs/releases git-lfs compiled] and drop it in <code>~/bin/</code>.<br>
Then you can download [https://github.com/git-lfs/git-lfs/releases git-lfs compiled] and drop it in <code>~/bin/</code>.<br>
Make sure <code>/bin</code> is in your path and run <code>git lfs install</code><br>
Make sure <code>/bin</code> is in your path and run <code>git lfs install</code><br>


;Notes
;Notes
* You can use git lfs with the older version of git but git clone will be slow because it will download lfs files sequentially.
* Make sure you have a recent version of git
** If you do decide to do this, use <code>git lfs clone</code> instead of <code>git clone</code>.
** E.g. <code>module load git/v2.25.1</code>

Revision as of 15:12, 4 March 2020

Notes on using UMIACS servers


Modules

Use modules to load programs you need to run.

Notes
  • You can load modules in your .bashrc file
# List loaded modules
module list

# Load a module
module load [my_module]

# List all available modules
module avail

Some useful modules in my .bashrc file

module load tmux
module load cuda/10.0.130
module load cudnn/v7.5.0
module load Python3/3.7.6
module load git

Python

Do not install anaconda. You will run out of space.
Load the Python 3 module adding the following to your .bashrc file

module load Python3/3.7.6
export PATH="${PATH}:$(python3 -c 'import site; print(site.USER_BASE)')/bin"

Then run the following to get pip installed

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py --user
Notes
  • You will need to install things with pip --user
  • You may need to add your local site-packages to your PYTHONPATH environment variable
    • Add this to .bashrc:
    • export PYTHONPATH="${PYTHONPATH}:/nfshomes/$(whoami)/.local/lib/python3.7/site-packages/"

Install PyTorch

pip install --user torch===1.3.1 torchvision===0.4.2 -f https://download.pytorch.org/whl/torch_stable.html

MBRC Cluster

See UMIACS MBRC

Job

srun --pty --gres=gpu:1 --mem=16G --qos=default --time=04:00:00 bash

.bashrc

My .bashrc
#PS1='\w$ '
PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$'

if test -f "/opt/rh/rh-php72/enable"; then
    source /opt/rh/rh-php72/enable
fi

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

command_exists() {
  type "$1" &> /dev/null ;
}


# Modules
if command_exists module ; then
  module load tmux
  module load cuda/10.0.130
  module load cudnn/v7.5.0
  module load Python3/3.7.6
fi
if command_exists python3 ; then
  alias python=python3
fi

if command_exists python3 ; then
  export PATH="${PATH}:$(python3 -c 'import site; print(site.USER_BASE)')/bin"
fi
export PYTHONPATH="${PYTHONPATH}:/nfshomes/dli7319/.local/lib/python3.7/site-packages/"

export PATH="${HOME}/bin/:${PATH}"

git

The MBRC cluster has an git available in the modules.
Then you can download git-lfs compiled and drop it in ~/bin/.
Make sure /bin is in your path and run git lfs install

Notes
  • Make sure you have a recent version of git
    • E.g. module load git/v2.25.1