This is an archive of the old MediaWiki-based ImageJ wiki. The current website can be found at imagej.net.

Linux command line tutorial

Connecting to the cluster

We will use SSH (secure shell) to connect to the cluster. On Mac OSX we need to simply open a terminal window (under Applications->Utilities), on PC we may need to download an ssh client such as Putty.

In the terminal type:

ssh madmax.mpi-cbg.de

and enter your password

tomancak@madmax.mpi-cbg.de's password: 

Now you are on the cluster

Last login: Sun May 19 09:01:24 2013 from 10.1.7.16
"Platform HPC" 3.1 (build 7211) Management Node
Welcome to the MPI-CBG cluster masternode.
[tomancak@madmax ~]$

Moving around

pwd - print working directory

[tomancak@madmax ~]$ pwd
/home/tomancak

cd - change directory

[tomancak@madmax ~]$ cd /projects
[tomancak@madmax projects]$ pwd
/projects

~ - tilda means home directory

[tomancak@madmax projetcs]$ cd ~
[tomancak@madmax ~]$ pwd
/home/tomancak

/ - slash demarcates directories and by itself means the root directory

[tomancak@madmax ~]$ cd /
[tomancak@madmax /]$ pwd
/

. - current directory, if we want to for example execute a script in the current directory we do it like this:

./executable_script

This tells the interpreter to look for the file executable_script in the current directory and not somewhere else.

.. - one directory up

[tomancak@madmax projects]$ cd ..
[tomancak@madmax /]

pressing tab - auto-completion, we don't need to type long names of directories, start typing the name and press tab and the linux system will either auto-complete or show you the options you have

[tomancak@madmax ~]$ cd /projects/toman + press tab
[tomancak@madmax ~]$ cd /projects/tomancak_lightsheet
[tomancak@madmax tomancak_lightsheet]$

Manipulating files

ls

more

tail

nano

mv

cp

scp

rm

mkdir

rmdir

gunzip

tar

Permissions

chmod

su

sudo

Pipes and redirects

|

>

wc

Misc

which

top

watch

grep

sort

find

man

Using it on the cluster

Here we put all of that goodness to work to make our life on a cluster easier.

Once we submit the jobs we want to monitor the progress. For example bjobs -r gives us an output summarising our running jobs. If there are many its not that useful. How about counting them?

bjobs -r | wc -l
160

More useful, we have 160 jobs running. Still how about the pending ones? We can do that:

bjobs -r | grep PEND | wc -l
40

Ok 40. Now, how many jobs actually finished successfully? Lets say they create some .tif files a few directories down

ls ../../output/*.tif | wc -l
200

Good, looks like 200 are finished. Its tempting to keep pressing enter on those commands but its also tiresome. Using

watch -n5 'ls ../../output/*.tif | wc -l' 

outputs continuously

Every 5.0s: ls ../../output/*.tif | wc -l                                                                                             
Mon May 20 21:16:37 2013
180

i.e. we get the number of files automatically updated every 5 seconds (without the -n paramater every 2s by default).