GNU Parallel

GNU Parallel is useful for running multiple processes over an input list, similar to xargs.

Usage

parallel tutorial

• -j lets you manually specify the number of parallel jobs.
  • By default this is the number of CPU cores.
  • You may want to decrease this to reduce memory usage or increase it if you're not CPU bound.

Progress

• --progress shows some progress information
• --bar shows a progress bar

Replacement strings

Parallel includes a handful of default replacement strings:

--rpl '{} '
--rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||=
         eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::'

Parent dirname

--rpl '{//} s:.*/(.*)/[^/]*:\1:;'