GNU Parallel: Difference between revisions

Latest revision as of 17:10, 23 April 2024

GNU Parallel is useful for running multiple processes over an input list, similar to xargs.

Usage

parallel tutorial

-j lets you manually specify the number of parallel jobs.
- By default this is the number of CPU cores.
- You may want to decrease this to reduce memory usage or increase it if you're not CPU bound.

Progress

--progress shows some progress information
--bar shows a progress bar

Replacement strings

Parallel includes a handful of default replacement strings:

--rpl '{} '
--rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||=
         eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::'

Parent dirname

--rpl '{//} s:.*/(.*)/[^/]*:\1:;'

@@ Line 6: / Line 6: @@
 * <code>-j</code> lets you manually specify the number of parallel jobs.
 ** By default this is the number of CPU cores.
-** You may want to decrease this to reduce memory usage or increase it if you're not CPU/IO bound.
+** You may want to decrease this to reduce memory usage or increase it if you're not CPU bound.
 ;Progress