GNU Parallel: Difference between revisions

From David's Wiki
No edit summary
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
GNU parallel is useful for running multiple processes over an input list, similar to xargs.<br>
GNU Parallel is useful for running multiple processes over an input list, similar to xargs.<br>


==Usage==
==Usage==
[https://www.gnu.org/software/parallel/parallel_tutorial.html parallel tutorial]
[https://www.gnu.org/software/parallel/parallel_tutorial.html parallel tutorial]
* <code>-j</code> lets you manually specify the number of parallel jobs.
** By default this is the number of CPU cores.
** You may want to decrease this to reduce memory usage or increase it if you're not CPU bound.
;Progress
* <code>--progress</code> shows some progress information
* <code>--bar</code> shows a progress bar


==Replacement strings==
==Replacement strings==

Latest revision as of 17:10, 23 April 2024

GNU Parallel is useful for running multiple processes over an input list, similar to xargs.

Usage

parallel tutorial

  • -j lets you manually specify the number of parallel jobs.
    • By default this is the number of CPU cores.
    • You may want to decrease this to reduce memory usage or increase it if you're not CPU bound.
Progress
  • --progress shows some progress information
  • --bar shows a progress bar

Replacement strings

Parallel includes a handful of default replacement strings:

--rpl '{} '
--rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||=
         eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::'
Parent dirname

--rpl '{//} s:.*/(.*)/[^/]*:\1:;'