GNU parallel

OVERVIEW

GNU parallel takes a list of inputs — hosts, URLs, files, users, or table rows — and runs a command or small script once per line, on several cores of one machine or across computers over SSH. A job can also be a command that reads from a pipe; parallel splits the stream and feeds blocks into workers.

If you already use xargs or shell loops, the option names will feel familiar. Unlike a naive background loop, it groups each job's output so the result looks like a sequential run, which means you can pipe it into the next tool. Job logs, ETA, retries, and --dry-run are built in. Default concurrency is the number of CPU cores, which is often too high against a live target — set -j yourself.

USE CASES

Practical use cases

• 01

  Running nmap, httpx, or nuclei against a host list in controlled batches instead of a serial loop.

• 02

  Probing a subdomain or URL list with curl, dig, or wget during recon.

• 03

  Applying the same enum or parse script to many loot files at once.

• 04

  Spreading long jobs across a few lab VMs with --sshlogin when one box is not enough.

QUICK START

When a recon or enum step is a loop over a list — hosts, URLs, files — and a serial for-loop is too slow.

1. Confirm every line in the list is inside the authorized scope.
2. Dry-run the expansion with --dry-run so you see the exact commands before they fire.
3. Start with a low -j, --eta, and --joblog so you can watch rate and keep evidence.
4. Raise concurrency only if the target and the rules of engagement still look healthy.

parallel -j 10 --eta --joblog recon.log nmap -Pn -T3 -sV {} :::: hosts.txt

BEFORE YOU RUN IT

What to check before running it