R-GAU
gau
Fetches known URLs for a domain from the Wayback Machine, Common Crawl, and AlienVault OTX.
OVERVIEW
gau ("getallurls") queries external archives — the Wayback Machine, Common Crawl, AlienVault OTX, and optionally URLScan — for every URL they've ever indexed under a given domain, and prints the list. Because it only talks to third-party archive services, not the target, it's one of the few recon steps that produces zero traffic against the actual asset.
The output regularly surfaces old parameters, deprecated API endpoints, backup files, and paths that no longer show up in a fresh crawl but may still be live or reveal how the application used to be structured — useful context before active enumeration starts.
USE CASES
Practical use cases
- 01
Surfacing old or forgotten endpoints and parameters not visible in a fresh crawl.
- 02
Building a seed URL list for parameter fuzzing without generating target traffic first.
- 03
Finding backup files, old API versions, or debug paths that once existed publicly.
- 04
Cross-referencing archived URLs against a current crawl (katana) to spot what's changed.
QUICK START
When you want a fast, passive list of historical and third-party-indexed URLs for a domain, with no traffic sent to the target itself.
- Confirm the domain is in scope — even though the queries hit third-party archives, not the target.
- Install gau via Go install or a precompiled release binary.
- Run it against the target domain and save the raw URL list.
- Deduplicate and filter the list by extension or path pattern for what's relevant.
- Re-verify liveness of interesting URLs with httpx before treating them as current.
echo target.example.com | gau --subs > urls.txtBEFORE YOU RUN IT
What to check before running it
Results come from historical archives, so a large fraction of URLs point to pages, parameters, or subdomains that are stale or decommissioned — re-verify with httpx before acting on any of them.
Archived URLs can include sensitive-looking parameters from years ago; treat old data with the same handling care as live findings.
Because it's purely passive against third-party services, gau tells you nothing about current behavior — it's a lead generator, not a confirmation step.