Training workloads for Profile-Guided Optimization (PGO) builds.

PGO uses runtime profile data to guide compiler optimizations (inlining, branch prediction, code layout), typically improving throughput by 5-20%.

The process has three phases:

1. Instrument — Build with -fprofile-generate
2. Train — Run representative workloads to collect profile data
3. Optimize — Rebuild with -fprofile-use

The two supported flows differ in how profile data is collected. Clang writes one .profraw file per process, which must be merged into a single .profdata before the optimize phase. GCC's libgcov instead merges counters into .gcda files next to each object file as each process exits, so there is no merge step.

From a VS Developer Command Prompt, at the repo root:

# Step 1: Build the instrumented binary
vcbuild.bat pgo-generate

# Step 2: Run workloads to collect profile data
python tools\pgo\pgo.py

# Step 3: Build the optimized binary
vcbuild.bat pgo-use

pgo.py expects the instrumented binary at Release\node.exe (produced by step 1) and writes node.profdata to the repo root (consumed by step 3).

The script finds llvm-profdata in the Visual Studio LLVM toolset, then PATH. Set LLVM_PROFDATA to the matching tool when using a different Clang installation.

# Optionally set a longer training duration (default: 15s per script)
python tools\pgo\pgo.py --duration=30

# Step 1: Build the instrumented binary
./configure --enable-pgo-generate
make

# Step 2: Run workloads to collect profile data
./out/Release/node tools/pgo/pgo-run-all.js --duration=15 --verbose

# Step 3: Build the optimized binary
./configure --enable-pgo-use
make

Step 2 needs no driver script. Each object file gets one counter file beside it, with the same basename and a .gcda extension:

out/Release/obj/src/node_base.node_binding.o    # from step 1
out/Release/obj/src/node_base.node_binding.gcda # from step 2

Keep out/ intact between steps 1 and 3. GCC records the .gcda path into each object at compile time, so make clean or make distclean discards the training data and step 3 silently produces an ordinary build.

The build passes -fprofile-correction, which is required here. Counter updates from the worker threads and the libuv thread pool race with each other, and GCC treats the resulting inconsistent profile as an error unless told to smooth it out.

From the repo root:

# Step 1: Build the instrumented binary
./configure --ninja --enable-pgo-generate
make

# Step 2: Run workloads to collect profile data
python3 tools/pgo/pgo.py

# Step 3: Build the optimized binary
./configure --ninja --enable-pgo-use
make

pgo.py expects the instrumented binary at out/Release/node (produced by step 1) and writes node.profdata to the repo root (consumed by step 3).

The script finds llvm-profdata through xcrun on macOS and PATH on Linux. Set LLVM_PROFDATA to the matching tool when using a different Clang installation.

# Optionally set a longer training duration (default: 15s per script)
python3 tools/pgo/pgo.py --duration=30

On all platforms, pgo.py collects raw profiles in a fresh directory and replaces node.profdata after a successful merge. It removes raw profiles after success and preserves them if training or merging fails. Training failures stop the script so the workloads can be fixed before trying again.

All scripts use only Node.js built-in modules (no npm dependencies). Each script is run as a separate process via fork().

The orchestrator can also be invoked directly (e.g. for testing individual workloads). When used with pgo.py, this is handled automatically.

# Run all scripts
./out/Release/node tools/pgo/pgo-run-all.js --duration=15 --verbose

# Run specific scripts
./out/Release/node tools/pgo/pgo-run-all.js --scripts=http-server,json,crypto --duration=30

# Show help
./out/Release/node tools/pgo/pgo-run-all.js --help

Each script reads the PGO_TRAINING_DURATION environment variable (in milliseconds) to determine how long to run. The orchestrator sets this automatically from the --duration flag (in seconds).

tools/pgo/
├── pgo.py                  # Clang training driver (collect + merge)
├── pgo-run-all.js          # Training orchestrator
├── pgo-http-server.js      # HTTP server + client workload
├── pgo-json.js             # JSON parse/stringify workload
├── pgo-crypto.js           # Crypto operations workload
├── pgo-streams-buffers.js  # Streams and Buffer workload
├── pgo-fs.js               # File system operations workload
├── pgo-async-patterns.js   # Promise/async, EventEmitter, timers workload
├── pgo-url-string.js       # URL parsing, string ops, regex workload
├── pgo-compression.js      # Gzip/brotli/deflate compression workload
├── pgo-net.js              # TCP networking and DNS workload
├── pgo-module-loading.js   # Module require/import, VM compilation workload
├── pgo-child-workers.js    # Worker threads workload
└── README.md               # This file