Visitar URL original
Revamp docs (#879) · exaloop/codon@9f41a7b · GitHub
Skip to content

Commit 9f41a7b

Browse files
arshajiiarshajii
authored
Revamp docs (#879)
* Revamp docs * Update footer * Update docs * Refactor * Add benchmarks * Update benchmarks * Remove redundant files
1 parent 6308896 commit 9f41a7b

590 files changed

Lines changed: 128855 additions & 3119 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -230,17 +230,18 @@ jobs:
230230
with:
231231
python-version: '3.11'
232232

233-
- name: Install dependencies
233+
- name: Set up Node.js
234+
uses: actions/setup-node@v6
235+
with:
236+
node-version: '22'
237+
cache: npm
238+
cache-dependency-path: docs/package-lock.json
239+
240+
- name: Install documentation dependencies
234241
run: |
235242
python -m pip install --upgrade pip
236-
python -m pip install properdocs \
237-
mkdocs-autorefs \
238-
mkdocs-macros-plugin \
239-
mkdocs-material \
240-
mkdocs-material-extensions \
241-
mkdocs-redirects
242-
sudo apt-get update
243-
sudo apt-get install -y pngquant
243+
python -m pip install -r docs/requirements.txt
244+
npm ci --prefix docs
244245
245246
- name: Download Artifact
246247
uses: actions/download-artifact@v8
@@ -255,14 +256,22 @@ jobs:
255256
codon-deploy-linux-x86_64/bin/codon doc codon-deploy-linux-x86_64/lib/codon/stdlib > docs.json
256257
python scripts/docgen.py docs.json docs/libraries/api "$PWD/codon-deploy-linux-x86_64/lib/codon/stdlib"
257258
258-
- name: Build MkDocs site
259-
run: properdocs build --strict
259+
- name: Build and validate Docusaurus site
260+
run: npm run build --prefix docs
261+
262+
- name: Upload documentation site
263+
uses: actions/upload-artifact@v7
264+
with:
265+
name: codon-docs
266+
path: docs/build/
267+
if-no-files-found: error
268+
retention-days: 7
260269

261270
- name: Deploy to GitHub Pages
262-
if: github.ref == 'refs/heads/master'
271+
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
263272
uses: peaceiris/actions-gh-pages@v4
264273
with:
265274
github_token: ${{ secrets.GITHUB_TOKEN }}
266-
publish_dir: ./site
275+
publish_dir: ./docs/build
267276
force_orphan: true
268277
cname: docs.exaloop.io

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ build*/
1818
install*/
1919
extra/python/src/jit.cpp
2020
extra/jupyter/build/
21-
/site
2221

2322
# Packages #
2423
############

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,12 @@ print(2 + 2) # EXPECT: 4
4848
that if you have, for example, an `EXPECT` in a loop, you will need to duplicate it however many times the loop is executed.
4949
Using `EXPECT` is helpful mainly in cases where you need to test control flow, **otherwise prefer the new style**.
5050

51+
## Documentation
52+
53+
Documentation lives in `docs/` and is built with Docusaurus. See the
54+
[documentation development guide](website/README.md) for API generation, local
55+
preview, shared-theme development, and production validation.
56+
5157
## Pull requests
5258

5359
Pull requests should generally be based on the `develop` branch. Before submitting a pull request, please make sure...

‎bench/README.md‎

Lines changed: 259 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,265 @@ by using various Codon-specific features, but their adaptations here are
1212
virtually identical to the original implementations (mainly just the use
1313
of the `pyperf` module is removed).
1414

15+
## Refreshing the public results
16+
17+
`refresh.py` runs 12 existing application workloads and 53 native NPBench kernels.
18+
The public catalogue groups `binary_trees`, `chaos`, `fannkuch`, `float`, `go`,
19+
`nbody`, and `spectral_norm` under **Pyperformance**; the other five workloads
20+
remain under **Applications**. The integer `sum` benchmark is excluded.
21+
It stages the executed sources, inputs, compiler commands, executable hashes,
22+
raw samples, validation outcomes, and runtime information in a work directory.
23+
`export.py` converts completed reports into the documentation site's dataset
24+
and downloadable source snapshots. It refuses incomplete runs and changed
25+
artifacts as chart entries. Completed timings are included when outputs differ
26+
or a diagnostic heap-growth threshold is exceeded; original diagnostics remain
27+
in the raw report, with brief output-difference notes on affected benchmarks.
28+
29+
The November 2026 macOS setup uses the locally built Codon 0.20.6, CPython 3.15.2,
30+
PyPy 8.0.1 (Python 3.12.15), Clang 23.1.5, NumPy 2.5.4, SciPy 1.18.1, and
31+
Numba 0.68.1. The machine is an Apple M1 Max with 64 GiB of RAM, running
32+
macOS 26.6.2. These are native CPU comparisons with OpenMP and numerical-library
33+
defaults set to one thread. Codon's Fannkuch explicitly requests four OpenMP
34+
threads, overriding that default; the parallel Primes loop uses one thread.
35+
These distinctions are noted on the corresponding benchmarks. The existing
36+
CPU Mandelbrot program is used for Codon because this host has no NVIDIA GPU.
37+
Clang uses the active macOS SDK's libc++ headers and system runtime, avoiding
38+
an ABI mismatch between upstream LLVM headers and Apple's loaded libc++.
39+
40+
### Timing and validation
41+
42+
Applications retain their existing timing boundaries. Module-qualified Python
43+
timers use `perf_counter`, and C++ uses `steady_clock`; existing directly imported
44+
`time` calls retain their original wall timer. Each sample runs in a fresh process; the first process
45+
is excluded, followed by three measured processes in rotating backend order.
46+
Application allocation and any PyPy JIT work inside the original timer remain
47+
included. The C++ comparison is available only where a C++ implementation
48+
already exists. Output values are checked against CPython; the Chaos PPM uses
49+
an exact image check. Completed timings with differing outputs are included
50+
with a small note; the raw output-check result is not relabeled as a pass.
51+
52+
NPBench uses the upstream **L** preset for every kernel, not the historical
53+
`paper` preset. Inputs are initialized once with upstream NumPy, saved with
54+
shape, dtype, and SHA-256 metadata, and shared across all implementations.
55+
Each worker holds one pristine and one working copy of every array. Working
56+
arrays are reset in place with `np.copyto`; input loading, garbage collection,
57+
reset, and output serialization are outside the kernel timer. Kernel-created
58+
temporary allocations remain inside it. One warm-up precedes three samples
59+
in the same process, excluding Numba specialization from recorded times.
60+
Backend order rotates between kernels.
61+
62+
A separate native reset audit checks that 32 resets preserve buffer addresses,
63+
allocate less than 64 KiB, and do not grow the GC heap. The measured process
64+
also records heap size and live bytes after collection between full calls.
65+
Live growth exceeding the larger of 16 MiB or 5% of input storage is recorded
66+
as a diagnostic flag in the original measurement report. It does not exclude
67+
completed timings from the published charts. Python and Numba record their actual BLAS
68+
thread pools; each reported pool must use one thread.
69+
70+
NPBench compares every output element, rejects nonfinite values, and uses
71+
upstream per-case tolerances (defaults: `rtol=1e-5`, `atol=1e-8`, with a
72+
relative L2-error fallback below `1e-5`). Integer results are exact. Floating
73+
dtype differences are recorded rather than hidden by casting. Mutated arrays
74+
are checked even when the kernel returns `None`. The staged Numba MLP fixes
75+
an upstream row-maximum axis error in softmax; all adaptations are recorded.
76+
MLP's input uses NumPy's legacy RNG with seed 42; its upstream weight generator
77+
already uses a separate seed of 42 and is unchanged. The initializer's legacy
78+
RNG state is restored afterward.
79+
All three `azimint_hist` implementations use the upstream Numba portable
80+
weighted-histogram helpers. Codon and Python omit the Numba import and JIT
81+
decorators; their kernel sources are identical. Python executes the weighted
82+
histogram loop in the interpreter. Unweighted counts still use each runtime's
83+
`np.histogram`, so shared source does not imply identical library internals.
84+
Native kernels link Apple Accelerate on macOS and OpenBLAS ILP64 on Linux;
85+
NumPy and Numba use their installed BLAS
86+
libraries. These are end-to-end software-stack comparisons, not isolated
87+
compiler comparisons. Numba kernels and helpers use `fastmath=False`; other
88+
upstream options and algorithms are retained. The workers verify the effective
89+
dispatcher options before timing. Codon is compiled without `-fast-math`.
90+
91+
### Reproduction
92+
93+
The driver supports macOS and Linux. Build the local compiler first, then
94+
create an isolated work directory. The macOS setup uses these default paths:
95+
96+
```text
97+
build/benchmarks-20261106/
98+
venv/bin/python
99+
npbench/ # https://github.com/spcl/npbench
100+
toolchains/LLVM-23.1.5-macOS-ARM64/bin/clang++
101+
toolchains/pypy3.12-v8.0.1-macos_arm64/bin/pypy3
102+
```
103+
104+
The virtual environment requires the versions above plus `threadpoolctl==3.8.0`.
105+
The saved report records the exact NPBench commit, local compiler commit,
106+
source-diff fingerprint, runtime hashes, and SDK version. For exact
107+
reproduction, check out that NPBench commit and use the recorded sources.
108+
The TAQ input is the first 10 million lines of the supplied file. Word count
109+
uses the first 16 MiB, ending at a complete line. Neither source dataset is
110+
modified or redistributed; the report records the staged input hashes.
111+
112+
Compile and prepare both suites before measuring either:
113+
114+
```sh
115+
WORK="$PWD/build/benchmarks-20261106"
116+
PYTHON="$WORK/venv/bin/python"
117+
118+
"$PYTHON" bench/refresh.py --suite applications --prepare-only --samples 3 \
119+
--taq-source /path/to/taq-file --word-source /path/to/text-file \
120+
--output "$WORK/applications-final"
121+
"$PYTHON" bench/refresh.py --suite npbench --preset L --prepare-only --samples 3 \
122+
--output "$WORK/npbench-final"
123+
124+
"$PYTHON" bench/refresh.py --suite applications --prepared --resume --samples 3 \
125+
--timeout 900 --output "$WORK/applications-final"
126+
"$PYTHON" bench/refresh.py --suite npbench --preset L --prepared --resume --samples 3 \
127+
--timeout 900 --output "$WORK/npbench-final"
128+
129+
"$PYTHON" bench/export.py --work "$WORK"
130+
```
131+
132+
#### Numba-only refresh
133+
134+
The November 8 refresh disables fast math for all 53 Numba kernels on both
135+
machines. It preserves the original software environment, input hashes, and
136+
Codon/Python timings for 52 kernels. MLP is the sole exception: upstream used
137+
an unseeded legacy RNG for its 24-element input, and the original arrays were
138+
no longer available. All three MLP runtimes were therefore remeasured on the
139+
same newly seeded input; the large weight arrays are unchanged.
140+
141+
Use a frozen original report as the baseline and a separate output directory:
142+
143+
```sh
144+
OUT="$WORK/numba-no-fastmath-L-20261108-seeded"
145+
"$PYTHON" bench/refresh.py --work "$WORK" --suite npbench --preset L \
146+
--samples 3 --timeout 900 --numba-only --numba-fastmath disabled \
147+
--refresh-cases mlp --baseline-report "$WORK/npbench-final/report.json" \
148+
--output "$OUT" --prepare-only
149+
"$PYTHON" bench/refresh.py --work "$WORK" --suite npbench --preset L \
150+
--samples 3 --timeout 900 --numba-only --numba-fastmath disabled \
151+
--refresh-cases mlp --baseline-report "$WORK/npbench-final/report.json" \
152+
--output "$OUT" --prepared --resume
153+
"$PYTHON" bench/export.py --work "$WORK" --report "$OUT/report.json" \
154+
--output "$WORK/numba-no-fastmath-export-20261108"
155+
```
156+
157+
Use `--timeout 3600` on AWS and its `/benchmarks/aws` export URL prefix.
158+
Outside the explicitly refreshed cases, Python runs once only to validate
159+
outputs; those reference executions are not new timing samples. Baseline
160+
hash mismatches reject affected cases during preparation. The all-runtime exception retains the old
161+
input descriptors and still checks shapes, types, parameters, and metadata.
162+
Export into staging, then merge only the measured implementations; an isolated
163+
export must not replace the complete public dataset.
164+
165+
The site records per-implementation measurement dates. Its raw report identifies
166+
the refresh environment through `measurementRuns` and each new result's
167+
`measurementRun`. Numba kernels and helpers always use `fastmath=False`;
168+
the driver does not support enabling fast math.
169+
170+
#### Uniform histogram update
171+
172+
A subsequent histogram-only update uses the common implementation above on
173+
both machines, with Numba fast math still disabled. It retains the Codon and
174+
Numba samples from the uniform comparison run and adds Python measurements
175+
of that same source. Each published implementation has three samples; the
176+
pooled six-sample Numba reference used in the diagnostic comparison is not
177+
used for publication. The original input hashes are unchanged, and all three
178+
outputs also pass validation against the original NumPy implementation.
179+
The other 64 published workloads are unchanged by this update.
180+
181+
The driver now stages this shared source by default. To reproduce just this
182+
workload in a separate directory:
183+
184+
```sh
185+
"$PYTHON" bench/refresh.py --work "$WORK" --suite npbench \
186+
--cases azimint_hist --preset L --samples 3 --timeout 900 \
187+
--numba-fastmath disabled --output "$WORK/azimint-hist-uniform-L"
188+
"$PYTHON" bench/export.py --work "$WORK" \
189+
--report "$WORK/azimint-hist-uniform-L/report.json" \
190+
--output "$WORK/azimint-hist-uniform-export"
191+
```
192+
193+
Use the AWS timeout and URL prefix described below when running there.
194+
As with the Numba-only refresh, merge the isolated export into the full
195+
catalogue rather than replacing unrelated results. The raw report retains
196+
the carried-forward sample provenance and the original-NumPy output checks.
197+
198+
#### Linux and GPU setup
199+
200+
On Linux, provide `toolchains.json` in the work directory with explicit paths
201+
to the comparison tools. Keep these installations separate from the system tools:
202+
203+
```json
204+
{
205+
"clang": "/path/to/LLVM/bin/clang++",
206+
"pypy": "/path/to/pypy/bin/pypy3"
207+
}
208+
```
209+
210+
Enable transparent huge pages before preparing or running NPBench:
211+
212+
```sh
213+
printf 'always\n' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
214+
cat /sys/kernel/mm/transparent_hugepage/enabled
215+
```
216+
217+
The driver requires the active policy to be `always`, records it with the
218+
machine metadata, and checks that it remains unchanged. It also explicitly sets
219+
`NUMPY_MADVISE_HUGEPAGE=1`. Each Linux NPBench worker records `AnonHugePages`
220+
from `/proc/self/smaps_rollup` after loading its persistent input arrays and
221+
before timing. Small workloads may not need any huge pages. The policy is a
222+
runtime system setting and must be checked again after a reboot.
223+
224+
With a supported NVIDIA GPU and CUDA installation, add `--gpu-mandelbrot` when
225+
preparing applications. This selects the existing Codon implementation with
226+
`@par(gpu=True, collapse=2)`; the Python and PyPy implementations remain CPU
227+
workloads. The application timer includes GPU transfers and launch overhead.
228+
NPBench remains a CPU comparison. Linux C++ builds use `-march=native`.
229+
230+
Use the same preparation and measurement commands above, supplying `--work`
231+
and the appropriate `--output` paths. The AWS sweep uses `--timeout 3600` so
232+
the full CPU Mandelbrot reference can complete; the macOS sweep used 900 seconds.
233+
Allow enough RAM for pristine and working
234+
input copies plus kernel temporaries: the MLP `L` configuration alone has
235+
about 6.7 GiB of inputs. A 64 GiB instance provides room for the complete suite.
236+
When using instance-store storage, back up results and build artifacts to
237+
persistent storage before stopping the instance.
238+
239+
Publish another machine into a separate directory and URL namespace:
240+
241+
```sh
242+
"$PYTHON" bench/export.py --work "$WORK" \
243+
--output docs/data/benchmarks/aws --url-prefix /benchmarks/aws
244+
```
245+
246+
Each export stores identical source downloads, excerpts, and measurement harnesses
247+
once, using SHA-256 filenames. After adding or merging machine datasets, consolidate
248+
their shared sources from the publication root:
249+
250+
```sh
251+
"$PYTHON" bench/export.py --deduplicate --output docs/data/benchmarks
252+
```
253+
254+
This verifies source hashes before updating download URLs and removing redundant
255+
copies. Measurements, validation records, executed-source hashes, and raw reports
256+
are unchanged. Run `python -m unittest discover -s bench -p test_export.py` to test
257+
source consolidation.
258+
259+
Keep the machine free of builds and other benchmark runs during measurement.
260+
The process timeout applies to a whole process, including warm-up and output
261+
serialization, not to an individual recorded sample. Timed-out or incomplete
262+
implementations are retained without a chart bar or speedup. NPBench array
263+
artifacts are removed after each completed case to bound disk use; use
264+
`--keep-arrays` for investigation. Reports and source snapshots are retained.
265+
`--resume` skips completed cases. `--quick` for applications and `--preset S`
266+
for NPBench are validation modes and are rejected by the publication exporter.
267+
Downloadable raw reports replace machine-specific paths with `$CODON_ROOT`,
268+
`$BENCH_WORK`, and `$HOME`; downloadable NPBench harnesses use the same path
269+
redaction. Source downloads normalize a missing final newline; hashes of both
270+
the original executed source and the download are retained. The exporter writes
271+
tracked data under `docs/data/benchmarks`; the documentation asset preparation
272+
step copies it to the generated static directory.
273+
15274
## Benchmarks
16275

17276
- `chaos`: [Pyperformance's `chaos` benchmark](https://github.com/python/pyperformance/blob/main/pyperformance/data-files/benchmarks/bm_chaos/run_benchmark.py).
@@ -21,7 +280,6 @@ of the `pyperf` module is removed).
21280
- `spectral_norm`: [Pyperformance's `spectral_norm` benchmark](https://github.com/python/pyperformance/blob/main/pyperformance/data-files/benchmarks/bm_spectral_norm/run_benchmark.py).
22281
- `mandelbrot`: Generates an image of the Mandelbrot set. Codon version uses GPU via one additional `@par(gpu=True, collapse=2)` line.
23282
- `set_partition`: Calculates set partitions. Code taken from [this Stack Overflow answer](https://stackoverflow.com/a/73549333).
24-
- `sum`: Computes sum of integers from 1 to 50000000 with a loop. Code taken from [this article](https://towardsdatascience.com/getting-started-with-pypy-ef4ba5cb431c).
25283
- `taq`: Performs volume peak detection on an NYSE TAQ file. Sample TAQ files can be downloaded and uncompressed from [here](https://ftp.nyse.com/Historical%20Data%20Samples/DAILY%20TAQ/)
26284
(e.g. `EQY_US_ALL_NBBO_20220705.gz`). We recommend using the first 10M lines for benchmarking purposes. The TAQ file path should be passed to the benchmark script
27285
through the `DATA_TAQ` environment variable.

‎bench/codon/npbench.codon‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -553,7 +553,7 @@ def syrk(M, N, datatype=np.float64):
553553
A = np.fromfunction(lambda i, j: ((i * j + 1) % N) / N, (N, M),
554554
dtype=datatype)
555555
return alpha, beta, C, A
556-
run(bench.syr2k, syr2k, M=1000, N=1200)
556+
run(bench.syrk, syrk, M=1000, N=1200)
557557

558558

559559
def trisolv(N, datatype=np.float64):

‎bench/codon/set_partition.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ template <class T> using vec = std::vector<T>;
88

99
inline vec<int> range(int start, int stop) {
1010
vec<int> v(stop - start);
11-
uint j = 0;
11+
unsigned int j = 0;
1212
for (int i = start; i < stop; i++)
1313
v[j++] = i;
1414
return v;

‎bench/codon/taq.cpp‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,11 @@ template <typename It> double stdev(It begin, It end) {
2020
auto n = std::distance(begin, end);
2121
double sum = std::accumulate(begin, end, 0.0);
2222
double mean = sum / n;
23-
double sq_sum = std::inner_product(begin, end, begin, 0.0);
24-
double stdev = std::sqrt(sq_sum / n - mean * mean);
23+
double sq_sum = std::accumulate(begin, end, 0.0, [mean](double total, double value) {
24+
double difference = value - mean;
25+
return total + difference * difference;
26+
});
27+
double stdev = std::sqrt(sq_sum / (n - 1));
2528
return stdev;
2629
}
2730

@@ -43,7 +46,7 @@ std::vector<int> find_peaks(const std::vector<double> &y) {
4346
std::vector<double> avg_filter(t);
4447
std::vector<double> std_filter(t);
4548
avg_filter[lag] = mean(y.begin(), y.begin() + lag);
46-
avg_filter[lag] = stdev(y.begin(), y.begin() + lag);
49+
std_filter[lag] = stdev(y.begin(), y.begin() + lag);
4750

4851
for (int i = lag; i < t; i++) {
4952
if (std::abs(y[i] - avg_filter[i - 1]) > threshold * std_filter[i - 1]) {

0 commit comments

Comments
 (0)