# Nodus / Modal L4 pilot: offline analysis package

Analysis package version: 1.0.0, published October 14, 2026.
Measurements: October 10–11, 2026, published by Nodus Compute.

Use this package to inspect the published evidence and recalculate its costs and
quality summaries. It runs locally with Python 3.10 or newer and its standard
library. It does not connect to either platform, download models, or spend money.

Download `benchmark-analysis.zip`, extract it, and run `python3 verify.py` in the
extracted directory. To regenerate the spreadsheet worksheet, run
`python3 verify.py --csv my-cost-worksheet.csv`. You can also inspect the Python
source and JSON records before running anything.

## What is included

- `results.json`: sanitized measurements, all eight paid attempts, timing
  definitions, exported-file digests and accounting boundaries.
- `method.json`: pinned model and dataset revisions, data selection, training
  arguments, recorded library versions, evaluation gates and protocol deviations.
- Four SVG charts from the original articles.
- `SHA256SUMS.txt`: the original checksums for those six evidence files.
- `verify.py`: checks those checksums, recalculates the campaign and per-run
  charges, validates evaluation-summary arithmetic, and compares reported weight
  digests. Successful execution exits with status 0; an inconsistency exits 1.
- `cost-worksheet.csv`: generated from the same records by `verify.py`.

The original evidence records and their checksum manifest are unchanged. The
website's `analysis-SHA256SUMS.txt` additionally covers the analysis script,
README, generated worksheet and ZIP. Checksums detect changed bytes relative to
the manifest; publisher-generated hashes are not an independent audit.

## Experimental scope

The pilot used Qwen/Qwen2.5-0.5B, one NVIDIA L4 per job, seed 11, and one accepted
run per task per platform. Fine-tuning used SST-2; continued pretraining used
WikiText-2 raw. Both started independently from the pinned base model. See
`method.json` for the exact revisions and serialized arguments.

The worksheet has three comparisons: successful fine-tuning attempts, fine-tuning
including Nodus's failed wrapper attempt, and successful continued-pretraining
attempts. They yield approximately 67.0%, 24.7%, and 42.5% lower recorded execution
charges, respectively. These are gross usage charges before credits, not final
cash invoices, recurring prices or forecasts for other models.

The retry-inclusive row leaves timing columns empty: the published records do
not establish the combined time to delivery across both attempts. Each platform's
campaign subtotal includes different diagnostic work; do not compare those totals
as platform savings. No failure rate or confidence interval can be inferred.

## What this package cannot reproduce

This is an offline analysis of published records, not the original training
harness. It does not rerun training or evaluation, authenticate provider receipts,
or hash actual model weights. Matching published digest strings do not independently
establish that the underlying files were identical.

The package does not contain the native execution worker, complete runnable
environments, trained weights, raw invoices or private source records. Digests of
private records identify them but do not make them available to a reader.

Effective native CPU limits, cache warmth and startup conditions were not fully
matched. Delivery clocks differ: Nodus waited for billing finalization before
export, while Modal usage was collected separately. Residual storage/control-plane
charges are not asserted finalized. Larger models, repeated runs, multi-GPU
workloads and independent replication need new measurements.

## Read the reports

- https://www.nodus-compute.ai/blog/nodus-vs-modal-fine-tuning-benchmark/
- https://www.nodus-compute.ai/blog/nodus-vs-modal-continued-pretraining-benchmark/

When citing this package, identify Nodus Compute as the measurement owner and
include the model, GPU, October 2026 dates, sample size, comparison row and charge
scope. Keep the recorded measurements separate from independent validation.