SRA Lite benchmarks

This project compares analysis results and resource usage when the same SRA accessions are supplied in normalized SRA and SRA Lite formats. RNA-Seq is the currently implemented analysis. FASTQ generated from normalized SRA can also be processed as a baseline.

Each input format is run through the same alignment, feature-counting, and QC steps. The resulting gene and exon counts, alignment summaries, RNA quality metrics, and runtime measurements can then be compared across formats.

• pipelines/ contains the RNA-Seq runners, step definitions, runtime configuration, helper scripts, and detailed execution documentation.
• input/ contains accession manifests and scripts for staging or splitting benchmark inputs.
• reference/ contains scripts used to construct and validate the mouse GRCm39 reference set.
• build/ contains Dockerfiles for the RNA-Seq pipeline and reference-set builder.
• cloud/ contains GCP VM setup and host-level monitoring scripts.
• build.sh builds a selected project Docker image.

The work/ and dock/ directories may be created locally for pipeline outputs and transient downloaded inputs; they are not source directories.

The RNA-Seq pipeline uses the mouse GRCm39 reference assembly and associated RefSeq and Ensembl annotations. The reference builder downloads the source files, prepares gene and exon annotations, creates known splice-site and QC files, masks the pseudoautosomal region, and builds the HISAT2 index.

Build the reference-builder image from the repository root:

docker build \
  -t sralite-reference \
  -f build/Dockerfile.reference \
  .

Create an output directory and run the builder:

mkdir -p reference-data

docker run --rm \
  -v $PWD/reference-data:/input \
  sralite-reference \
  --work /input

The build downloads data from NCBI and Ensembl and therefore requires network access. HISAT2 index construction requires approximately 150 GB of free memory (refer to hisat2 manual on --ss option). The completed directory is mounted as the pipeline reference root; in the RNA-Seq container it is normally available below /static.

The setup script can also be run directly when all dependencies are already installed:

./reference/setup-ref-data.sh --work ./reference-data

Build the local RNA-Seq image with:

./build.sh --pipeline rnaseq --version v0.0.2 --test

The image is tagged rnaseq. The build includes SRA Toolkit, HISAT2, samtools, Subread featureCounts, RSeQC, and the pipeline scripts. The --test option stops after the local image is built; registry publication is not part of this command.

build.sh reads Git metadata and the configured GCP project. Docker, Git, and the Google Cloud CLI must therefore be available in the build environment.

The RNA-Seq pipeline supports three execution modes:

• directly on a local host, using locally installed tools and a staged input;
• locally in Docker, using the project image and mounted input, work, and reference directories;
• on a Google Cloud VM, where the batch wrapper can process a set of accessions, collect container and cgroup metrics, and upload results to Google Cloud Storage.

All modes use the same RNA-Seq step definitions and produce the same primary analysis outputs. The runners differ mainly in how they stage input, provide dependencies and reference data, and collect operational metrics.

See pipelines/README.md for local, remote-input, Docker, and GCP batch commands, along with input formats, checkpoints, outputs, and performance metrics.