Repository navigation
STAR .sam files chunks clog the work area #281
Description
Hi,
I's using egapx, but after short-read STAR alignment and sorting it leaves a lot of temporary files. This is due to the default behaviour - STAR produces SAM files.
The solution seems to be quite feasible - to change the default parameter --outSAMtype from "SAM" to "BAM Unsorted". Because samtools can work equally well with the SAM and BAM files, but it will save computational time on compression and IO disk operations
I'm currently testing this solution, but repeatedly running into the issues with argument parsing. I've tries single-quoting, double-quoting of the --outSAMtype value, but this is not working.
Maybe there is an obvious workaround I can't see?
Maybe you can also suggest the strategy for quick testing - currently I'm relaunching the full pipeline from the beginning - this takes a lot of time :(
I'm using these parameters from yaml, and the release 1.0.1
genome: /full/path/to/genome/EG_chr1.fa
taxid: 51953
trnascan:
enabled: false
cmsearch:
enabled: false
max_intron: 250000
tasks:
star_wnode:
star-params: >-
--alignSJoverhangMin 8
--outFilterMultimapNmax 200
--outFilterMismatchNmax 50
--genomeLoad NoSharedMemory
--outSAMtype BAM\ Unsorted
--outSAMprimaryFlag AllBestScore
--outFilterMultimapScoreRange 50
--seedSearchStartLmax 15
--limitOutSAMoneReadBytes 1000000
--outSJtype None
--runRNGseed 777
--outMultimapperOrder Old_2.4
star_wnode: -preserve-star-logs
short_reads:
- - DRR016807
- - /scratch/users/paniushevn/EG11_annotation_data/DRR016807_1.fastq.gz
- /scratch/users/paniushevn/EG11_annotation_data/DRR016807_2.fastq.gz
Activity
NickPanyushev commentedon Oct 23, 2026
Here is the full error message from the .out
murphyte commentedon Oct 23, 2026
This won't work.
star_wnodepipes the SAM output to a custom programcompact_samwhich applies some cleanup to the output to reduce artifact alignments spanning neighboring paralogs.compact_samthen applies its own selection for top hits (after removing the artifacts), and the SAM output is then converted to BAM. Changing--outSAMtypewill cause a failure atcompact_sam.Are you having files persist in
wrkarea?NickPanyushev commentedon Oct 23, 2026
Yes, i've just relaunched the pipeline with small files and got this
Last time the total volume of egapx_work grew up to !48 Tb! with small RNA-seq runs and blocked completely the disk. And sam chunks occupied quite a lot of space.
Currently I'm seeing in realtime how the occupied space is growing during the cmsearch. - it's already occupied 654Gb!!
This is the latest status of running egapx
So, the problem is not solely in sam chunks, I guess. But last time they occupied a significant fraction of the space
murphyte commentedon Oct 23, 2026
Are you running on a full cluster? How many nodes are you allowing in your config? How many RNA-seq runs are you aligning.
That'll relate to how many STAR runs can run in parallel, which will affect temp space usage.
NickPanyushev commentedon Oct 26, 2026
Yes, I'm running on the full SLURM cluster.
This is the process_resources.config used for the run
And for the test run I've used the small yaml with only 3 RNA-seq datasets. And used only 1 chromosome as a reference. The biggest of all three RNA-seq files is only 1.7 GB
And the growth of the space occupied started well before the actual STAR run even started! As I've withnessed this on Friday it happened when the cmsearch job started to run, but maybe it's just a coincidence.
Andy-B-123 commentedon Oct 26, 2026
Hi, just chiming in, I was trying to troubleshoot why I had some very long runs and I think I hit a similar thing (also on SLURM). In our case, this was mostly explained by large proportions of rRNA in some of the input RNA-Seq samples. Our assemblies contain rDNA tandem arrays on a few small contigs, and with --outFilterMultimapNmax 200 each rRNA read gets written many times. One 62 M pair library produced 1.26 billion alignments (half of them on three rDNA contigs totalling 0.26 Mb) and >1.4 TB of SAM in the work area. I tested the longest sample from an egapx run with and without filtering with sortmeRNA (table below). The time saving comes mostly from writing ~10x less SAM rather than from STAR mapping faster.
I came across this from another project where we had problems with rRNA in input samples. I compared filtering all ~10x input RNA-Seqs with sortmeRNA and providing to egapx, and this resulted in a ~0.5-1% decrease in BUSCOs, so I would not suggest that filtering all input RNA-Seqs to remove rRNA is appropriate, since some of them do appear to be contributing to 'real' genes. But I thought I'd mention it in case people have some very very long running jobs. Identifying input RNA-Seqs that are very high rRNA proportion (samtools idxstats on a run_star BAM shows it quickly) and gently filtering those might improve overall run times. Capping how many STAR tasks run at once (maxForks on run_star) also helped keep peak disk use down for us.
(Claude Code summary)
murphyte commentedon Oct 28, 2026
We're still discussing what we might do here, but our options are limited. According to Google's AI, even if we could use
--outSAMtype BAM(which would require converting back to SAM before processing throughcompact_sam), the chunk files written to tmp would still be SAM. STAR is supposed to delete the tmp directory on successful completion, but it'll get left behind with a failed task. The parameters we use in order to enable thecompact_samprocessing are likely also contributing. I'd love to use a different aligner, but we've found STAR gives better results than other options.The rRNA observation is interesting and not surprising, especially as rDNA copy number has increased in modern assemblies. 31.6% or reads classified as rRNA is impressive(ly high), assuming the run wasn't supposed to be total RNA. We have occasionally seen long run times and/or high memory consumption for some chainer jobs associated with rDNA, and at times we've filtered out the alignments before rnaseq_collapse. We haven't implemented that option into EGAPx because it introduces a dependency on running Rfam. But with that caveat, we could use Rfam to hardmask rDNA arrays before STAR indexing, reducing multicopy arrays down to a single copy. That would leave a sink to align the rRNA reads while reducing the multimapper rate, and not require pre-filtering the reads (with some risk of over-filtering, as you found). We'll need to collect some more data on how often rRNA is such a culprit before going down that path.
In the meantime, we'll add some documentation on how to reduce concurrent STAR jobs to help control tmp space usage.