Repository navigation
Writing fasterq-dump output as a streamable not currently possible #508
Description
Hi,
With fastq-dump in --split-3 mode, the output files can be pregenerated as named pipes, which makes usage more flexible e.g. this set of code from https://github.com/ababaian/serratus/blob/abc1d649d69899727458095c5da933b9b0f4cc1a/containers/serratus-dl/run_dl-sra.sh#L26
# Outputs created by fastq-dump
FQ_1="$SRA"_1.fastq
FQ_2="$SRA"_2.fastq
FQ_3="$SRA".fastq
# Prefetch the data before processing
# This should be VERY fast, and will cause fastq-dump to have
# smoother CPU usage in the end.
prefetch $SRA 1> /dev/null
# Create some named pipes for fastq-dump to put its data into.
mkfifo "$FQ_1" "$FQ_2" "$FQ_3"
fastq-dump -X $FQMAX --split-e $SRA & pid=$!
However, the same cannot be done for fasterq-dump. If tried, the fifo files get replaced by regular files. Is there some technical reason why replacing them is necessary?
Thanks, ben
Reacted by Rayan Chikhi
Activity
wraetz commentedon May 17, 2021
I have to look into this - to find out why.
Even if it works - the output-files are generated as the last step.
You will not see anything going on with the pipes - until the 'concat' step takes place.
wwood commentedon May 18, 2021
OK, sure. Thanks.
durbrow commentedon Apr 25, 2025
Have you tried the
-A|--appendoption forfasterq-dump?