Repository navigation
Inconsistent host annotation for ON022733.1 in datasets CLI and NCBI website #595
Description
Before opening an issue, please:
- Make sure you are using the latest version using
datasets --version - Review our documentation
Describe the bug
When ingesting NCBI data to mirror on Pathoplexus, we noticed a difference between the data produced by the datasets CLI versus the NCBI website.
Specifically, we noticed that accession ON022733.1 has a annotation for host (Callithrix) on the website, which is not present in the datasets download output. In the past (2026-03-12), the host was included in the datasets download output. See here for a comparison of an older version of the ON022733 accession we ingested in the past, versus what we received today: https://pathoplexus.org/seq/PP_005MRGX.2/versions?compare=1%2C2
In parallel, we've seen quite a few updates to host information for other sequences we ingest from NCBI (would be happy to provide specifics if you like).
Could this be due to a bug in the datasets CLI? Or does it maybe mean there has been an update to the data and it just takes a little while for the website and the datasets outputs to sync?
Indicate what operating system you're using
macOS
To Reproduce
Steps to reproduce the behavior:
datasets download virus genome taxon 3046277
unzip ncbi_dataset.zip
# when I look at this output, there is no `host` annotated
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl
# (or with jq for formatting, if installed)
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl | jq '.'Compare to:
- Go to https://www.ncbi.nlm.nih.gov/nuccore/ON022733.1
- Search for 'host'
- See
/host="Callithrix"
Expected behavior
I would expect both sources of information to be consistent, i.e., for both the datasets output and the website have host: Callithrix or neither of them to have it.
Activity
corneliusroemer commentedon Jul 28, 2026
There seems to have been a lot of sequences across multiple viruses that had their host taxon id field change recently in
datasets downloadoutput. On the order of 400 West Nile sequences, 30 Yellow Fever, 10 Dengue.There are 2 types of changes in datasets output:
/host="Callithrix", this is also what datasets used to output in the past as ncbitaxon:9481 but now the host tax id field is empty for this sequence in datasets download).We (at Pathoplexus) track all changes to
datasets downloadrecords over time which is how we noticed: https://pathoplexus.org/seq/PP_005MH3X.2/versions?compare=1%2C2My speculation is that something in NCBI Virus/datasets processing pipeline (or its inputs) changed how host taxon is parsed from Genbank file. The cases where there are updates look correct (case 1 above, where the new taxon id matches the host string of genbank file). The cases where taxonid gets lost altogether look like bugs in processing.
ericcox1 commentedon Jul 28, 2026
Hi @maverbiest @corneliusroemer,
Thank you for opening this issue and for the detailed information. We are working with the NCBI Virus team on this and I will comment on this thread with any updates.
Best,
Eric
corneliusroemer commentedon Jul 29, 2026
Thanks @ericcox1!
I've looked into this a bit more and there are more examples of hosts dropped from NCBI Virus output that actually do have Genbank host strings. These are all the ones that we noticed at Pathoplexus which parses only a subset of all virus sequences:
corneliusroemer commentedon Jul 29, 2026
There's another interesting case I've just noticed. There are 2 sequences that have hosts annotated by NCBI Virus with a taxonomy id that doesn't actually resolve at NCBI taxonomy:
OX442276.1 has tax id 3507587 which NCBI virus says is Buteo sp. but the tax hierarchy doesn't exist within the datasets output - so datasets also can't resolve that tax id.
Same with: PP938641.1 Ochlerotatus argyrothorax 3230309
I wonder if those tax ids were deleted, mismatched by NCBI virus, or are new and taxonomy just doesn't have them yet.
It seems Buteo sp. was added at some point and then deleted again - looking at a few taxonomy dump times:
https://ftp.ncbi.nih.gov/pub/taxonomy/taxdump_archive/Buteo was added and deleted in Dec 2025. Ochlerotatus argyrothorax was added in Jan 2024 and deleted in Jul 2024.
ericcox1 commentedon Jul 31, 2026
@maverbiest @corneliusroemer,
We have determined that this is due to a recent change on our end to how we assign host taxonomy IDs, which is causing some previously-mapped hosts (like "Callithrix") to no longer get assigned. We’re working on a fix to restore the expected mappings for common cases like this one.
For the other examples from @corneliusroemer, including OX442276.1, we think there's a different bug and we are looking into that too.
-Eric