Repository navigation
Import completion crawls too many files #845
Description
Activity
This isn't broken! Our import completion correctly traverses directories which do not contain an
__init__.py, since #792, because empty__init__.pyfiles are no longer required for namespace packages.-
Maybe we shouldn't crawl these at all. Apparently unittest doesn't crawl them, a sort of similar problem. At some point in the past, IPython skipped these too.
-
Maybe we should use different crawling logic for the current directory that bpython is run from than we do for other locations on
sys.path: namespace packages in other locations will still be picked up, but running bpython from a filesystem path which contains a massive node_modules directory or similar won't chew up CPU. Users who manually add e.g. their home directory to sys.path would still see this issue.
-
We could implement a (user-configurable) skip list which is checked before traversing into directories. We could make it default to
node_modules,.git, and maybe other well known directories which won't contain Python modules.Reacted by Tom BallingerI still see a lot of directories visited
$ python -m bpython.importcompletion | wc -l 63247(this is a script from #943)
from all the junk I have in my home directory so I'm thinking about this again. Same ideas as before: since running bpython from my home directory is very common for me and it's the
''entry in sys.path that causes all the trouble for me, maybe this entry could be treated differently; and maybe traversing non-init.py directories could be an option to turn off, or even off by default.Another IPython reference bug ipython/ipython#8875, they suggest Jedi for this.
- added a commit that references this issue
on Feb 25, 2026
@pyang30 as you mentioned in #806 I'm seeing bpython access a ton of files it shouldn't need to in order to initialize the import completion set. Like for you, it spends most of its time in node_modules folders.
There's something very broken here, we're crawling way too much.