Repository navigation
Manage a global package environment for easy GHCi access #3737
Description
I think, essentially, the request is for cabal new-install to put the requested package into some global packages set, which we keep up-to-date upon invocations to new-install. Maybe we should even write the package environment in the place GHC looks by default, so a bare ghci invocation works.
@hvr thought extra-packages might be what you want (if it were implemented) but I'm not sure; I think it would cause any such mentioned packages to be considered local, and thus built inplace. But maybe that's a misinterpretation.
CC @cocreature
Activity
cocreature commentedon Aug 30, 2016
Is there any documentation on
extra-packages? Having them treated as local packages doesn’t sound like what I want.Personally I like viewing the global package set as just another project that is chosen if no other project is found. That removes the need for separate commands in the UI. Thinking about it at least the name
extra-packagessounds like what I want, so just a list of packages that should be available in a project without them being declared as a dependency in a cabal file. For packagescabal new-installcould just add the package to that list automatically and build it. If I am in a project it gets installed there or if I am somewhere else it gets installed into the global project.ezyang commentedon Aug 30, 2016
There is no documentation on
extra-packages; but to its credit, they are also not implemented (only the syntax bits are in.) It's hard to say what the semantics were supposed to be... @dcoutts ? I do think your interpretation makes sense.dcoutts commentedon Aug 31, 2016
The motivation for
extra-packagesis things likeghc-vis, libraries that you want to have available in your development environment to use in ghci, but that is not a dependency of any package in your project.So
extra-packagesare to be added into the project environment, solved for etc. They can come from the store or be built locally following the usual rule. They're additional to the ones automatically pulled in by deps of local packages. So where this really shows up is that you can use them as a target of build (e.g. if it's not built in the store already) and that they should go into the.ghc.environmentfile.If the
extra-packagesis a exe then I guess it means (apart from building etc) just make it available in any managed shell environment or managed bin dir we might do.hvr commentedon Aug 31, 2016
For consistency with the per-components framework, I'd expect
extra-packagesto build only library components (although maybe it should be renamed toextra-libs), and for when exe components are to be requested there should be a separateextra-toolsdirective (c.f. #3708)cocreature commentedon Sep 2, 2016
Reading @dcoutts’ description, extra-packages in combination with a global project that all commands default to if there is no cabal.project (or cabal file) seems like it would be exactly what I’m looking for.
23Skidoo commentedon Sep 2, 2016
@dcoutts Do we plan to make
cabal new-install fooupdateextra-packagesin the global project?dcoutts commentedon Sep 8, 2016
@23Skidoo I'm not sure I'd totally nailed down what global install of libs should mean, but if we can make it behave more or less like a project then that would be easier to implement and might be a good UI as it'd follow the same mental model. So if we can say that the interpretation of install of libs user-wide is to update a virtual or real user-wide project with
extra-packagesand rebuild then great. But note that this implies full consistency, as with projects, and possibly rebuilding other packages that were previously installed to make them consistent (though at least we can do atomic switchovers so we don't bork things if rebuilding fails).dcoutts commentedon Sep 8, 2016
Oh and then note that this is not at all as simple as what Simon Marlow was suggesting in that same discussion which was to just blindly register the lib in the user db ("I know what I'm doing dammit!").
So I'd suggest that we proceed by implementing
extra-packagesfor local projects and in parallel think about whether it fully works UI-wise to have the user-wide environment (ghc/ghci libs and exes) managed like a local project.First thing to note there is that you don't want full consistency for deps of exes. I should be able to install an exe user-wide even if it doesn't have the same consistent deps as other user-wide binaries. So perhaps we should say that libraries available user-wide are managed like a project, but exes are done differently (or equivalently they're solved for independently aka
--independent-goalsbut--independent-exe-goalsonly).ezyang commentedon Sep 8, 2016
You are right that there is a tension between (1) minimizing build time (if I say cabal new-install, I want the package to install ASAP) and (2) keeping a consistent database. Old cabal install always tried to minimize build time, by trying to use preexisting packages and being willing to install inconsistent things.
You might think that if you are willing to wait, it would always be right to keep the database consistent. But this might not be possible if a user wants to add two separate packages to their environment which don't have compatible dependencies. So inconsistency can be useful, even if it is not completely right. So, we could try the "global project" model, but it definitely sounds like letting each of the library roots be an independent goal would let more plans go through. Fortunately, I don't see why
extra-packagescouldn't be setup to allow independent goals, so you still ought to be able to use the project codepath. But maybe you want to warn when some of the dependencies are inconsistent!simonmar commentedon Oct 8, 2016
I'd like to suggest something really really simple: have
cabal new-installfor a library build and install the library into the store.Additionally it would be nice to have a way to do
ghci -package-db ~/.cabal/store/ghc-8.0.1/package.dbvia cabal, but I can always do that myself.I'm sure you're thinking "but why would you do that? You'll get a random collection of packages". My point is that usually it does the right thing, and when it doesn't do the right thing I can give it a hint by adding a
-packageflag. I don't want to mess with global state or a global package environment or environment files or anything - just let me build a package and use it.hvr commentedon Oct 8, 2016
@simonmar You don't get a random collection of packages, but rather a forrest of install-plans into scope. But your suggestion would work fine, if you care only about a single package. However, if quickly testing something in GHCI is your interest, there's a related feature #3843 which has a similiar problem. So what about having a convenient command:
cabal ghci --transitive-deps <list-of-deps>which throws you into a GHCi session with those packages in scope (possibly building them first), as well as their transitive deps (but in a consistent plan)?EDIT: Just to clarify, I think the
cabal new-install <library-package>suggestion is reasonable in its own right (I currently emulate it by creating a dummy package.cabalfile, which contains abuild-dependswith the libraries I want to get populated into the nix store); my comment is rather about the 2nd part withghci-invocation sugar.ezyang commentedon Oct 8, 2016
Well, I don't see any reason why we can't go ahead and implement @simonmar's suggestion. I don't think it will be that useful, but it would be pretty easy to do.
phadej commentedon Oct 9, 2016
I realised on the plane: It would be useful, to tell cabal to only use packages in the store, or solver to consider only versions with tars cached. I had to reinvent constraints manually to make offline work possible :) in that light
new-installwill be useful to populate caches23Skidoo commentedon Oct 9, 2016
@phadej Do you know that
cabal installhas an--offlineflag?phadej commentedon Oct 9, 2016
@23Skidoo but it doesn't work with
new-build:(ezyang commentedon Oct 10, 2016
Yes, offline mode would be great. But it does make the process less deterministic (because the solver has to prefer versions for which we have local tarballs.)
tom-bop commentedon Oct 25, 2017
For people who don't think this feature will be useful, can I clarify?: it's my understanding that, without using "old-install", the
ghcicommand becomes effectively useless, except to toy with functions from the Prelude. Is that right?fgaz commentedon Oct 27, 2017
@tom-bop
Well, except that now there are ghc environment files. So if you invoke
ghciin a project directory it will include the libraries your packages depend on.new-installfor libraries will basically be a global env file plus #4825 plussomea lot of consistency checkingfgaz commentedon Oct 27, 2017
@simonmar #4825 does what you want
tom-bop commentedon Oct 27, 2017
@fgaz #4825 is awesome, but is for nonlocal executables, and @simonmar and I are talking about nonlocal libraries. (Unless you're saying that #4825 additionally installs packages in the global store?)
fgaz commentedon Oct 27, 2017
@tom-bop Yeah, it does. It installs the selected package in the store and symlinks its exes, if there are any. So you can use it to add libraries to the store too. You'll have to manually create a ghc environment file or explicitly pass -package* flags to ghci though.
hvr commentedon Oct 27, 2017
@tom-bop @fgaz @simonmar you may want to take a look at https://gist.github.com/a076fbd34083a44faecd45ffd12b9772 which is a shell script I've been using to create package environment files (which exposes all transitive deps for the dependency spec specified at creation). It requires
cabal2.1 and GHC 8.0.2 (or later).fgaz commentedon Dec 15, 2020
@phadej duplicate of #6481?
phadej commentedon Dec 15, 2020
@fgaz, if #6481 is accepted as "the way we want to proceed" then yes, otherwise no. So until we know what we want, I wouldn't close this.