Opened 4 years ago

Closed 4 years ago

#11244 closed bug (fixed)

Compiler plugins should not have visibility controlled by -package

Reported by: ezyang Owned by: ezyang
Priority: normal Milestone: 8.0.1
Component: Compiler Version: 7.11
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D1661
Wiki Page:

Description (last modified by ezyang)

The current way compiler plugins are recommended to be specified is by putting it in a package and then referring to the exported module name in the -fplugin flag. Consequently, the recommended way to use a plugin from Cabal is to put it in build-depends. For example, ghc-typelits-natnormalise is used by clash-prelude, whose build-depends depends on it explicitly.

There are numerous downsides to operating in this manner:

  1. Cabal will always link in any library which is build-depended on, even if there is no runtime dependency on the package in question. For example, suppose the default cabal-init package with a build-depends: ghc and doesn't use it, you get:
    ezyang@sabre:~/Dev/labs/link$ cat Main.hs
    module Main where
    
    main :: IO ()
    main = putStrLn "Hello, Haskell!"
    ezyang@sabre:~/Dev/labs/link$ cabal configure --enable-executable-dynamic; cabal build
    # elided
    ezyang@sabre:~/Dev/labs/link$ ldd dist/build/link/link | grep libHSghc
    	libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so => /srv/code/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc_JzwEp1oQ8kA7NFNTGk1ho5/libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so (0x00007f591f47a000)
    
    That's terrible. And it's inflicted on every package which depends on a library that uses the plugin. The situation is better with static linking, since the linker can drop unreferenced object files; but we shouldn't be passing it to the linker in the first place.
  2. It should be possible use a compiler plugin with a cross-compiler (e.g. GHCJS, etc.); unlike Template Haskell, the plugin must be written with the host compiler in mind. But the package database contains libraries compiled in the target language; whereas a compiler plugin must be compiled in the host language.

It seems clear that using -package to bring a plugin into scope is highly undesirable. So what should we do? The necessary information to load a plugin is:

  1. The package DB entry for the package it lives in (and information about all its transitive dependencies, because we may need to load code from them as well),
  2. The actual module which contains the plugin.

This leaves us with a number of options on the GHC end:

  1. We could have a explicitly, IPID qualified method of specifying plugins. So, instead of saying -fplugin ModuleName and being at the mercy of what is exposed, you could say -fplugin foo-0.1-xxxx:ModuleName and as long as that package is usable (not ignored/broken) it would get loaded.
  1. We could have a *completely separate* package database for plugins. You would control it using -plugin-package rather than -package; you don't even have to use the default system/user databases (which means that it might be possible to support a cross-compiler, although the details are fuzzy in my head). A benefit of this approach is that you can support the -fplugin MyModule mode of use, and have Cabal feed in the extra plugin package arguments to give meaning to this expression.

An alternate command line interface (suggested by SPJ) would be that to load a plugin, you specify a path to the package database, package, and module in question, something like -fplugin /path/to/package.conf/foo-0.1-xxxx.conf:MyPlugin (one difficulty with this approach is that you may need multiple package databases to get this to work.)

  1. We could use the same databases, but a *separate* package namespace. So I can say ghc -hide-all-packages -plugin-package-id foo-0.1-xxx; the -plugin-package-id makes it so that I can say -fplugin FooPlugin later in the command line options. This is desirable because it allows you to continue to say {-# OPTIONS -fplugin FooPlugin #-} in a Haskell file, turning on the plugin per-module, while letting Cabal configure where it points to.

There would also be necessary Cabal adjustments, to get people to not put their plugins in build-depends, but a different dependency section, and how to have Cabal feed the IPID of the plugin to GHC.

I'm happy to implement, but I'm looking for some consensus on what to do.

Change History (7)

comment:1 Changed 4 years ago by ezyang

(dcoutts writes on IRC that he prefers the explicit qualification approach, since Cabal is going to have to play ball anyway.)

comment:2 Changed 4 years ago by ezyang

Description: modified (diff)

comment:3 Changed 4 years ago by ezyang

Differential Rev(s): Phab:D1156

comment:4 Changed 4 years ago by ezyang

Differential Rev(s): Phab:D1156Phab:D1556

comment:5 Changed 4 years ago by ezyang

Differential Rev(s): Phab:D1556Phab:D1661

comment:6 Changed 4 years ago by Edward Z. Yang <ezyang@…>

In 1faf1fc/ghc:

Implement -hide-all-plugin-packages and -plugin-package(-id), fixing #11244

Summary:
The basic idea is that we have a new set of "exposed modules"
which are /only/ used for plugins, i.e. -fplugin Foo and
--frontend Foo.  You can interact with this namespace
using the flags -plugin-package-id and -plugin-package.
By default, this namespace contains all modules in the
user namespace (as before), but you can toggle that using
-hide-all-plugin-packages.

There is one nasty hack: GhcMake respects -fplugin in
GHC_OPTIONS to make local plugins work correctly.  It also
bails out of you have an import of a module which doesn't
exist locally or in the package database.  The upshot is
that we need to be sure to check in the plugin modules
too, so we don't give a spurious failure when a plugin
is in the plugin namespace but not the main namespace.
A better way to fix this would be to distinguish between
plugin and normal dependencies in ModSummary.

I cheated a little and tweaked a few existing plugins
tests to exercise the new code paths.

TODO: Documentation

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin, simonpj, duncan

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1661

GHC Trac Issues: #11244

comment:7 Changed 4 years ago by ezyang

Resolution: fixed
Status: newclosed

Needs Cabal support though!

Note: See TracTickets for help on using tickets.