Opened 7 years ago

Closed 16 months ago

Last modified 16 months ago

#7414 closed feature request (fixed)

plugins always trigger recompilation

Reported by: jwlato Owned by:
Priority: high Milestone: 8.6.1
Component: Compiler Version: 7.6.1
Keywords: plugin, RecompilationCheck, GHCProposal Cc: conrad@…, conal@…, osa1
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: #12567 Differential Rev(s): Phab:D4366
Wiki Page:

Description

When compiling code with a ghc plugin, e.g. ghc -O -fplugin SomePlugin Main.hs, recompilation is triggered for every module. This can make plugins difficult to use in environments with many modules shared between executables.

Since a plugin is a GHC library, I would like to propose that plugin arguments and interface hashes be included in the dependencies for compiled modules, in a similar way to how compiler flags are already included. This would enable ghc to avoid recompilation of modules when using plugins, provided that the plugin and plugin options haven't changed.

Change History (19)

comment:1 Changed 7 years ago by conrad

Cc: conrad@… added

comment:2 Changed 6 years ago by igloo

difficulty: Unknown
Milestone: 7.8.1

We might also want the plugin to indicate whether it's returned a deterministic result or not.

comment:3 Changed 5 years ago by thoughtpolice

Milestone: 7.8.37.10.1

Moving to 7.10.1

comment:4 Changed 5 years ago by thoughtpolice

Milestone: 7.10.17.12.1

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:5 Changed 4 years ago by thoughtpolice

Milestone: 7.12.18.0.1

Milestone renamed

comment:6 Changed 4 years ago by ezyang

Plugin interface hash is insufficient, since you want to recompile if the plugin implementation changes (even if its unfoldings do not.) It's almost as if you just want the file modification time (but that would be nondeterministic)...

Last edited 4 years ago by ezyang (previous) (diff)

comment:7 Changed 4 years ago by ezyang

Related #7277, which has the opposite problem where rely only on the interface hash (and not an "implementation" hash.)

comment:8 Changed 4 years ago by thomie

Milestone: 8.0.1

comment:9 Changed 4 years ago by conal

Cc: conal@… added

comment:10 Changed 2 years ago by bgamari

Keywords: RecompilationCheck added; recompilation removed

comment:11 Changed 20 months ago by mpickering

Milestone: 8.6.1
Priority: normalhigh

Is there anything blocking this ticket or does someone just need to implement it?

It makes using plugins quite a bit more hassle than necessary. See also #12567

I really think this should be a priority for the next release (8.6)

Ben said in an email thread last year that these should be the steps. But, if the implementation is slightly broken, implementing anything will be better than the current situation I posit! We can get back to the current situation by using -fforce-recomp after all.

I think the real question is what sort of interface do plugin authors
need?

I suspect there are a few distinct tasks here,

 * compute and record module implementation hashes in interface files

 * to include plugin implementation hashes in the recompilation check

 * to provide an interface allowing a plugin to compute a hash of its
   arguments which can be included into the recompilation check. One way
   of realising this would be to add a field like the following to Plugin,

       pluginHash :: [CommandLineOption] -> Maybe Fingerprint
           -- Nothing would denote "always rebuild"

   Would this help in your case?

This would allow us to fix the TH problem in #7277 and fix the plugins
problem in #7414 and #12567 in a nearly optimal way (assuming the plugin
author is able to precisely define a hash).

None of this is terribly difficult and given Nick's recent work on his
row types plugin, it seems like it's getting more urgent.

Cheers,

- Ben

comment:12 Changed 20 months ago by bgamari

Is there anything blocking this ticket or does someone just need to implement it?

There's nothing particularly blocking. The plan that I wrote about may or may not be the right way to go about this. It's also quite unclear what the right way forward would be if it turned out that implementation hashes are expensive to compute.

Regardless, osa1 may be looking into this for Well-Typed client at some point in the next few months.

comment:13 Changed 20 months ago by osa1

Cc: osa1 added

comment:14 Changed 20 months ago by bgamari

mpickering has written a proposal describing one way to fix this for core-to-core plugins.

Last edited 20 months ago by bgamari (previous) (diff)

comment:16 Changed 17 months ago by bgamari

Differential Rev(s): Phab:D4366
Status: newpatch

comment:17 Changed 16 months ago by Ben Gamari <ben@…>

In 1d1e2b7/ghc:

Implement "An API for deciding whether plugins should cause recompilation"

This patch implements the API proposed as pull request #108 for plugin
authors to influence the recompilation checker.

It adds a new field to a plugin which computes a `FingerPrint`. This is
recorded in interface files and if it changes then we recompile the
module. There are also helper functions such as `purePlugin` and
`impurePlugin` for constructing plugins which have simple recompilation
semantics but in general, an author can compute a hash as they wish.

Fixes #12567 and #7414

https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/002
2-plugin-recompilation.rst

Reviewers: bgamari, ggreif

Reviewed By: bgamari

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #7414, #12567

Differential Revision: https://phabricator.haskell.org/D4366

comment:18 Changed 16 months ago by bgamari

Resolution: fixed
Status: patchclosed

comment:19 Changed 16 months ago by RyanGlScott

Keywords: GHCProposal added
Note: See TracTickets for help on using tickets.