Opened 14 years ago

Closed 8 years ago

Last modified 2 years ago

#481 closed bug (fixed)

Recompilation check fails for TH

Reported by: simonpj Owned by: simonmar
Priority: normal Milestone: 7.2.1
Component: Template Haskell Version: 6.4.1
Keywords: Cc: jonas.duregard@…, saurabhnanda, nh2
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Incorrect result at runtime Test Case: TH_recompile
Blocked By: Blocking:
Related Tickets: #14769 Differential Rev(s):
Wiki Page:

Description (last modified by igloo)

The recompilation check only recompiles a module when the interface of a module it imports changes. But with Template Haskell, it may need to be recompiled when the implementation changes.

Concrete example below. It's quite awkward to fix.

  • Perhaps a module that contains any splices should be recompiled always.
  • Perhaps a module that exports any TH stuff (how would we tell?) should be flagged as changed if anything about it changes.


The following scenario reproduces this error (thanks to Bulat Ziganshin bulatz@…):

1) create Main.hs containing code

module Main where
import Sub
main = print $x

and Sub.hs containing code

module Sub where
x = [| 1 |]

2) compile them with --make:

C:\!\Haskell\!>ghc --make -fth Main.hs
Chasing modules from: Main.hs
Compiling Sub              ( ./Sub.hs, ./Sub.o )
Compiling Main             ( Main.hs, Main.o )
Loading package base-1.0 ... linking ... done.
Loading package haskell98-1.0 ... linking ... done.
Loading package template-haskell-1.0 ... linking ... 
Linking ...


3) now change Sub.hs to the following code:

module Sub where
x = [| 2 |]

4) and recompile program:

C:\!\Haskell\!>ghc --make -fth Main.hs
Chasing modules from: Main.hs
Compiling Sub              ( ./Sub.hs, ./Sub.o )
Skipping  Main             ( Main.hs, Main.o )
Linking ...


As you see, Main.hs is not recompiled despite the fact that definition of x is changed and now program must print "2"

Change History (21)

comment:1 Changed 14 years ago by simonmar

Architecture: Unknown
Description: modified (diff)
difficulty: Unknown
Operating System: Unknown
Version: None6.4.1

comment:2 Changed 13 years ago by igloo

Description: modified (diff)
Milestone: _|_
Test Case: TH_recompile

comment:3 Changed 11 years ago by simonmar

Architecture: UnknownUnknown/Multiple

comment:4 Changed 11 years ago by simonmar

Operating System: UnknownUnknown/Multiple

comment:5 Changed 10 years ago by simonmar

Type of failure: Incorrect result at runtime

comment:6 Changed 10 years ago by igloo

Description: modified (diff)

comment:7 Changed 9 years ago by simonpj

Cc: jonas.duregard@… added

See also

Jonas says "This has caused headaches for me on several occasions lately". I wonder how many other people have experienced such headaches.


comment:8 Changed 9 years ago by simonmar

I think we should just disable the recompilation check for modules that use TH or annotations. The compilation manager will still do the basic check based on modification times of object files, so if nothing has changed below the TH module we won't recompile it, but if anything has changed we will.

comment:9 in reply to:  8 Changed 9 years ago by JonasDuregard

Replying to simonmar:

I think we should just disable the recompilation check for modules that use TH or annotations.

I agree. The problem now is that the recompilation checker gives a false negative, a few false positives is not a big issue in comparison.

Here is an alternative solution, if I have understood the mechanisms involved correctly:

  • We flag modules as one of three three change levels: unchanged, implementation-changed and interface-changed. This should be simple.
  • We mark imports as compile time and/or run time depending on if imported functions are used in compile time code. I guess this might be a bit more tricky.
  • We recompile if any import is interface-changed or if a compile time import is implementation-changed.

As a bonus, marking imports is a step towards an optimization which I'm not sure is currently implemented: if an import is marked TH only, then we can drop it after evaluating compile time code (this only works if the generated code does not use functions from the imported module).

I have intended to get hacking on GHC for some time, so maybe looking into this alternative is a good start.


comment:10 Changed 9 years ago by simonpj

Milestone: _|_7.2.1
Owner: changed from simonpj to simonmar
Priority: lownormal

Simon's going to implement the simplest correct thing.

comment:11 Changed 9 years ago by marlowsd@…

commit 48bc81ad466edfc80237015dbe5d78ba70eb5095

Author: Simon Marlow <>
Date:   Wed Jul 20 09:37:54 2011 +0100

    Fix #481: use a safe recompilation check when Template Haskell is
    being used.
    We now track whether a module used any TH splices in the ModIface (and
    at compile time in the TcGblEnv and ModGuts).  If a module used TH
    splices last time it was compiled, then we ignore the results of the
    normal recompilation check and recompile anyway, *unless* the module
    is "stable" - that is, none of its dependencies (direct or indirect)
    have changed.  The stability test is pretty important - otherwise ghc
    --make would always recompile TH modules even if nothing at all had
    changed, but it does require some extra plumbing to get this
    information from GhcMake into HscMain.
    test in driver/recomp009

 compiler/deSugar/Desugar.lhs      |   10 +++-
 compiler/iface/BinIface.hs        |   14 ++++--
 compiler/iface/LoadIface.lhs      |    1 +
 compiler/iface/MkIface.lhs        |   66 +++++++++++++++------------
 compiler/main/DriverPipeline.hs   |   24 ++++++----
 compiler/main/GHC.hs              |    5 ++
 compiler/main/GhcMake.hs          |   23 +++++----
 compiler/main/HscMain.lhs         |   92 +++++++++++++++++++++++++------------
 compiler/main/HscTypes.lhs        |   33 ++++++++++++-
 compiler/typecheck/TcRnDriver.lhs |    5 +-
 compiler/typecheck/TcRnMonad.lhs  |    9 +++-
 compiler/typecheck/TcRnTypes.lhs  |    5 ++
 compiler/typecheck/TcSplice.lhs   |    2 +
 13 files changed, 196 insertions(+), 93 deletions(-)

comment:12 Changed 9 years ago by simonmar

Status: newmerge

comment:13 Changed 8 years ago by igloo

Resolution: Nonefixed
Status: mergeclosed

comment:14 Changed 6 years ago by Ian Lynagh <igloo@…>

In 27d0701d92b71c29f5260998d7579e60611ef87d/ghc:

Add test TH_recompile for trac #481 (Recompilation check fails for TH)

comment:15 Changed 5 years ago by Edward Z. Yang <ezyang@…>

In af4d99803ea7676f88f250ad56a8c31c1c8cd5bc/ghc:

Don't do a half-hearted recompilation check in compileOne

The isNothing maybe_old_linkable check predates
48bc81ad466edfc80237015dbe5d78ba70eb5095, which fixed #481 by requiring
recompilation information to be passed in as an argument to compileOne.
As a result, the check here is redundant: the client has already taken
a look at the object file to see if it is available or not.

Signed-off-by: Edward Z. Yang <>

Test Plan: validate

Reviewers: simonmar, austin

Subscribers: carter, thomie

Differential Revision:

comment:16 Changed 2 years ago by saurabhnanda

Cc: saurabhnanda added

Just wondering what would be the perf-impact of doing the extra "stability check" (as mentioned in )? A lot of the code that we write uses TH heavily (eg. deriving ToJSON/FromJSON instances, test-suites, deriving lenses, etc) and I was wondering if this particular issue is the reason for generally slow compile times. (Not that I have isolated this to be the reason -- just asking.)

We have 1,200+ modules in our code-base and we use TH so heavily that it's enabled by default (via the Cabal file).

comment:17 Changed 2 years ago by simonmar

We already do that stability check. But recompilation of TH could be improved if we had a better idea of what the TH code actually depended on - when it's arbitrary I/O, we have to be necessarily conservative and assume that anything could change. There's some design to do here to figure out what a better solution would look like.

comment:18 Changed 2 years ago by saurabhnanda

We already do that stability check.

My earlier comment was more about whether anyone has studied the perf-impact of the existing stability check.

comment:19 Changed 2 years ago by simonmar

I don't think there's any perf impact, the stability check is already being done. The problem you're most likely seeing is that the recompilation check for TH is more conservative than we need in many cases, because the compiler doesn't know what your TH code depends on so it has to assume the worst.

comment:20 Changed 2 years ago by nh2

Cc: nh2 added

comment:21 Changed 2 years ago by nh2

I believe I have found a bug with the current implementation of the [TH]/RecompBecause "TH" check: It doesn't work when the build is interrupted. See #14769

Note: See TracTickets for help on using tickets.