Opened 10 years ago

Closed 8 years ago

Last modified 4 years ago

#3589 closed bug (fixed)

Recompilation checker doesn't take into account CPP headers

Reported by: simonmar Owned by:
Priority: normal Milestone: 7.4.1
Component: Compiler Version: 6.10.4
Keywords: Cc: ndmitchell@…
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case: driver/recomp011/recomp011
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

When using CPP, the recompilation checker doesn't check the date on any header files that might be #included. We already have to run CPP on the file anyway in order to extract the imports and module name, so we might as find the #included files and check their modification times.

See also #3588.

Change History (9)

comment:1 Changed 10 years ago by NeilMitchell

Cc: ndmitchell@… added
Type of failure: None/Unknown

I've got this logic in a build system I wrote. At the moment I need to run GHC with the force recompilation flag to make sure GHC builds when it should, after headers have changed.

comment:2 Changed 10 years ago by igloo

Milestone: 6.14 branch6.14.1

comment:3 Changed 9 years ago by igloo

Milestone: 7.0.17.0.2

comment:4 Changed 9 years ago by igloo

Milestone: 7.0.27.2.1

comment:5 Changed 8 years ago by igloo

Milestone: 7.2.17.4.1

comment:6 Changed 8 years ago by marlowsd@…

commit 3f34e0913efcc82dd90be56d04c5e57ec60d3677

Author: Simon Marlow <marlowsd@gmail.com>
Date:   Fri Nov 18 12:46:01 2011 +0000

    Track #included files for recompilation checking (#4900, #3589)
    
    This was pretty straightforward: collect the filenames in the lexer,
    and add them in to the tcg_dependent_files list that the typechecker
    collects.
    
    Note that we still don't get #included files in the ghc -M output.
    Since we don't normally lex the whole file in ghc -M, this same
    mechanism can't be used directly.

 compiler/main/GHC.hs              |   11 ++++++---
 compiler/main/HscMain.hs          |   41 +++++++++++++++++++++++++++++-------
 compiler/main/HscTypes.lhs        |   21 +++++++++++++++++++
 compiler/parser/Lexer.x           |    6 +++++
 compiler/typecheck/TcRnDriver.lhs |   16 ++++++++++---
 compiler/typecheck/TcRnMonad.lhs  |    6 +++++
 compiler/typecheck/TcRnTypes.lhs  |    2 +-
 7 files changed, 86 insertions(+), 17 deletions(-)

comment:7 Changed 8 years ago by simonmar

Resolution: fixed
Status: newclosed

Mostly fixed, see also #4900.

comment:8 Changed 5 years ago by Austin Seipp <austin@…>

In 803fc5db31f084b73713342cdceaed5a9c664267/ghc:

Add API Annotations

Summary:
The final design and discussion is captured at
https://ghc.haskell.org/trac/ghc/wiki/GhcAstAnnotations

This is a proof of concept implementation of a completely
separate annotation structure, populated in the parser,and tied to the
AST by means of a virtual "node-key" comprising the surrounding
SrcSpan and a value derived from the specific constructor used for the
node.

The key parts of the design are the following.

== The Annotations ==

In `hsSyn/ApiAnnotation.hs`

```lang=haskell
type ApiAnns = (Map.Map ApiAnnKey SrcSpan, Map.Map SrcSpan [Located Token])

type ApiAnnKey = (SrcSpan,AnnKeywordId)

-- ---------------------------------------------------------------------

-- | Retrieve an annotation based on the @SrcSpan@ of the annotated AST
-- element, and the known type of the annotation.
getAnnotation :: ApiAnns -> SrcSpan -> AnnKeywordId -> Maybe SrcSpan
getAnnotation (anns,_) span ann = Map.lookup (span,ann) anns

-- |Retrieve the comments allocated to the current @SrcSpan@
getAnnotationComments :: ApiAnns -> SrcSpan -> [Located Token]
getAnnotationComments (_,anns) span =
  case Map.lookup span anns of
    Just cs -> cs
    Nothing -> []

-- | Note: in general the names of these are taken from the
-- corresponding token, unless otherwise noted
data AnnKeywordId
         = AnnAs
         | AnnBang
         | AnnClass
         | AnnClose -- ^ } or ] or ) or #) etc
         | AnnComma
         | AnnDarrow
         | AnnData
         | AnnDcolon
         ....
```

== Capturing in the lexer/parser ==

The annotations are captured in the lexer / parser by extending PState to include a field

In `parser/Lexer.x`

```lang=haskell
data PState = PState {
        ....
        annotations :: [(ApiAnnKey,SrcSpan)]
        -- Annotations giving the locations of 'noise' tokens in the
        -- source, so that users of the GHC API can do source to
        -- source conversions.
     }
```

The lexer exposes a helper function to add an annotation

```lang=haskell
addAnnotation :: SrcSpan -> Ann -> SrcSpan -> P ()
addAnnotation l a v = P $ \s -> POk s {
  annotations = ((AK l a), v) : annotations s
  } ()

```

The parser also has some helper functions of the form

```lang=haskell
type MaybeAnn = Maybe (SrcSpan -> P ())

gl = getLoc
gj x = Just (gl x)

ams :: Located a -> [MaybeAnn] -> P (Located a)
ams a@(L l _) bs = (mapM_ (\a -> a l) $ catMaybes bs) >> return a
```

This allows annotations to be captured in the parser by means of

```
ctypedoc :: { LHsType RdrName }
        : 'forall' tv_bndrs '.' ctypedoc {% hintExplicitForall (getLoc $1) >>
                                            ams (LL $ mkExplicitHsForAllTy $2 (noLoc []) $4)
                                                [mj AnnForall $1,mj AnnDot $3] }
        | context '=>' ctypedoc         {% ams (LL $ mkQualifiedHsForAllTy   $1 $3)
                                               [mj AnnDarrow $2] }
        | ipvar '::' type               {% ams (LL (HsIParamTy (unLoc $1) $3))
                                               [mj AnnDcolon $2] }
        | typedoc                       { $1 }
```

== Parse result ==

```lang-haskell
data HsParsedModule = HsParsedModule {
    hpm_module    :: Located (HsModule RdrName),
    hpm_src_files :: [FilePath],
       -- ^ extra source files (e.g. from #includes).  The lexer collects
       -- these from '# <file> <line>' pragmas, which the C preprocessor
       -- leaves behind.  These files and their timestamps are stored in
       -- the .hi file, so that we can force recompilation if any of
       -- them change (#3589)
    hpm_annotations :: ApiAnns
  }

-- | The result of successful parsing.
data ParsedModule =
  ParsedModule { pm_mod_summary   :: ModSummary
               , pm_parsed_source :: ParsedSource
               , pm_extra_src_files :: [FilePath]
               , pm_annotations :: ApiAnns }
```

This diff depends on D426

Test Plan: sh ./validate

Reviewers: austin, simonpj, Mikolaj

Reviewed By: simonpj, Mikolaj

Subscribers: Mikolaj, goldfire, thomie, carter

Differential Revision: https://phabricator.haskell.org/D438

GHC Trac Issues: #9628

comment:9 Changed 4 years ago by thomie

Test Case: driver/recomp011/recomp011

In commit 5262def14ba7b8b6c8d1aea6860226da65cc4248:

Author: Simon Marlow <marlowsd@gmail.com>
Date:   Fri Nov 18 12:40:47 2011 +0000

    test for #3589
Note: See TracTickets for help on using tickets.