Opened 2 years ago

Closed 2 years ago

Last modified 21 months ago

#14697 closed bug (fixed)

Redundant computation in fingerprintDynFlags when compiling many modules

Reported by: niteria Owned by:
Priority: normal Milestone:
Component: Compiler Version:
Keywords: Cc: simonmar
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): phab:D4445
Wiki Page:

Description (last modified by niteria)

I profiled a build of a production code base with thousands of modules and computing fingerprintDynFlags is 7% of time and 14% of allocations.

Here's a synthetic test case inspired by what I observed:


for i in $(seq -w 1 $SIZE); do                      
  echo "module A$i where" > A$i.hs                  
  echo "data A$i = A$i" >> A$i.hs                   

This generates a 1000 modules each with one datatype. Compiling them with:

inplace/bin/ghc-stage2 A*.hs -optP-D__F{1..10000}__

results in fingerprintDynFlags being the top cost centre in the profile. AFAICT there's only one module dependent piece that goes into computing fingerprintDynFlags and the rest is the same between those 1000 modules.

Now, why would I have so many preprocessor flags? This is how the Buck build system currently works. If a Haskell library depends on a C++ library then the GHC invocation gets the C++ library's directory as include path (-optP -I -optP some/library/path). This can grow quite big.

Change History (5)

comment:1 Changed 2 years ago by niteria

Cc: simonmar added
Description: modified (diff)

comment:2 Changed 2 years ago by niteria

Differential Rev(s): phab:D4445
Status: newpatch

comment:3 Changed 2 years ago by Bartosz Nitka <niteria@…>

In b8f03bbe/ghc:

Cache the fingerprint of sOpt_P

Before this change we would compute a hash of
all the command line -optP flags once per file.
With a lot of files and many -optP flags, that's a lot
of repeated work.

I added a new Note that explains the approach and rationale.

Test Plan: new test

Reviewers: simonmar, simonpj, bgamari

Reviewed By: simonpj

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #14697

Differential Revision:

comment:4 Changed 2 years ago by niteria

Resolution: fixed
Status: patchclosed

comment:5 Changed 21 months ago by Ben Gamari <ben@…>

In af986f9d/ghc:

testsuite: Disable T14697 on Windows

Test Plan: Validate on Windows

Subscribers: thomie, carter

GHC Trac Issues: #14697, #15072

Differential Revision:
Note: See TracTickets for help on using tickets.