haddock-2.4.2 shipped with ghc-6.10.2 is extremely slow

[maeder]

Calling haddock for our Hets project http://www.dfki.de/sks/hets takes around 40 minutes after upgrading to ghc-6.10.2. (haddock 2.3.0 with ghc-6.10.1 took below 1 minute.)

Unfortunately, reproducing this behavior requires to install many packages. You need fgl, xml, and time to compile anything at all within Hets.

Check out: https://svn-agbkb.informatik.uni-bremen.de/Hets/trunk https://svn-agbkb.informatik.uni-bremen.de/uni/trunk

You need the uni packages and hxtfilter-8.2.0 to cause haddock to become slow. (Look inside Hets/INSTALL)

Within Hets I call:

cabal configure -v -f -gtkglade -f -programatica -f -shellac -f -haxml -f -tabular -f -tar

to avoid dependencies on even further packages.

Compiling 433 files using "time cabal build" takes:

real    4m34.159s
user    4m21.308s
sys     0m7.456s

Running now haddock using "time cabal haddock --executables" takes about 40 minutes

I'll attach the corresponding log files. (The sources contain one template haskell file Hets/SoftFOL/MathServCommunication.hs)

[maeder]

compiler and haddock are those of http://www.haskell.org/ghc/dist/6.10.2/ghc-6.10.2-i386-unknown-linux.tar.bz2 but haddock has this problem on other platforms, too. (my haddock output isn't ready, yet)

[maeder]

The haddock documentation created by "make doc" within hets usually looks like: http://www.dfki.de/sks/hets/src-distribution/versions/Hets/docs/

[maeder]

"top" frequently displays "cc1" instances and sometimes "ghc-asm" (although cc1 is not used when building).

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
32166 maeder    25   0 28944  23m 3396 R   64  0.8   0:01.91 cc1
29256 maeder    24   0  344m 339m  26m S    5 11.2   6:09.98 haddock

[maeder]

here is my haddock output:

Preprocessing executables for Hets-0.94...
Running Haddock for Hets-0.94...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: HAIFA-0.12, regex-compat-0.71.0.1,
rts-1.0, syb-generics-2.9
Warning: SoftFOL.MathServCommunication: could not find link destinations for:
    Text.XML.Serializer.Core.XMLData Text.XML.Serializer.Datatypes.XMLNamespace Data.Generics2.Basics.Data Text.XML.Serializer.Datatypes.DictXMLData
Warning: SoftFOL.MathServParsing: could not find link destinations for:
    Network.Service.HTTPTransport
Warning: OWL.Namespace: could not find link destinations for:
    OWL.Namespace.TranslationMap
Warning: Comorphisms.PPolyTyConsHOL2IsaUtils: could not find link destinations for:
    Comorphisms.PPolyTyConsHOL2IsaUtils.Simplifier
Warning: GUI.GraphAbstraction: could not find link destinations for:
    GUI.GraphAbstraction.OurGraph
Warning: Static.AnalysisLibrary: could not find link destinations for:
    Static.AnalysisLibrary.LNS
Documentation created: dist/doc/html/Hets/hets/index.html

real    46m19.788s
user    42m10.762s
sys     3m24.797s

[maeder]

The problem may be related to the above mentioned template haskell file or the underlying HAIFA and syp-generics packages, because without it (by adding "cpp-options: -DNOMATHSERVER" and compiling only 428 files), haddock is fast:

maeder@leibniz:~/Hets> time cabal haddock --executables
Preprocessing executables for Hets-0.94...
Running Haddock for Hets-0.94...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: HAIFA-0.12, regex-compat-0.71.0.1,
rts-1.0, syb-generics-2.9
Warning: OWL.Namespace: could not find link destinations for:
    OWL.Namespace.TranslationMap
Warning: Comorphisms.PPolyTyConsHOL2IsaUtils: could not find link destinations for:
    Comorphisms.PPolyTyConsHOL2IsaUtils.Simplifier
Warning: GUI.GraphAbstraction: could not find link destinations for:
    GUI.GraphAbstraction.OurGraph
Warning: Static.AnalysisLibrary: could not find link destinations for:
    Static.AnalysisLibrary.LNS
Documentation created: dist/doc/html/Hets/hets/index.html

real    0m35.475s
user    0m34.674s
sys     0m0.776s

[waern]

The reason for the slowdown is that Haddock switches from compiling nothing to compiling everything via C when at least one module uses Template Haskell. I don't know if there's an easy way to determine which modules need compiling and which don't.

[simonmar]

Why does it use via-C, and not asm?

[waern]

Replying to simonmar:

Why does it use via-C, and not asm?

No particular reason. It should probably use asm if that works. However, that will not speed it up enough, right? So we should ideally only compile/interpret the files that really need to be compiled/interpreted. Do you think GHC could provide an easy way to know this?

Do you think that, in the meantime, it would be worth using the interpreted mode, letting Haddock fail on modules that need compilation?

[simonmar]

I remember talking with nominolo about this, but don't remember the details. Interpreted mode may well be better, ideally there should be a way for the user to override it if they know that compilation is required.

[nominolo]

Interpreted mode works as long as no module needs certain features that are not supported by interpreted mode. This includes FFI exports and unboxed tuples (and perhaps something else which I forgot). There's a flag that forces modules to be compiled, but last time I tested this it didn't work completely as expected; I forgot the details.

Unfortunately, I don't know of a good way to detect which files will need code, because we only find out which functions are used during Template Haskell code execution.

[waern]

Since we can't select which modules to load in a certain way, it seems to risky to use the interpreted mode. So I think we'll continue to compile, although using asm until the GHC API provides some way to know which files need interpretation or compilation.

[anonymous]

Milestone 2.5.0 deleted