Opened 19 months ago

Last modified 10 months ago

#14974 new bug

2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package

Reported by: hvr Owned by: davide
Priority: high Milestone: 8.8.1
Component: Compiler Version: 8.4.1
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description (last modified by bgamari)

I haven't had time yet to diagnose where the memory is going to, but here's how to reproduce:

cabal get mmark-0.0.5.6 && cd mmark-0.0.5.6/

cat > cabal.project <<EOF
packages: .
package mmark
  ghc-options: -Rghc-timing
EOF

Then, cabal new-build -w ghc-8.4.1 and cabal new-build -w ghc-8.2.2 will respectively output

Resolving dependencies...
Build profile: -w ghc-8.4.1 -O1
In order, the following will be built (use -v for more details):
 - mmark-0.0.5.6 (lib) (first run)
Configuring library for mmark-0.0.5.6..
<<ghc: 13510872 bytes, 13 GCs, 806197/1299656 avg/max bytes residency (3 samples), 4M in use, 0.001 INIT (0.000 elapsed), 0.007 MUT (0.025 elapsed), 0.018 GC (0.018 elapsed) :ghc>>
Preprocessing library for mmark-0.0.5.6..
Building library for mmark-0.0.5.6..
[1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o )
[2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o )
[3 of 9] Compiling Text.MMark.Type  ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Type.o )
[4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Trans.o )
[5 of 9] Compiling Text.MMark.Util  ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Util.o )
[6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Render.o )
[7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser.o )
[8 of 9] Compiling Text.MMark       ( Text/MMark.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark.o )
[9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Extension.o )
<<ghc: 89663269464 bytes, 1856 GCs, 338140365/1430147624 avg/max bytes residency (23 samples), 4130M in use, 0.000 INIT (0.000 elapsed), 58.546 MUT (62.124 elapsed), 31.536 GC (31.505 elapsed) :ghc>>
<<ghc: 65387008 bytes, 50 GCs, 2189716/5047200 avg/max bytes residency (5 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.021 MUT (0.207 elapsed), 0.042 GC (0.042 elapsed) :ghc>>

and

Resolving dependencies...
Build profile: -w ghc-8.2.2 -O1
In order, the following will be built (use -v for more details):
 - mmark-0.0.5.6 (lib) (first run)
Configuring library for mmark-0.0.5.6..
<<ghc: 11200976 bytes, 11 GCs, 853349/1518976 avg/max bytes residency (3 samples), 5M in use, 0.001 INIT (0.000 elapsed), 0.017 MUT (0.037 elapsed), 0.021 GC (0.021 elapsed) :ghc>>
Preprocessing library for mmark-0.0.5.6..
Building library for mmark-0.0.5.6..
[1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o )
[2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o )
[3 of 9] Compiling Text.MMark.Type  ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Type.o )
[4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Trans.o )
[5 of 9] Compiling Text.MMark.Util  ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Util.o )
[6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Render.o )
[7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser.o )
[8 of 9] Compiling Text.MMark       ( Text/MMark.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark.o )
[9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist-newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Extension.o )
<<ghc: 89565902800 bytes, 2004 GCs, 225836231/772393000 avg/max bytes residency (27 samples), 2179M in use, 0.001 INIT (0.000 elapsed), 58.237 MUT (61.472 elapsed), 25.823 GC (25.795 elapsed) :ghc>>
<<ghc: 79930048 bytes, 56 GCs, 3522920/10072040 avg/max bytes residency (6 samples), 20M in use, 0.001 INIT (0.000 elapsed), 0.028 MUT (0.213 elapsed), 0.059 GC (0.059 elapsed) :ghc>>

Consequently,

  • GHC 8.2.2: 2179M in use, 0.001 INIT (0.000 elapsed), 58.237 MUT (61.472 elapsed), 25.823 GC (25.795 elapsed)
  • GHC 8.4.1: 4130M in use, 0.000 INIT (0.000 elapsed), 58.546 MUT (62.124 elapsed), 31.536 GC (31.505 elapsed)

Attachments (3)

terms_per_pass.png (134.1 KB) - added by davide 14 months ago.
Line plot of core size (in number of terms) for each pass when compiling Text.MMark.Parser. GHC 8.2.2 vs GHC 8.4.1.
bench_8.2.2.txt (4.0 KB) - added by davide 13 months ago.
Benchmark results for GHC 8.2.2
bench_8.4.1.txt (3.9 KB) - added by davide 13 months ago.
Benchmark results for GHC 8.4.1

Download all attachments as: .zip

Change History (14)

comment:1 Changed 19 months ago by hvr

Description: modified (diff)

comment:2 Changed 19 months ago by bgamari

Description: modified (diff)

comment:3 Changed 14 months ago by davide

Owner: set to davide

comment:4 Changed 14 months ago by simonpj

Milestone: 8.8.1

comment:5 Changed 14 months ago by davide

Building mmark With -dShow-passes, I get an increase in allocation for SpecConstr passes. The majority of this is from compiling the Text.MMark.Parser module:

  • 8.2.2: 1530MB
  • 8.4.1: 2900MB

Last edited 14 months ago by davide (previous) (diff)

comment:6 Changed 14 months ago by davide

With -ddump-core-stats the final core size of the Text.MMark.Parser module has not changed significantly, though the number of coercions has decreased:

  • GHC Version: terms,types,coercions
  • 8.2.2: 325846 296315 33104
  • 8.4.1: 344903 300944 26870

Changed 14 months ago by davide

Attachment: terms_per_pass.png added

Line plot of core size (in number of terms) for each pass when compiling Text.MMark.Parser. GHC 8.2.2 vs GHC 8.4.1.

comment:7 Changed 14 months ago by davide

The 2 peaks of the terms_per_pass.png​ plot are the core size after 3 consecutive 3 passes (for both the GHC 8.2.2 and GHC 8.4.1 peak): SpecConstr, SpecConstr, Simplifier. Note there are no other SpecConstr passes. The core size between GHC 8.2.2 and GHC 8.4.1 track very closely until immediately before the SpecConstr pass. Perhaps a small change in core (before SpecConstr) is allowing for more specialization to happen, or SpecConstr is needlessly creating more core.

comment:8 Changed 14 months ago by simonpj

Interesting questions

  • Do we get more or better specialisations from SpecConstr?
  • Do any of the library benchmarks run faster?

Maybe invite the author to give insight, once you have characterised better what is happening.

Changed 13 months ago by davide

Attachment: bench_8.2.2.txt added

Benchmark results for GHC 8.2.2

Changed 13 months ago by davide

Attachment: bench_8.4.1.txt added

Benchmark results for GHC 8.4.1

comment:9 Changed 13 months ago by davide

Benchmarks (with the exception of the benchmark using "data/bench-indented-code-block.md") have improved. It's possible that this is (at least in part) due to more specialization.

comment:10 Changed 10 months ago by sgraf

davide: Any news on this? Mind if I look into it?

comment:11 Changed 10 months ago by bgamari

Please do pick this up, sgraf!

Note: See TracTickets for help on using tickets.