Opened 9 months ago

Last modified 8 months ago

#16017 new bug

ghc-8.6.1 and ghc-8.6.2 use a log of memory

Reported by: newhoggy Owned by:
Priority: normal Milestone:
Component: Compiler Version: 8.6.2
Keywords: Cc: gidyn
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

Currently GHC uses a lot of memory to build a relatively small module and causes my CI to fail due to there being a 4G memory limit.

The source code can be found here:

https://github.com/haskell-works/hw-json/tree/73368cee21dc72eedd5291ba689f9abf10e7fcd2

The problem module is here:

https://github.com/haskell-works/hw-json/blob/73368cee21dc72eedd5291ba689f9abf10e7fcd2/test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs

The build output follows:

cabal new-build --enable-tests --enable-benchmarks --project-file="cabal.project" -j${CABAL_THREADS:-4} all
Build profile: -w ghc-8.6.2 -O2
In order, the following will be built (use -v for more details):
 - hw-json-0.9.0.1 (lib) (first run)
 - hw-json-0.9.0.1 (test:hw-json-test) (first run)
 - hw-json-0.9.0.1 (exe:hw-json) (first run)
 - hw-json-0.9.0.1 (bench:bench) (first run)
Configuring library for hw-json-0.9.0.1..
Preprocessing library for hw-json-0.9.0.1..
Building library for hw-json-0.9.0.1..
[ 1 of 32] Compiling HaskellWorks.Data.Json.DecodeError ( src/HaskellWorks/Data/Json/DecodeError.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/DecodeError.o )
[ 2 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.MakeIndex ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.o )
[ 3 of 32] Compiling HaskellWorks.Data.Json.Internal.Index ( src/HaskellWorks/Data/Json/Internal/Index.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Index.o )
[ 4 of 32] Compiling HaskellWorks.Data.Json.Internal.PartialIndex ( src/HaskellWorks/Data/Json/Internal/PartialIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/PartialIndex.o )
[ 5 of 32] Compiling HaskellWorks.Data.Json.Internal.Token.Types ( src/HaskellWorks/Data/Json/Internal/Token/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token/Types.o )
[ 6 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Token.Tokenize ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.o )
[ 7 of 32] Compiling HaskellWorks.Data.Json.Internal.Token ( src/HaskellWorks/Data/Json/Internal/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token.o )
[ 8 of 32] Compiling HaskellWorks.Data.Json.Internal.Value ( src/HaskellWorks/Data/Json/Internal/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Value.o )
[ 9 of 32] Compiling HaskellWorks.Data.Json.Internal.Word8 ( src/HaskellWorks/Data/Json/Internal/Word8.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word8.o )
[10 of 32] Compiling HaskellWorks.Data.Json.Internal.Word64 ( src/HaskellWorks/Data/Json/Internal/Word64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word64.o )
[11 of 32] Compiling HaskellWorks.Data.Json.Internal.CharLike ( src/HaskellWorks/Data/Json/Internal/CharLike.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/CharLike.o )
[12 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Blank ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.o )
[13 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.BlankedJson ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.o )
[14 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToInterestBits64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.o )
[15 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToBalancedParens64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.o )
[16 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.IbBp ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.o )
[17 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.o )
[18 of 32] Compiling HaskellWorks.Data.Json.Backend.Simple.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.o )
[19 of 32] Compiling HaskellWorks.Data.Json.LightJson ( src/HaskellWorks/Data/Json/LightJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LightJson.o )
[20 of 32] Compiling HaskellWorks.Data.Json.PartialValue ( src/HaskellWorks/Data/Json/PartialValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/PartialValue.o )
[21 of 32] Compiling HaskellWorks.Data.Json.Type ( src/HaskellWorks/Data/Json/Type.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Type.o )
[22 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Cursor ( src/HaskellWorks/Data/Json/Backend/Standard/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Cursor.o )
[23 of 32] Compiling HaskellWorks.Data.Json.Cursor ( src/HaskellWorks/Data/Json/Cursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Cursor.o )
[24 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Cursor.Token ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.o )
[25 of 32] Compiling HaskellWorks.Data.Json.Value ( src/HaskellWorks/Data/Json/Value.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Value.o )
[26 of 32] Compiling HaskellWorks.Data.Json.FromValue ( src/HaskellWorks/Data/Json/FromValue.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/FromValue.o )
[27 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.LoadCursor ( src/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.o )
[28 of 32] Compiling HaskellWorks.Data.Json.LoadCursor ( src/HaskellWorks/Data/Json/LoadCursor.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LoadCursor.o )
[29 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Load ( src/HaskellWorks/Data/Json/Backend/Standard/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Load.o )
[30 of 32] Compiling HaskellWorks.Data.Json.Load ( src/HaskellWorks/Data/Json/Load.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Load.o )
[31 of 32] Compiling HaskellWorks.Data.Json ( src/HaskellWorks/Data/Json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/HaskellWorks/Data/Json.o )
[32 of 32] Compiling Paths_hw_json    ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/opt/build/Paths_hw_json.o )
Configuring test suite 'hw-json-test' for hw-json-0.9.0.1..
Configuring benchmark 'bench' for hw-json-0.9.0.1..
Configuring executable 'hw-json' for hw-json-0.9.0.1..
Preprocessing test suite 'hw-json-test' for hw-json-0.9.0.1..
Building test suite 'hw-json-test' for hw-json-0.9.0.1..
Preprocessing executable 'hw-json' for hw-json-0.9.0.1..
Preprocessing benchmark 'bench' for hw-json-0.9.0.1..
Building benchmark 'bench' for hw-json-0.9.0.1..
Building executable 'hw-json' for hw-json-0.9.0.1..
[ 1 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.Cursor.InterestBitsSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.o )
[1 of 2] Compiling Main             ( bench/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Main.o )
[1 of 7] Compiling App.Commands.Types ( app/App/Commands/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Types.o )
[2 of 7] Compiling App.Lens         ( app/App/Lens.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Lens.o )
[3 of 7] Compiling App.Commands.Demo ( app/App/Commands/Demo.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Demo.o )
[2 of 2] Compiling Paths_hw_json    ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Paths_hw_json.o )
Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/b/bench/opt/build/bench/bench ...
[4 of 7] Compiling App.Commands.CreateIndex ( app/App/Commands/CreateIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/CreateIndex.o )
[5 of 7] Compiling App.Commands     ( app/App/Commands.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/App/Commands.o )
[ 2 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.CursorSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t/hw-json-test/opt/build/hw-json-test/hw-json-test-tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.o )
[6 of 7] Compiling Main             ( app/Main.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Main.o )
[7 of 7] Compiling Paths_hw_json    ( /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json-tmp/Paths_hw_json.o )
Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json ...
cabal: Failed to build test:hw-json-test from hw-json-0.9.0.1. The build
process was killed (i.e. SIGKILL). The typical reason for this is that there
is not enough memory available (e.g. the OS killed a process using lots of
memory).

Exited with code 1

Change History (4)

comment:1 Changed 9 months ago by newhoggy

For my own purposes, breaking up the module into two smaller modules was sufficient to avoid triggering the problem, but it would be good if the memory usage could be reduced for this case.

comment:2 Changed 9 months ago by gidyn

Cc: gidyn added

comment:3 Changed 9 months ago by bgamari

Milestone: 8.6.3

Ticket retargeted after milestone closed

comment:4 Changed 8 months ago by simonpj

Type of failure: None/UnknownCompile-time performance bug
Note: See TracTickets for help on using tickets.