Opened 22 months ago

Last modified 18 months ago

#14789 new bug

GHCi fails to garbage collect declaration `l = length [1..10^8]` entered at prompt

Reported by: kabuhr Owned by:
Priority: normal Milestone:
Component: GHCi Version: 8.2.2
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Runtime performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

This was recently reported on Stack Overflow (https://stackoverflow.com/questions/48723119/binding-len-length-xs-and-then-calculating-len-causes-ghc-to-consume-lots).

With GHC 8.2.2, assigning the length of a large list as a top-level declaration at the GHCi prompt and evaluating it:

Prelude> l = length [1..10^8]
Prelude> l
10000000
Prelude>

allocates about 7 gigs of memory before displaying a result, and then memory continues to grow over the next few seconds (at the idle GHCi prompt) until 10 gigs of memory has been allocated. None of it is freed until the interpreter session ends.

In contrast, directly evaluating the expression or assigning it with a let-statement does not exhibit this behavior:

Prelude> length [1..10^8]
10000000
Prelude> let l = length [1..10^8]
Prelude> l
10000000
Prelude>

Also, loading the top-level declaration from a file and then evaluating it at the prompt does not exhibit the problem.

I took a look at Ticket 14336 and applied the -fno-it patch referenced there. This slightly changed the pattern of memory usage: it grew to about 4 gigs before displaying the result, but then continued to grow for several seconds at the GHCi prompt before landing at about 10 gigs.

I verified that the bug still exists on the current "master" branch (though I didn't check the "ghc-8.4" branch).

Change History (2)

comment:1 Changed 18 months ago by bgamari

Just noticed this; quite an interesting issue. I was indeed able to reproduce this in GHC 8.4.1 and confirmed that performMajorGC does not cause reclaim.

comment:2 Changed 18 months ago by bgamari

Here is the code produced (from -ddump-rn -ddump-simpl -ddump-bcos) by the let l form,

λ> let l = length [1..10^8]

==================== Renamer ====================
let l_a2bl = length [1 .. 10 ^ 8]


==================== Simplified expression ====================
GHC.Base.returnIO
  @ [()]
  (GHC.Types.:
     @ ()
     ((Data.Foldable.length
         @ []
         Data.Foldable.$fFoldable[]
         @ GHC.Integer.Type.Integer
         (GHC.Enum.enumFromTo
            @ GHC.Integer.Type.Integer
            GHC.Enum.$fEnumInteger
            1
            (GHC.Real.^
               @ GHC.Integer.Type.Integer
               @ GHC.Integer.Type.Integer
               GHC.Num.$fNumInteger
               GHC.Real.$fIntegralInteger
               10
               8)))
      `cast` (UnsafeCo representational GHC.Types.Int ()
              :: (GHC.Types.Int :: *) ~R# (() :: *)))
     (GHC.Types.[] @ ()))



==================== Proto-BCOs ====================
ProtoBCO ExprTopLevel_E0#0 []:
   let sat_s3Wi = ... in ...
   bitmap:  0 []
   ALLOC_AP    0
   PUSH_BCO
     ProtoBCO sat_s3Wi#0 []:
        let sat_s3Wh = ... in ...
        bitmap:  0 []
        ALLOC_AP    0
        PUSH_BCO
          ProtoBCO sat_s3Wh#0 []:
             let sat_s3Wg = ... in ...
             bitmap:  0 []
             ALLOC_AP    0
             PUSH_BCO
               ProtoBCO sat_s3Wg#0 []:
                  let sat_s3Wf = ... in ...
                  bitmap:  0 []
                  PUSH_UBX (1) 8#
                  PACK     GHC.Integer.Type.S# 1
                  PUSH_UBX (1) 10#
                  PACK     GHC.Integer.Type.S# 1
                  PUSH_LL  1 0
                  PUSH_G   GHC.Real.$fIntegralInteger
                  PUSH_G   GHC.Num.$fNumInteger
                  PUSH_APPLY_PPPP
                  PUSH_G   GHC.Real.^
                  SLIDE    6 2
                  ENTER
             MKAP     0 words, 1 stkoff
             PUSH_UBX (1) 1#
             PACK     GHC.Integer.Type.S# 1
             PUSH_LL  1 0
             PUSH_G   GHC.Enum.$fEnumInteger
             PUSH_APPLY_PPP
             PUSH_G   GHC.Enum.enumFromTo
             SLIDE    5 2
             ENTER
        MKAP     0 words, 1 stkoff
        PUSH_L   0
        PUSH_G   Data.Foldable.$fFoldable[]
        PUSH_APPLY_PP
        PUSH_G   Data.Foldable.length
        SLIDE    4 1
        ENTER
   MKAP     0 words, 1 stkoff
   PUSH_G   GHC.Types.[]
   PUSH_L   1
   PACK     : 2
   PUSH_L   0
   PUSH_APPLY_P
   PUSH_G   GHC.Base.returnIO
   SLIDE    3 2
   ENTER

Here is the same output for the let-less form,

λ> l = length [1..10^8]

==================== Renamer ====================
interactive:Ghci2.l = length [1 .. 10 ^ 8]


==================== Tidy Core ====================
Result size of Tidy Core
  = {terms: 25, types: 11, coercions: 0, joins: 0/0}

-- RHS size: {terms: 10, types: 5, coercions: 0, joins: 0/0}
l :: Int
[GblId]
l = length
      @ []
      Data.Foldable.$fFoldable[]
      @ Integer
      (enumFromTo
         @ Integer
         GHC.Enum.$fEnumInteger
         1
         (^ @ Integer
            @ Integer
            GHC.Num.$fNumInteger
            GHC.Real.$fIntegralInteger
            10
            8))

-- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0}
$trModule1_r3XB :: GHC.Prim.Addr#
[GblId, Caf=NoCafRefs]
$trModule1_r3XB = "interactive"#

-- RHS size: {terms: 2, types: 0, coercions: 0, joins: 0/0}
$trModule2_r3XM :: GHC.Types.TrName
[GblId, Caf=NoCafRefs]
$trModule2_r3XM = GHC.Types.TrNameS $trModule1_r3XB

-- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0}
$trModule3_r3XN :: GHC.Prim.Addr#
[GblId, Caf=NoCafRefs]
$trModule3_r3XN = "Ghci2"#

-- RHS size: {terms: 2, types: 0, coercions: 0, joins: 0/0}
$trModule4_r3XO :: GHC.Types.TrName
[GblId, Caf=NoCafRefs]
$trModule4_r3XO = GHC.Types.TrNameS $trModule3_r3XN

-- RHS size: {terms: 3, types: 0, coercions: 0, joins: 0/0}
interactive:Ghci2.$trModule :: GHC.Types.Module
[GblId, Caf=NoCafRefs]
interactive:Ghci2.$trModule
  = GHC.Types.Module $trModule2_r3XM $trModule4_r3XO




==================== Proto-BCOs ====================
ProtoBCO sat_s3XU#0 []:
   let sat_s3XT = ... in ...
   bitmap:  0 []
   ALLOC_AP    0
   PUSH_BCO
     ProtoBCO sat_s3XT#0 []:
        let sat_s3XS = ... in ...
        bitmap:  0 []
        PUSH_UBX (1) 8#
        PACK     GHC.Integer.Type.S# 1
        PUSH_UBX (1) 10#
        PACK     GHC.Integer.Type.S# 1
        PUSH_LL  1 0
        PUSH_G   GHC.Real.$fIntegralInteger
        PUSH_G   GHC.Num.$fNumInteger
        PUSH_APPLY_PPPP
        PUSH_G   GHC.Real.^
        SLIDE    6 2
        ENTER
   MKAP     0 words, 1 stkoff
   PUSH_UBX (1) 1#
   PACK     GHC.Integer.Type.S# 1
   PUSH_LL  1 0
   PUSH_G   GHC.Enum.$fEnumInteger
   PUSH_APPLY_PPP
   PUSH_G   GHC.Enum.enumFromTo
   SLIDE    5 2
   ENTER
 
ProtoBCO Ghci2.l#0 []:
   Data.Foldable.length
     @ [] Data.Foldable.$fFoldable[] @ GHC.Integer.Type.Integer sat_s3XU
   bitmap:  0 []
   PUSH_G   sat_s3XU
   PUSH_G   Data.Foldable.$fFoldable[]
   PUSH_APPLY_PP
   PUSH_G   Data.Foldable.length
   ENTER
 
ProtoBCO $trModule2_r3XM#0 []:
   GHC.Types.TrNameS $trModule1_r3XB
   bitmap:  0 []
   PUSH_UBX (1) 26770624##
   PACK     GHC.Types.TrNameS 1
   ENTER
 
ProtoBCO $trModule4_r3XO#0 []:
   GHC.Types.TrNameS $trModule3_r3XN
   bitmap:  0 []
   PUSH_UBX (1) 26770592##
   PACK     GHC.Types.TrNameS 1
   ENTER
 
ProtoBCO Ghci2.$trModule#0 []:
   GHC.Types.Module $trModule2_r3XM $trModule4_r3XO
   bitmap:  0 []
   PUSH_G   $trModule4_r3XO
   PUSH_G   $trModule2_r3XM
   PACK     GHC.Types.Module 2
   ENTER

It looks to me like CorePrep is floating out the enumFromTo application, which then gets retained.

Last edited 18 months ago by bgamari (previous) (diff)
Note: See TracTickets for help on using tickets.