Opened 7 years ago

Closed 7 years ago

#7257 closed bug (fixed)

Regression: pinned memory fragmentation

Reported by: jwlato Owned by: simonmar
Priority: highest Milestone: 7.6.2
Component: Compiler Version: 7.6.1
Keywords: Cc: jwlato@…, tkn.akio@…, bgamari@…
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Runtime performance bug Test Case: perf/should_run/T7257
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:


In certain cases, ghc-7.6.1 seems to have much worse memory behavior than 7.4. I have attached a simplified program that I believe demonstrates the problem.

-- compiled with ghc-7.4.2 -rtsopts -O2
./Foo  +RTS -s
   3,549,666,928 bytes allocated in the heap
      55,512,376 bytes copied during GC
     335,555,096 bytes maximum residency (10 sample(s))
       3,787,576 bytes maximum slop
             453 MB total memory in use (0 MB lost due to fragmentation)
-- compiled with ghc-7.6.1 -rtsopts -O2
   2,699,298,272 bytes allocated in the heap
      55,077,544 bytes copied during GC
     327,246,968 bytes maximum residency (11 sample(s))
       3,767,408 bytes maximum slop
             858 MB total memory in use (398 MB lost due to fragmentation)

One of our applications uses 3-4GB more RAM when compiled with ghc-7.6.1 compared to 7.4, all due to memory fragmentation issues.

Attachments (1)

Foo.hs (700 bytes) - added by jwlato 7 years ago.

Download all attachments as: .zip

Change History (9)

Changed 7 years ago by jwlato

Attachment: Foo.hs added


comment:1 Changed 7 years ago by akio

Cc: tkn.akio@… added

comment:2 Changed 7 years ago by jwlato

Addendum: with ghc-7.6.1, I see the same memory behavior with multiple versions of ByteString and Data.Vector.Storable, but not Text, Data.Vector, or Data.Vector.Unboxed.

comment:3 Changed 7 years ago by simonmar

difficulty: Unknown
Milestone: 7.6.2
Owner: set to simonmar
Priority: normalhighest

Sorry about this. I know what the problem is, and a fix is on the way. As a bonus, I have made the program go faster by a factor of 10.

comment:4 Changed 7 years ago by marlowsd@…

commit 016fd74d6517512b62b36ff12cdccf2e723a0fb3

Author: Simon Marlow <>
Date:   Fri Sep 21 13:18:49 2012 +0100

    Cache the result of countOccupied(gen->large_objects) as gen->n_large_words (#7257)
    The program in #7257 was spending 90% of its time counting the live
    data in gen->large_objects.  We already avoid doing this for small
    objects, but in this example the old generation was full of large
    objects (actually pinned ByteStrings).

 includes/rts/storage/GC.h |    1 +
 rts/sm/GC.c               |    5 ++++-
 rts/sm/Storage.c          |    3 ++-
 3 files changed, 7 insertions(+), 2 deletions(-)

comment:5 Changed 7 years ago by simonmar

Status: newmerge

Please merge the following commits:

commit a68df77ede928e6c7790dacb5925625792a904d3
Author: Simon Marlow <>
Date:   Tue Aug 21 11:39:06 2012 +0100

    Reduce fragmentation when using +RTS -H (with or without a size)

commit a8179622f84bbd52e127a9596d2d4a918ca64e0c
Author: Simon Marlow <>
Date:   Fri Sep 7 13:36:09 2012 +0100

    Some further tweaks to reduce fragmentation when allocating the nursery

commit 1f5d83648dfda39d999eb8a9e8192339b3eea540
Author: Simon Marlow <>
Date:   Fri Sep 21 13:11:22 2012 +0100

    Allow allocNursery() to allocate single blocks (#7257)
    Forcing large allocations here can creates serious fragmentation in
    some cases, and since the large allocations are only a small
    optimisation we should allow the nursery to hoover up small blocks
    before allocating large chunks.

commit c19f2e320a575a66d31c83d846ce3cc954c4ab3b
Author: Simon Marlow <>
Date:   Fri Sep 21 13:20:37 2012 +0100

    Include pinned memory in the stats for allocated memory
    This broke with the changes to the pinned object handling in

comment:6 Changed 7 years ago by simonmar

Test Case: perf/should_run/T7257

comment:7 Changed 7 years ago by bgamari

Cc: bgamari@… added
Note: See TracTickets for help on using tickets.