Opened 9 years ago

Closed 7 years ago

#4421 closed bug (fixed)

Compilation performance regression

Reported by: simonmar Owned by: igloo
Priority: high Milestone: 7.6.1
Component: Compiler Version: 6.12.3
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

This patch

Tue Oct 19 08:45:52 PDT 2010  Ian Lynagh <igloo@earth.li>
  * seq the unfolding in setUnfoldingInfo
  Contrary to the comment, for the module in #4367 at least, it is a big
  improvement. Without it we get a huge spike of drag.

Had a big impact on compile times, we probably need to address the leak some other way. HEAD before the patch, compiling Cabal with -O:

  45,244,270,096 bytes allocated in the heap
   7,967,118,472 bytes copied during GC
     162,341,616 bytes maximum residency (40 sample(s))
      21,318,984 bytes maximum slop
             472 MB total memory in use (0 MB lost due to fragmentation)

  Generation 0: 85611 collections,     0 parallel, 25.94s, 25.95s elapsed
  Generation 1:    40 collections,     0 parallel, 10.13s, 10.14s elapsed

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   42.92s  ( 45.75s elapsed)
  GC    time   36.07s  ( 36.09s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time   78.99s  ( 81.84s elapsed)

and after the patch:

  67,912,400,256 bytes allocated in the heap
  10,084,908,352 bytes copied during GC
      88,734,416 bytes maximum residency (67 sample(s))
      18,782,600 bytes maximum slop
             268 MB total memory in use (0 MB lost due to fragmentation)

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   57.62s  ( 60.52s elapsed)
  GC    time   47.92s  ( 47.96s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time  105.55s  (108.48s elapsed)

overall space usage is much better (268M vs. 472M), but allocations and total time are much worse.

Results for 6.12.3:

  26,404,281,056 bytes allocated in the heap
   6,533,229,296 bytes copied during GC
      91,037,368 bytes maximum residency (49 sample(s))
      17,657,552 bytes maximum slop
             271 MB total memory in use (4 MB lost due to fragmentation)

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   34.32s  ( 36.88s elapsed)
  GC    time   24.46s  ( 25.25s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time   58.78s  ( 62.13s elapsed)

and for STABLE (before the offending patch):

  32,812,959,504 bytes allocated in the heap
   7,358,388,512 bytes copied during GC
     159,592,480 bytes maximum residency (39 sample(s))
      20,403,664 bytes maximum slop
             466 MB total memory in use (0 MB lost due to fragmentation)

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   38.59s  ( 41.45s elapsed)
  GC    time   30.72s  ( 30.74s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time   69.32s  ( 72.19s elapsed)

HEAD is worse than STABLE right now, probably due to the containers package changes.

STABLE has still regressed relative to 6.12.3, though.

Change History (10)

comment:1 Changed 9 years ago by igloo

Owner: set to igloo

comment:2 Changed 9 years ago by igloo

With the patch, I was seeing

  70,966,109,224 bytes allocated in the heap
  10,517,401,920 bytes copied during GC
      88,885,320 bytes maximum residency (67 sample(s))
      19,679,064 bytes maximum slop
             273 MB total memory in use (0 MB lost due to fragmentation)

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   74.56s  ( 77.79s elapsed)
  GC    time   78.38s  ( 78.31s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time  152.95s  (156.10s elapsed)

but after

Wed Oct 20 14:15:39 BST 2010  Ian Lynagh <igloo@earth.li>
  * Avoid hanging on to old unfoldings; fixes #4367 (compiler space regression)
Wed Oct 20 15:37:10 BST 2010  Ian Lynagh <igloo@earth.li>
  * Don't seq unfoldings

I see

  46,804,639,544 bytes allocated in the heap
   7,954,212,312 bytes copied during GC
     123,843,728 bytes maximum residency (47 sample(s))
      20,877,992 bytes maximum slop
             377 MB total memory in use (0 MB lost due to fragmentation)

  INIT  time    0.00s  (  0.00s elapsed)
  MUT   time   54.89s  ( 58.56s elapsed)
  GC    time   55.02s  ( 55.03s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time  109.91s  (113.59s elapsed)

so time and allocs look good, but total mem usage is high (but not as high as it used to be). Presumably we still sometimes retain old unfoldings, but not in cases that happen with the #4367 testcase.

comment:3 Changed 9 years ago by simonpj

Milestone: 7.0.17.0.2

comment:4 Changed 9 years ago by igloo

Milestone: 7.0.27.0.3

comment:5 Changed 8 years ago by batterseapower

Type of failure: None/UnknownCompile-time performance bug

comment:6 Changed 8 years ago by igloo

Priority: highesthigh

comment:7 Changed 8 years ago by igloo

Milestone: 7.2.17.4.1

comment:8 Changed 8 years ago by igloo

difficulty: Unknown
Milestone: 7.4.17.4.2

comment:9 Changed 8 years ago by igloo

Milestone: 7.4.27.6.1

comment:10 Changed 7 years ago by igloo

Resolution: fixed
Status: newclosed

The regression is mostly fixed, and compiler performance is just something we need to keep a general eye on. I've done some profiling and things generally look OK, so I think I'll just close this ticket now.

Note: See TracTickets for help on using tickets.