Opened 5 years ago

Closed 5 years ago

#9771 closed bug (fixed)

Excessive memory usage compiling T3064

Reported by: hvr Owned by: simonpj
Priority: highest Milestone: 7.10.1
Component: Compiler Version: 7.9
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

With 64dc4d1085a7db375f6532362bf7e23fac3a25eb the testsuite suffers from T3064 allocating an excessive amount of memory (several GiBs) and finally getting killed due to timeout and/or out-of-memory.

=====> T3064(normal) 3054 of 4150 [0, 0, 0] 
cd ./perf/compiler && '/home/hvr/ghc/inplace/bin/ghc-stage2' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history -c T3064.hs   +RTS -V0 -tT3064.comp.stats --machine-readable -RTS  >T3064.comp.stderr 2>&1
Compile failed (status 35072) errors were:
Killed

Failed to find field:  peak_megabytes_allocated
*** framework failure for T3064(normal) do_test exception 
Traceback (most recent call last):
  File "/home/hvr/ghc/testsuite/driver/testlib.py", line 793, in do_test
    result = func(*[name,way] + args)
  File "/home/hvr/ghc/testsuite/driver/testlib.py", line 995, in compile
    return do_compile( name, way, 0, '', [], extra_hc_opts )
  File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1024, in do_compile
    result = simple_build( name, way, extra_hc_opts, should_fail, top_mod, 0, 1, force, override_flags )
  File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1262, in simple_build
    statsResult = checkStats(name, way, stats_file, opts.compiler_stats_range_fields)
  File "/home/hvr/ghc/testsuite/driver/testlib.py", line 1136, in checkStats
    val = int(m.group(1))
AttributeError: 'NoneType' object has no attribute 'group'

OVERALL SUMMARY for test run started at Wed Nov  5 08:29:10 2014 CET
 0:04:14 spent to go through
    4150 total tests, which gave rise to
   16187 test cases, of which
   16186 were skipped

       0 had missing libraries
       0 expected passes
       0 expected failures

       1 caused framework failures
       0 unexpected passes
       0 unexpected failures
       0 unexpected stat failures

make[1]: Leaving directory '/home/hvr/ghc/testsuite/tests'

real	4m13.317s
user	3m49.476s
sys	0m5.831s

Change History (4)

comment:1 Changed 5 years ago by Herbert Valerio Riedel <hvr@…>

In 77f8221103a98b38384edd8c2caae6cc2c4ffd72/ghc:

Temporarily disable T3064 (see #9771)

This disables T3064 temporarily as it puts a strain on buildbots
during validation exhausting all available memory.

comment:2 Changed 5 years ago by simonpj

Thanks Herbert. I'll look into this.

There really ought to be a way to ensure that tests don't go AWOL and eat all resources, though.

Simon

comment:3 in reply to:  2 Changed 5 years ago by hvr

Replying to simonpj:

There really ought to be a way to ensure that tests don't go AWOL and eat all resources, though.

Well, one could either pass some +RTS options to ghc to limit its heap via -M<size> (which otoh distorts GHC's reported memory stats, and relies on the RTS to correctly enforce that limit), or (at least on Linux, so I'm not sure what to do for Windows buildbots) you can use ulimit to impose a memory allocation limit for a process (and its children).

comment:4 Changed 5 years ago by simonpj

Resolution: fixed
Status: newclosed

I believe I have fixed the long compile time on T3064. I've re-enabled:

commit c79cbacb6d9161c529ac13685ff29ac058a3ebfa
Author: Simon Peyton Jones <simonpj@microsoft.com>
Date:   Thu Nov 6 15:46:11 2014 +0000

    Re-enable T3064, which works now

 testsuite/tests/perf/compiler/all.T |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Note: See TracTickets for help on using tickets.