#14972 closed bug (fixed)

MacOS panic on TH

Reported by: harpocrates Owned by:
Priority: normal Milestone: 8.4.2
Component: Compiler Version: 8.4.1
Keywords: Cc: angerman
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time crash or panic Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D4553
Wiki Page:

Description

I recently did a clean build of master (affdea82bb70e5a912b679a169c6e9a230e4c93e) and, while everything successfully finished, I'm getting panics every time I try to use this GHC for something that involves TH.

For example:

{-# language TemplateHaskell #-}

pure []
main = pure []

Crashes with

$ ./inplace/bin/ghc-stage2 th.hs
[1 of 1] Compiling Main             ( th.hs, th.o )
ghc-stage2: loadArchive: Failed reading header from `/Users/atheriault/Documents/code/ghc/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180325 for x86_64-apple-darwin):
        loadArchive "/Users/atheriault/Documents/code/ghc/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Normally I'd assume I haven't correctly configured something, yet I had no problem building and running GHC on this machine a month or so ago...

Change History (14)

comment:1 Changed 21 months ago by mpickering

How did you build GHC? This kind of error occurs if you build with nix and don't use configurePhase and buildPhase.

comment:2 in reply to:  1 Changed 21 months ago by harpocrates

Replying to mpickering:

How did you build GHC? This kind of error occurs if you build with nix and don't use configurePhase and buildPhase.

I just followed the instructions from the Build page. Pretty much just

$ ./boot
$ ./configure --with-ghc=ghc-8.4.1
$ make -j4 V=0

I've been bisecting, but it is slow progress. The problem is not present in a2d03c69b782212e6c476cfc1870bae493a4ac89. I'll report back when I hit a commit.

comment:3 Changed 21 months ago by bgamari

Very odd; I wonder if this is related to #14891 (which is sadly present in 8.4.1).

comment:4 Changed 21 months ago by harpocrates

Alright. Without really having looked at the contents of commits, I can say something about the behaviour of certain ranges.

In terms of the way in which the crash happens: I've run into both segfaults and panics.

Since the possibly offending commit bumps Cabal, I'm not sure if this isn't just another manifestation of https://github.com/haskell/cabal/issues/5222.

comment:5 Changed 21 months ago by ezyang

This is even simpler to reproduce: ghc --interactive doesn't work on Mac OS X:

MacBook-Pro-97:unifier-ghc ezyang$ ~/Dev/ghc/inplace/bin/ghc-stage2 --interactive 
GHCi, version 8.5.20180327: http://www.haskell.org/ghc/  :? for help
ghc-stage2: loadArchive: Failed reading header from `/Users/ezyang/Dev/ghc/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180327 for x86_64-apple-darwin):
	loadArchive "/Users/ezyang/Dev/ghc/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

This was a 'perf' build of:

commit b58282a009ae67cbd4befbe062452026c82afd51 (HEAD -> master, origin/master, origin/HEAD)
Author: Ben Gamari <ben@smart-cactus.org>
Date:   Tue Mar 27 09:57:01 2018 -0400

    More format string fixes

comment:6 Changed 21 months ago by harpocrates

Another observation: if I make install, the installed GHC does not run into any of these problems.

comment:7 Changed 21 months ago by ezyang

I've diagnosed the proximate cause. In the failing version of the compiler, an inplace build creates a gmp directory in libraries/integer-gmp/dist-install/build/:

MacBook-Pro-97:ghc-quick1 ezyang$ ls libraries/integer-gmp/dist-install/build/
GHC						gmp
HSinteger-gmp-1.0.1.0.o				include
autogen						integer-gmp.buildinfo
cbits						libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib
config.log					libHSinteger-gmp-1.0.1.0.a
config.status

Unfortuantely, this changes the behavior of our print file name invocation:

MacBook-Pro-97:ghc-quick1 ezyang$ gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE -B/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build --print-file-name gmp
/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp

This should return gmp. When it returns a file path, we attempt to open this location, which clearly will not work:

*** gcc:
gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE -B/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build --print-file-name gmp

addLibrarySearchPath: dll_path = `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build'
Loading package integer-gmp-1.0.1.0 ... addDLL: dll_name = '/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib'
internal_dlopen: dll_name = '/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib'
loadArchive: start
loadArchive: Loading archive `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: loadArchive: Failed reading header from `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp'
loadArchive: done
*** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180331 for x86_64-apple-darwin):
	loadArchive "/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I hazard some change in Cabal caused the 'gmp' directory to be created. Not entirely sure what the correct fix is yet.

BTW, there is a really goofy workaround, which is just to delete/move away the offending directory. (It'll probably get recreated when you rebuild though...)

Last edited 21 months ago by ezyang (previous) (diff)

comment:8 Changed 21 months ago by ezyang

Cc: angerman added

angerman, CC'ing you, because the gmp folder exists in build because of https://github.com/haskell/cabal/pull/4874/commits

I think the easiest fix we can do for now is just rename the folder we put the config.mk in, but we should work out a way to dodge this footgun in general.

comment:9 Changed 21 months ago by angerman

@ezyang, right. So what we actually want is to improve the archive lookup logic.

comment:10 Changed 21 months ago by angerman

Differential Rev(s): D4553
Status: newpatch

Differential that ensures we verify whatever gcc --print-file-name returns is actually a file, and ignore it if not. (Should ignore folders).

@ezyang, thanks for the analysis!

comment:11 Changed 21 months ago by ezyang

Differential Rev(s): D4553Phab:D4553

Hmm. Suppose that gcc --print-file-name would have printed a real file name, and instead the existence of gmp directory in basedir causes gcc to print the gmp in basedir. Wouldn't we now have lost information that gcc --print-file-name would have otherwise given? OTOH I guess there isn't really any way to get the correct information now (unless you remove basedir, I suppose)

comment:12 Changed 20 months ago by Ben Gamari <ben@…>

In 2534164/ghc:

Move gmp/config.mk.in to config.mk.in, fix #14972

Here's how the rube goldberg machine triggered the old bug:

1. If you have a file gmp/config.mk.in, then Cabal will
create a generated file in $DIST/build/gmp/config.mk

2. When you attempt to load inplace integer-gmp via GHCi, it will
ask gcc (aka clang on OS X) for the file name of 'gmp', with
base directory set to $DIST/build

3. There is a folder named 'gmp', and so this folder is returned
as the 'library' for gmp

4. GHCi loadArchive chokes to death trying to open a library
that is actually a folder

This patch solves the problem by breaking the chain at (1): if we
don't put config.mk in a folder named gmp, NO PROBLEM.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Test Plan: validate

Reviewers: angerman, hvr, bgamari

Reviewed By: angerman

Subscribers: erikd, thomie, carter

GHC Trac Issues: #14972

Differential Revision: https://phabricator.haskell.org/D4552

comment:13 Changed 20 months ago by bgamari

Milestone: 8.4.2

comment:14 Changed 20 months ago by bgamari

Resolution: fixed
Status: patchclosed
Note: See TracTickets for help on using tickets.