#14972 closed bug (fixed)

MacOS panic on TH

Reported by: harpocrates Owned by:
Priority: normal Milestone: 8.4.2
Component: Compiler Version: 8.4.1
Keywords: Cc: angerman
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time crash or panic Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D4553
Wiki Page:

Description

I recently did a clean build of master (affdea82bb70e5a912b679a169c6e9a230e4c93e) and, while everything successfully finished, I'm getting panics every time I try to use this GHC for something that involves TH.

For example:

{-# language TemplateHaskell #-}

pure []
main = pure []

Crashes with

$ ./inplace/bin/ghc-stage2 th.hs
[1 of 1] Compiling Main             ( th.hs, th.o )
ghc-stage2: loadArchive: Failed reading header from `/Users/atheriault/Documents/code/ghc/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180325 for x86_64-apple-darwin):
        loadArchive "/Users/atheriault/Documents/code/ghc/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Normally I'd assume I haven't correctly configured something, yet I had no problem building and running GHC on this machine a month or so ago...

Change History (14)

comment:1 Changed 20 months ago by mpickering

How did you build GHC? This kind of error occurs if you build with nix and don't use configurePhase and buildPhase.

comment:2 in reply to:  1 Changed 20 months ago by harpocrates

Replying to mpickering:

How did you build GHC? This kind of error occurs if you build with nix and don't use configurePhase and buildPhase.

I just followed the instructions from the Build page. Pretty much just

$ ./boot
$ ./configure --with-ghc=ghc-8.4.1
$ make -j4 V=0

I've been bisecting, but it is slow progress. The problem is not present in a2d03c69b782212e6c476cfc1870bae493a4ac89. I'll report back when I hit a commit.

comment:3 Changed 20 months ago by bgamari

Very odd; I wonder if this is related to #14891 (which is sadly present in 8.4.1).

comment:4 Changed 20 months ago by harpocrates

Alright. Without really having looked at the contents of commits, I can say something about the behaviour of certain ranges.

In terms of the way in which the crash happens: I've run into both segfaults and panics.

Since the possibly offending commit bumps Cabal, I'm not sure if this isn't just another manifestation of https://github.com/haskell/cabal/issues/5222.

comment:5 Changed 20 months ago by ezyang

This is even simpler to reproduce: ghc --interactive doesn't work on Mac OS X:

MacBook-Pro-97:unifier-ghc ezyang$ ~/Dev/ghc/inplace/bin/ghc-stage2 --interactive 
GHCi, version 8.5.20180327: http://www.haskell.org/ghc/  :? for help
ghc-stage2: loadArchive: Failed reading header from `/Users/ezyang/Dev/ghc/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180327 for x86_64-apple-darwin):
	loadArchive "/Users/ezyang/Dev/ghc/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

This was a 'perf' build of:

commit b58282a009ae67cbd4befbe062452026c82afd51 (HEAD -> master, origin/master, origin/HEAD)
Author: Ben Gamari <ben@smart-cactus.org>
Date:   Tue Mar 27 09:57:01 2018 -0400

    More format string fixes

comment:6 Changed 20 months ago by harpocrates

Another observation: if I make install, the installed GHC does not run into any of these problems.

comment:7 Changed 20 months ago by ezyang

I've diagnosed the proximate cause. In the failing version of the compiler, an inplace build creates a gmp directory in libraries/integer-gmp/dist-install/build/:

MacBook-Pro-97:ghc-quick1 ezyang$ ls libraries/integer-gmp/dist-install/build/
GHC						gmp
HSinteger-gmp-1.0.1.0.o				include
autogen						integer-gmp.buildinfo
cbits						libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib
config.log					libHSinteger-gmp-1.0.1.0.a
config.status

Unfortuantely, this changes the behavior of our print file name invocation:

MacBook-Pro-97:ghc-quick1 ezyang$ gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE -B/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build --print-file-name gmp
/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp

This should return gmp. When it returns a file path, we attempt to open this location, which clearly will not work:

*** gcc:
gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE -B/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build --print-file-name gmp

addLibrarySearchPath: dll_path = `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build'
Loading package integer-gmp-1.0.1.0 ... addDLL: dll_name = '/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib'
internal_dlopen: dll_name = '/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-1.0.1.0-ghc8.5.20180331.dylib'
loadArchive: start
loadArchive: Loading archive `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp'
ghc-stage2: loadArchive: Failed reading header from `/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp'
loadArchive: done
*** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
ghc-stage2: panic! (the 'impossible' happened)
  (GHC version 8.5.20180331 for x86_64-apple-darwin):
	loadArchive "/Users/ezyang/Dev/ghc-quick1/libraries/integer-gmp/dist-install/build/gmp": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I hazard some change in Cabal caused the 'gmp' directory to be created. Not entirely sure what the correct fix is yet.

Version 0, edited 20 months ago by ezyang (next)

comment:8 Changed 20 months ago by ezyang

Cc: angerman added

angerman, CC'ing you, because the gmp folder exists in build because of https://github.com/haskell/cabal/pull/4874/commits

I think the easiest fix we can do for now is just rename the folder we put the config.mk in, but we should work out a way to dodge this footgun in general.

comment:9 Changed 20 months ago by angerman

@ezyang, right. So what we actually want is to improve the archive lookup logic.

comment:10 Changed 20 months ago by angerman

Differential Rev(s): D4553
Status: newpatch

Differential that ensures we verify whatever gcc --print-file-name returns is actually a file, and ignore it if not. (Should ignore folders).

@ezyang, thanks for the analysis!

comment:11 Changed 20 months ago by ezyang

Differential Rev(s): D4553Phab:D4553

Hmm. Suppose that gcc --print-file-name would have printed a real file name, and instead the existence of gmp directory in basedir causes gcc to print the gmp in basedir. Wouldn't we now have lost information that gcc --print-file-name would have otherwise given? OTOH I guess there isn't really any way to get the correct information now (unless you remove basedir, I suppose)

comment:12 Changed 19 months ago by Ben Gamari <ben@…>

In 2534164/ghc:

Move gmp/config.mk.in to config.mk.in, fix #14972

Here's how the rube goldberg machine triggered the old bug:

1. If you have a file gmp/config.mk.in, then Cabal will
create a generated file in $DIST/build/gmp/config.mk

2. When you attempt to load inplace integer-gmp via GHCi, it will
ask gcc (aka clang on OS X) for the file name of 'gmp', with
base directory set to $DIST/build

3. There is a folder named 'gmp', and so this folder is returned
as the 'library' for gmp

4. GHCi loadArchive chokes to death trying to open a library
that is actually a folder

This patch solves the problem by breaking the chain at (1): if we
don't put config.mk in a folder named gmp, NO PROBLEM.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Test Plan: validate

Reviewers: angerman, hvr, bgamari

Reviewed By: angerman

Subscribers: erikd, thomie, carter

GHC Trac Issues: #14972

Differential Revision: https://phabricator.haskell.org/D4552

comment:13 Changed 19 months ago by bgamari

Milestone: 8.4.2

comment:14 Changed 19 months ago by bgamari

Resolution: fixed
Status: patchclosed
Note: See TracTickets for help on using tickets.