Opened 6 years ago

Closed 5 years ago

Last modified 5 years ago

#8683 closed feature request (fixed)

add "cpp program" and "cpp program options" to settings file

Reported by: carter Owned by: carter
Priority: high Milestone: 7.8.3
Component: Compiler Version: 7.8.1-rc2
Keywords: Cc: tibbe
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: #8439 Differential Rev(s):
Wiki Page:

Description

so theres a pretty clear consensus that we need to move away from using our c compiler for CPP, so should we add that info to the settings file?

I propose adding the following fields to the settings file:

"use c compiler cpp", which can have either "YES" or "NO". If "YES", the settings file indicated C compiler is used unless the c compiler selection is overrided by ghc at runtime via -pgmc or -pgmp(p?) if "NO", then the "cpp program" and "cpp program options" fields would be used, unless overridden by -pgmp

there'd be some associated patched in the compiler/main/DriverPipeline.hs too i think, but relatively minimal.

this would make it much easier to distributed a GHC with its own CPP tool that works

Change History (72)

comment:1 Changed 6 years ago by hvr

comment:2 Changed 6 years ago by carter

Ok, i've started digging into this, with the idea of having the settings file have the "Haskell CPP Command" and "Haskell CPP Flags" fields.

the idea being that the settings file would determine the CPP and flags unless pgmp and optp are set.

soo, some of the CPP stuff is welded into certain config and other steps currently

eg in mk/config.mk.in

# It's not easy to separate the CPP program from its flags, as
# AC_PROG_CPP defines CPP as "/usr/bin/gcc -E"
CPP			= @CPP@ @CPPFLAGS@
#
# RAWCPP_FLAGS are the flags to give to cpp (viz, gcc -E) to persuade it to
# behave plausibly on Haskell sources.
#
# Clang in particular is a bit more annoying, so we suppress some warnings.
RAWCPP_FLAGS            = -undef -traditional
ifeq "$(CC_CLANG_BACKEND)" "1"
RAWCPP_FLAGS           += -Wno-invalid-pp-token -Wno-unicode -Wno-trigraphs
endif

which then in compiler/ghc.mk as part of the Config module has the line

	@echo 'cRAWCPP_FLAGS         :: String'                             >> $@
	@echo 'cRAWCPP_FLAGS         = "$(RAWCPP_FLAGS)"'                   >> $@

and then in compiler/main/SysTools.lhs we have

       let cpp_prog  = gcc_prog
           cpp_args  = Option "-E"
                     : map Option (words cRAWCPP_FLAGS)
                    ++ gcc_args

Interestingly, the -pgmP flag corresponds to

sPgm_P   = (cpp_prog, cpp_args),

and the optP flag is by default empty!

     sOpt_P       = []

so optP is meant for adding *additional* flags to cpp, but shouldn't be part of the default config, AND pgmP should be used to specify the full "cpp program and the prefix of args?"

also currently it looks like cpp_args also passes in applicable gcc_args. (are there some examples of when thats used?!)

theres probably some other things i'm overlooking, but thats the obvious stuff. I think I now understand enough to be able to cook up a strawman patch for this stuff, including the config time settings.in stuff, the piece that currently goes into the magic Config.hs hack that now should move into settings, and some of the dynflags / settings stuff.

I'll try to hack it out tomorrow late afternoon. (though if someone beats me to it, they're welcome, i've never messed with autoconf before, though I think i can cargo cult write the correct changes there,)

comment:3 Changed 6 years ago by carter

heres my current notes, which sort of restate my prior comment

NOTES

from config.mk.in

CONTEXT_DIFF        = @ContextDiffCmd@
CP          = cp
# It's not easy to separate the CPP program from its flags, as
# AC_PROG_CPP defines CPP as "/usr/bin/gcc -E"
CPP         = @CPP@ @CPPFLAGS@
#
# RAWCPP_FLAGS are the flags to give to cpp (viz, gcc -E) to persuade it to
# behave plausibly on Haskell sources.
#
# Clang in particular is a bit more annoying, so we suppress some warnings.
RAWCPP_FLAGS            = -undef -traditional
ifeq "$(CC_CLANG_BACKEND)" "1"
RAWCPP_FLAGS           += -Wno-invalid-pp-token -Wno-unicode -Wno-trigraphs
endif

RAWCPP is then used in compiler/ghc.mk to define

    @echo 'cRAWCPP_FLAGS         :: String'                             >> $@
    @echo 'cRAWCPP_FLAGS         = "$(RAWCPP_FLAGS)"'                   >> $@

which is then used in compiler/main/SysTools.lhs to do

  234         gcc_prog <- getSetting "C compiler command"
  235:        gcc_args_str <- getSetting "C compiler flags"
  ....

       let cpp_prog  = gcc_prog
289             cpp_args  = Option "-E"
290                       : map Option (words cRAWCPP_FLAGS)
291:                     ++ gcc_args

and then intiizes (still in SysTools)

                   sPgm_P   = (cpp_prog, cpp_args),

then in compiler/main/DynFlags.hs

    1807: setPgmP   f = let (pgm:args) = words f in alterSettings (\s -> s { sPgm_P   = (pgm, map Option args)})

and then still in compiler/main/DynFlags.hs

     2144:   , Flag "pgmP"           (hasArg setPgmP)

comment:4 Changed 6 years ago by carter

ok, I htink i now understand enough of the autoconf to post a link to a strawman patch shortly.

comment:5 Changed 6 years ago by carter

heres a WIP patch, haven't tested it yet, and theres probably stuff i'm overlooking. (also I don't know m4 /autoconf :) )

https://github.com/cartazio/ghc/compare/cpp_settings

comment:6 Changed 6 years ago by carter

The patch nearly works, I'm having trouble getting m4/autconf to work quite right, could someone who's more familiar with it chime in? I *think* once the stuff involving "$WhatIsHaskellCPPCalled" and "$WhatAreCPPFlags"

is fixed.

I'm somehow not using the autconf quite correctly to properly instantiate the CPP cmd and CPP Flags fields of settings correctly. Some help would be appreciated!

i've also done a test build where i by hand set

 ("Haskell CPP command","/Users/carter/bin/gcc"),
 ("Haskell CPP flags","-E -undef -traditional" ),

in my "settings" file after the configure step ran to validate the rest of the build process, and that works (ie testing that if i set it by hand, the new /DynFlags.hs etc logic behaves correctly). Namely that I manage to get stage2 to build correctly

sooo, I think all thats left is figuring out what i'm doing wrong on the m4/ autoconf front

comment:7 Changed 6 years ago by carter

comment:8 in reply to:  7 Changed 6 years ago by trommler

Replying to carter:

the patch can be found via https://github.com/cartazio/ghc/compare/cpp_settings

I sent a pull request to your github repository. See https://github.com/cartazio/ghc/pull/1

Last edited 6 years ago by trommler (previous) (diff)

comment:9 Changed 6 years ago by carter

thanks Peter! (merged it in) that solves one set of problems, next we need to understand why 1) when i set --with-gcc=clang, the clang variant of the CPP flags isn't chosen 2) while the cpp: foo, cpp-flags : blah info is printed in the configure script summary, it isn't substituted into the settings file! the CPP field is empty, and the CPP-Flags field @variableName@ isn't even substituted away. I suspect I'm not understanding how to use AC_SUBST([blah]) correctly

comment:10 Changed 6 years ago by carter

Status: newpatch

ok, the configure setup pieces now work! (thanks peter!)

one last thing that peter has suggested, and I agree with, is that we should add that using --with-cpp=foo then requires you to specify a --with-cpp-flags=blah (because theres no general way to determine the right flags, so they must be user supplied if you're specifying your own cpp)

comment:11 Changed 6 years ago by carter

So i think this patch *works* now. Feedback appreciated / welcome!

comment:12 Changed 6 years ago by carter

by works i mean I've tested it :)

comment:13 Changed 6 years ago by carter

I've made some improvements and simplifications based upon Austin's excellent feedback.

the logic now correctly sets the haskell cpp-flags automatically when the selected cpp program is one of gcc, clang, or cpphs, and issues a warning if its a different program from those three and

suggest explicitly setting --with-cpp-flags=FLAGS

https://github.com/cartazio/ghc/compare/cpp_settings

comment:14 Changed 6 years ago by carter

so as a way of "fuzzing" this patch, i'm trying out doing stuff like

./configure --with-gcc=clang --with-cpp=gcc and ./configure --with-gcc=gcc --with-cpp=cpphs and ./configure --with-gcc=gcc --with-cpp=clang

(which is perhaps a bit daft, but seems to be a good way to understand what i'm not doing right for the build system :) )

comment:15 Changed 6 years ago by carter

so it seems i'm designing this a bit wrong, or at the very least, wrong for stage0

comment:16 Changed 6 years ago by carter

here an error from ./configure --with-gcc=clang --with-cpp=gcc

(where i try to add the cpp flags to mk/config.in with a patch i've not pushed yet)

Writing new package config file... done.
"inplace/bin/ghc-cabal" configure libraries/terminfo dist-boot "" --with-ghc="/usr/bin/ghc" --with-ghc-pkg="/usr/bin/ghc-pkg"  --package-db=/Users/carter/Desktop/repoScratcher/ghc/libraries/bootstrapping.conf --disable-library-for-ghci --enable-library-vanilla --disable-library-profiling --disable-shared --with-hscolour="/Users/carter/Library/Haskell/bin/HsColour" --configure-option=CFLAGS=" -m64 -fno-stack-protector   " --configure-option=LDFLAGS=" -m64  " --configure-option=CPPFLAGS=" -m64 -E -undef -traditional   "   --constraint "Cabal == 1.18.1.3"   --constraint "hpc == 0.6.0.1"   --constraint "bin-package-db == 0.0.0.0"   --constraint "hoopl == 3.10.0.0"   --constraint "transformers == 0.3.0.0"   --constraint "terminfo == 0.4.0.0" --with-gcc="gcc" --configure-option=--with-cc="gcc" --with-ar="/usr/bin/ar" --with-ranlib="ranlib" --with-alex="/Users/carter/Library/Haskell/bin/alex" --with-happy="/Users/carter/Library/Haskell/bin/happy"
Configuring terminfo-0.4.0.0...
configure: WARNING: unrecognized options: --with-compiler, --with-gcc
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/Users/carter/Desktop/repoScratcher/ghc/libraries/terminfo':
configure: error: C compiler cannot create executables
See `config.log' for more details
make[1]: *** [libraries/terminfo/dist-boot/package-data.mk] Error 77
make: *** [all] Error 2

with the associated log being

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by Haskell terminfo package configure 0.2, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure --with-compiler=ghc CFLAGS= -m64 -fno-stack-protector    LDFLAGS= -m64   CPPFLAGS= -m64 -E -undef -traditional    --with-cc=gcc --with-gcc=/Users/carter/bin/gcc

## --------- ##
## Platform. ##
## --------- ##

hostname = Carters-MacBook-Air.local
uname -m = x86_64
uname -r = 13.0.0
uname -s = Darwin
uname -v = Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64

/usr/bin/uname -p = i386
/bin/uname -X     = unknown

/bin/arch              = unknown
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = Mach kernel version:
	 Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64
Kernel configured for up to 4 processors.
2 processors are physically available.
4 processors are logically available.
Processor type: i486 (Intel 80486)
Processors active: 0 1 2 3
Primary memory available: 4.00 gigabytes
Default processor set: 183 tasks, 925 threads, 4 processors
Load average: 1.99, Mach factor: 2.00
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /Users/carter/.opam/system/bin
PATH: /opt/local/bin
PATH: /Users/carter/bin
PATH: /usr/local/Cellar/smlnj/110.74/libexec/bin
PATH: /Applications/Postgres.app/Contents/MacOS/bin
PATH: /Users/carter/.scripts
PATH: /Users/carter/Library/Haskell/bin
PATH: /Users/carter/docTemplates
PATH: /usr/local/share/npm/bin
PATH: /usr/local/share/python
PATH: /usr/local/Cellar/ruby/1.9.3-p194/bin
PATH: /usr/local/bin
PATH: /usr/local/sbin
PATH: /usr/local/cuda/bin
PATH: /usr/bin
PATH: /bin
PATH: /usr/sbin
PATH: /sbin
PATH: /usr/local/bin
PATH: /opt/X11/bin
PATH: /usr/texbin
PATH: /usr/texbin


## ----------- ##
## Core tests. ##
## ----------- ##

configure:2117: checking for gcc
configure:2144: result: gcc
configure:2373: checking for C compiler version
configure:2382: gcc --version >&5
i686-apple-darwin11-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:2393: $? = 0
configure:2382: gcc -v >&5
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /Volumes/Media/Builds/gcc-5666.3/build/obj/src/configure --disable-checking --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++,fortran --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)
configure:2393: $? = 0
configure:2382: gcc -V >&5
gcc-4.2: argument to `-V' is missing
configure:2393: $? = 1
configure:2382: gcc -qversion >&5
i686-apple-darwin11-gcc-4.2.1: no input files
configure:2393: $? = 1
configure:2413: checking whether the C compiler works
configure:2435: gcc  -m64 -fno-stack-protector     -m64 -E -undef -traditional     -m64   conftest.c  >&5
# 1 "conftest.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "conftest.c"









int
main ()
{

  ;
  return 0;
}
configure:2439: $? = 0
configure:2477: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "Haskell terminfo package"
| #define PACKAGE_TARNAME "terminfo"
| #define PACKAGE_VERSION "0.2"
| #define PACKAGE_STRING "Haskell terminfo package 0.2"
| #define PACKAGE_BUGREPORT "judah dot jacobson at gmail dot com"
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:2482: error: in `/Users/carter/Desktop/repoScratcher/ghc/libraries/terminfo':
configure:2484: error: C compiler cannot create executables
See `config.log' for more details

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_env_CC_set=
ac_cv_env_CC_value=
ac_cv_env_CFLAGS_set=set
ac_cv_env_CFLAGS_value=' -m64 -fno-stack-protector   '
ac_cv_env_CPPFLAGS_set=set
ac_cv_env_CPPFLAGS_value=' -m64 -E -undef -traditional   '
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=set
ac_cv_env_LDFLAGS_value=' -m64  '
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_prog_ac_ct_CC=gcc

## ----------------- ##
## Output variables. ##
## ----------------- ##

CC='gcc'
CFLAGS=' -m64 -fno-stack-protector   '
CPP=''
CPPFLAGS=' -m64 -E -undef -traditional   '
DEFS=''
ECHO_C='\c'
ECHO_N=''
ECHO_T=''
EGREP=''
EXEEXT=''
GREP=''
LDFLAGS=' -m64  '
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
OBJEXT=''
PACKAGE_BUGREPORT='judah dot jacobson at gmail dot com'
PACKAGE_NAME='Haskell terminfo package'
PACKAGE_STRING='Haskell terminfo package 0.2'
PACKAGE_TARNAME='terminfo'
PACKAGE_URL=''
PACKAGE_VERSION='0.2'
PATH_SEPARATOR=':'
SHELL='/bin/sh'
TERMINFO_INCLUDES=''
TERMINFO_INCLUDE_DIRS=''
TERMINFO_LIB=''
TERMINFO_LIB_DIRS=''
ac_ct_CC='gcc'
bindir='${exec_prefix}/bin'
build_alias=''
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='NONE'
host_alias=''
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='NONE'
program_transform_name='s,x,x,'
psdir='${docdir}'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "Haskell terminfo package"
#define PACKAGE_TARNAME "terminfo"
#define PACKAGE_VERSION "0.2"
#define PACKAGE_STRING "Haskell terminfo package 0.2"
#define PACKAGE_BUGREPORT "judah dot jacobson at gmail dot com"
#define PACKAGE_URL ""

configure: exit 77

comment:17 Changed 6 years ago by carter

HVR also shared a wee patch that may be needed for some of the cleanup on this http://lpaste.net/99054

comment:18 Changed 6 years ago by joelteon

It seems like this patch hardcodes /usr/bin/gcc to be the compiler during the build of in-tree ghc-pwd, which results in a lot of build-related headaches on OSX, since /usr/bin/gcc is clang pretending to be gcc.

comment:19 Changed 6 years ago by carter

carter repoScratcher/ghc ‹cpp_settings*› » ag /usr/bin/gcc
compiler/main/SysTools.lhs
154:     extra_ghc_opts = ["-pgmc/usr/bin/gcc","-pgml${topdir}/bin/unlit", ... etc.],


mk/config.mk.in
619:# AC_PROG_CPP defines CPP as "/usr/bin/gcc -E"

rts/ghc.mk
526:# gcc as its preprocessor. If gcc isn't at /usr/bin/gcc, or we need

gives some fixme locations

@joelton, sounds like your test build was just with master, Try the CPP settings branch WIP :) (mind you, still broken, but maybe less so than unpatched would be)

comment:20 in reply to:  16 ; Changed 6 years ago by trommler

Replying to carter: OK, I see what the issue is:

here an error from ./configure --with-gcc=clang --with-cpp=gcc

(where i try to add the cpp flags to mk/config.in with a patch i've not pushed yet)

[...] with the associated log being

 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.
 
 It was created by Haskell terminfo package configure 0.2, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   $ ./configure --with-compiler=ghc CFLAGS= -m64 -fno-stack-protector    LDFLAGS= -m64   CPPFLAGS= -m64 -E -undef -traditional    --with-cc=gcc --with-gcc=/Users/carter/bin/gcc
 
 [...]
 configure:2413: checking whether the C compiler works
 configure:2435: gcc  -m64 -fno-stack-protector     -m64 -E -undef -traditional     -m64   conftest.c  >&5
[...]

$CPPFLAGS will be used in compiler commands too. The compiler command configure tests is something like $CC $CFLAGS $CPPFLAGS $LDFLAGS.

-E must not be part of CPPFLAGS but CPP should be set to $CC -E unless you specify the preprocessor (/usr/bin/cpp on my system) directly. If you call the preprocessor through a compiler driver like gcc or clang then $CPP should include the -E flag. I think the user should be required to specify --with-cpp="gcc -E" rather than --with-cpp=gcc and likewise for clang.

Last edited 6 years ago by trommler (previous) (diff)

comment:21 in reply to:  20 Changed 6 years ago by trommler

Replying to trommler:

Replying to carter: OK, I see what the issue is:

here an error from ./configure --with-gcc=clang --with-cpp=gcc

(where i try to add the cpp flags to mk/config.in with a patch i've not pushed yet)

[...] with the associated log being

 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.
 
 It was created by Haskell terminfo package configure 0.2, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   $ ./configure --with-compiler=ghc CFLAGS= -m64 -fno-stack-protector    LDFLAGS= -m64   CPPFLAGS= -m64 -E -undef -traditional    --with-cc=gcc --with-gcc=/Users/carter/bin/gcc
 
 [...]
 configure:2413: checking whether the C compiler works
 configure:2435: gcc  -m64 -fno-stack-protector     -m64 -E -undef -traditional     -m64   conftest.c  >&5
[...]

Wait a minute! Why are those flags passed to the C compiler at all. I thought they were meant to be used by ghc to preprocess Haskell files?!

comment:22 Changed 6 years ago by carter

@trommler, hrmmmm

I was doing the command + flag design because that maps to the current GHC dynflags model,

perhaps it'd be better to have distinct CPP programs for C and Haskell in the build model? Does this mean they should be disjoint in GHC's dynflags too?

comment:23 in reply to:  22 Changed 6 years ago by trommler

Replying to carter:

@trommler, hrmmmm

I was doing the command + flag design because that maps to the current GHC dynflags model,

... and I tried to set gcc -E for cpp and failed. GHC tries to exec gcc -E then and that does not exist. We could fix that by parsing CPP in configure (once) or in GHC (at every invocation) or stick with your original design (which I did for now in my branch).

perhaps it'd be better to have distinct CPP programs for C and Haskell in the build model?

I think, yes.

I don't fully understand what --with-gcc is supposed to do. It changes the C compiler half way through configure and then even sets CC. The wiki says it specified the compiler used to compile C files but GHC will use its own C compiler. To me that sounds like we should actually set the C compiler using CC=<C compiler> on the configure command line. But I might be missing a major issue here.

Does this mean they should be disjoint in GHC's dynflags too?

Do you mean there should be another set of CPP and CPPFLAGS for C files? I don't think so.

comment:24 Changed 6 years ago by carter

OK, theres several things going on here, and i'm going to lay it out so i can be less confused (and hopefully i can understand your ideas better), and perhaps you can be less confused to! (though you are pointing out a real bug i think too)

1) theres currently (and unfortunately), some places in the build system where GCC is hard coded.

a) historically the only C compiler GHC would support was GCC. thus historically --with-gcc was just a way of picking which gcc to use system with more than one gcc installed (I think, i could be wrong here) b) the clang support for ghc is pretty recent, and theres definitely a few corners that still need cleanup

the CPP mode for Haskell code is always "traditional mode", so the flags passed to haskell CPP != the flags passed to C, though there may be some overlap

theres also some matters relating to stage0 vs stage1 vs stage2 ghcs that is also going on too I think...

Last edited 6 years ago by carter (previous) (diff)

comment:25 Changed 6 years ago by carter

hrmm, so in theory if we wanna clean up the cruft, we'd want to make it --with-cc rather than --with-gcc

comment:26 Changed 6 years ago by carter

Priority: normalhighest
Version: 7.77.8.1-rc2

We need this or something very much like it to get into 7.8.1/7.8.2, or we'll have some pain transparently supporting a range of mac OS versions in OS X, future weird config situations, or even nicely baking CPPHS into ghc distro as a near term solution setting priority higher accordingly.

I will try to make time this week to fix things up, but I've a few other professional obligations that might block that.

comment:27 Changed 6 years ago by thoughtpolice

Status: patchnew

comment:28 Changed 6 years ago by tibbe

Cc: tibbe added

comment:29 Changed 6 years ago by carter

I've rebased it against 7.8 tip, will see if i can push things along this afternooon https://github.com/cartazio/ghc/compare/ghc:ghc-7.8...cpp_settings

comment:30 Changed 6 years ago by carter

ok, also fixed it up so it properly has all the commit data

comment:31 Changed 6 years ago by tibbe

Nice. Could the patches be rebased for ease of reviewing?

comment:32 Changed 6 years ago by carter

just rebased it

(I think i did the rebase correctly, but I have no clueeee)

theres two parts to this patch 1) making ghc have its CPP settable / configable in the settings file. This seems work 2) allowing ghc to be built with different CC and CPP. this doesn't quite work, but it does allow use to set it explicitly, and initializes the right fields in settings, but theres CPP stuff lying all around the build system that i'm still unfamiliar with and haven't been address here.

this also add --with-cpp and --with-cpp-flags flags to configure

currently for building things, -with-cpp and with-gcc need to point to the same program or various bugs in the build system start creeping up! It also seems to have surfaced at least one new race condition in the build system. I will try to dig into that more later.

What i also need to finally test properly is if the resulting ghc can handle its settings file being pointed to cpphs

Also i thought i had added logic to handle cpphs as a cpp option, but it looks like i lost track of that commit

comment:33 Changed 6 years ago by carter

basically: this has surfaced some design issues in the build system, but thats orthogonal to the work to enable a bindist to be bundled with cpphs

comment:34 Changed 6 years ago by carter

i'm doing a quick make test cycle to check that this all works out

comment:35 Changed 6 years ago by carter

theres also a bunch of places where GCC is hard coded in the build system still

comment:36 Changed 6 years ago by carter

tl;dr

the current state is such that if you do mix and match any combo of clang / gcc (and anything not them ) for with-gcc and with-cpp, you'll hit build failures because of how the libraries make files are setup.

Also it currently doesn't have the right logic for CPPHS as an alternative to gcc/clang in the build side. BUT you can put that info in yourself in the settings file.

comment:37 Changed 6 years ago by carter

ok, 7.8 Tip builds fine, but the rebased patches on top, https://github.com/cartazio/ghc/compare/ghc:ghc-7.8...cpp_settings somehow totally break building libraries. And i'm quite stumped about how they could be causing that. (partly because i'm not familiar with that part of the build system in details)

I'm not going to have time to revisit this for another week or likely two (perhaps three depending on things i cant predict). If someone wants to figure out whats going on or write a better patch in the mean time, please do!

comment:38 Changed 6 years ago by carter

Huh, reverting HVR's patch that touches compiler/main/DynFlags.hs and compiler/main/DriverPipeline.hs allows build to run fine when CPP==CC (ie both GCC or Both clang)

comment:39 Changed 6 years ago by carter

i'm now (post build) edit of the settings file to do

("Haskell CPP command","cpphs"), ("Haskell CPP flags","--cpp"),

and running make test

comment:40 Changed 6 years ago by carter

OVERALL SUMMARY for test run started at Tue Mar 18 14:58:23 2014 EDT
 0:54:39 spent to go through
    3903 total tests, which gave rise to
   15250 test cases, of which
   11683 were skipped

      42 had missing libraries
    3444 expected passes
      62 expected failures

       3 caused framework failures
       1 unexpected passes
      15 unexpected failures

Unexpected passes:
   codeGen/should_run  cgrun071 (normal)

Unexpected failures:
   driver/objc    objc-hi [exit code non-0] (normal)
   driver/objc    objcpp-hi [exit code non-0] (normal)
   perf/compiler  T1969 [stat not good enough] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/compiler  T3294 [stat not good enough] (normal)
   perf/compiler  T4801 [stat not good enough] (normal)
   perf/compiler  T5030 [stat not good enough] (normal)
   perf/compiler  T5321FD [stat not good enough] (normal)
   perf/compiler  T5321Fun [stat not good enough] (normal)
   perf/compiler  T5631 [stat not good enough] (normal)
   perf/compiler  T5642 [stat not good enough] (normal)
   perf/compiler  T5837 [stat not good enough] (normal)
   perf/compiler  T6048 [stat not good enough] (optasm)
   perf/compiler  T783 [stat not good enough] (normal)
   perf/compiler  parsing001 [stat not good enough] (normal)

is the result of make test

comment:41 Changed 6 years ago by carter

so current status of my branch:

  • the build system currently needs some love to handle none gcc/clang CPP, (and fails in way i'm not very good at understanding when i try using cpphs). It seems --with-gcc=gcc-4.8 --with-hs-cpp=clang works, i'm in the midst of testing that and the opposite presently
  • subject being build with one of those two, we can then patch the settings file to point to cpphs and the resulting configured GHC passes the test suite
Last edited 6 years ago by carter (previous) (diff)

comment:42 Changed 6 years ago by carter

Status: newpatch

Ok, it all works!

some notes

a) you still have to build using clang / gcc as the choice in cpp for GHC building, but the hooks are there to set it otherwise with --with-hs-cpp and --with-hs-cpp-flags, but various hard codings in the build system prevent ghc builds from succeeding. Its just those issue are more discoverable :-)

b) ghc will correctly use the new fields in the settings file, as if invoked with -pgmP and -optP being passed in. I managed to build all of lens's deps successfully using CPPHS! I did hit a bug in CPPHS wrt building Lens 4.0.7, but again thats an issue with using CPPHS, not with the patch

c) Theres still lots of hardcoded GCC / CPP things in the build system

d) this gives the core of whats needed to provide a bindist that doesn't use clang/gcc for CPP, though because of other issues that need to be address (and should become a new ticket perhaps), you can't as yet build GHC with a CPP thats not clang or gcc

is there any documentation i should add? (either to the places patched or the documentation somewhere in the manual / wiki)

comment:44 Changed 6 years ago by carter

based on the discussion on the lens thread, i have changed the CPPHS case in the configure.ac for --with-hs-cpp set the Haskell CPP flags to "--cpp --traditional"

mind you, until those other hard coded cases in the build system are addressed (simon m said he'll be helping resolve some of them), doesn't change rest of patch :)

comment:45 Changed 6 years ago by carter

somehow i'm tripping a "make has restarted 3 times" bug when I run validate, heres the tail of the transcript

nb: i did configure with --with-gcc=gcc-4.8 and --with-hs-cpp=gcc-4.8

heres a link to the tail of the console output http://lpaste.net/101605

the relevant last few lines being

Writing new package config file... done.
"inplace/bin/ghc-cabal" configure ghc stage1 "" --with-ghc="/usr/local/bin/ghc" --with-ghc-pkg="/usr/local/bin/ghc-pkg"  --flags=stage1 --constraint "ghc == 7.8.0.20140322" --package-db=/Users/carter/Desktop/repoScratcher/ghc/libraries/bootstrapping.conf --disable-library-for-ghci --disable-library-vanilla --disable-library-profiling --disable-shared --with-hscolour="/Users/carter/.cabal/bin/HsColour" --configure-option=CFLAGS="-Wall -m64 -fno-stack-protector    -Werror=unused-but-set-variable -Wno-error=inline" --configure-option=LDFLAGS=" -m64  " --configure-option=CPPFLAGS=" -m64  "   --constraint "Cabal == 1.18.1.3"   --constraint "hpc == 0.6.0.1"   --constraint "bin-package-db == 0.0.0.0"   --constraint "hoopl == 3.10.0.0"   --constraint "transformers == 0.3.0.0"   --constraint "terminfo == 0.4.0.0" --with-gcc="gcc-4.8" --configure-option=--with-cc="gcc-4.8" --with-ar="/usr/bin/ar" --with-ranlib="ranlib" --with-alex="/Users/carter/.cabal/bin/alex" --with-happy="/Users/carter/.cabal/bin/happy"
Configuring ghc-bin-7.8.0.20140322...
Warning: 'data-dir: ..' is a relative path outside of the source tree. This
will not work when generating a tarball with 'sdist'.
ghc.mk:100: *** Make has restarted itself 2 times; is there a makefile bug? See http://ghc.haskell.org/trac/ghc/wiki/Building/Troubleshooting#Makehasrestarteditself3timesisthereamakefilebug for details.  Stop.

comment:46 Changed 6 years ago by carter

I just got validate --fast --no-dphworking, with configure as above -with-gcc=gcc-4.8 and --with-hs-cpp=gcc-4.8


Unexpected results from:
TEST="cgrun071 objc-hi objcpp-hi haddock.base T3064"

OVERALL SUMMARY for test run started at Sat Mar 22 23:57:35 2014 EDT
 0:27:29 spent to go through
    3903 total tests, which gave rise to
   15250 test cases, of which
   11683 were skipped

      28 had missing libraries
    3472 expected passes
      62 expected failures

       0 caused framework failures
       1 unexpected passes
       4 unexpected failures

Unexpected passes:
   codeGen/should_run  cgrun071 (normal)

Unexpected failures:
   driver/objc    objc-hi [exit code non-0] (normal)
   driver/objc    objcpp-hi [exit code non-0] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/haddock   haddock.base [stat not good enough] (normal)

== Start post-testsuite package check
Timestamp 2014-03-23 03:22:17 UTC for /Users/carter/Desktop/repoScratcher/ghc/inplace/lib/package.conf.d/package.cache
Timestamp 2014-03-23 03:22:17 UTC for /Users/carter/Desktop/repoScratcher/ghc/inplace/lib/package.conf.d (same as cache)
using cache: /Users/carter/Desktop/repoScratcher/ghc/inplace/lib/package.conf.d/package.cache
== End post-testsuite package check
-------------------------------------------------------------------
Oops!  Looks like you have some unexpected test results or framework failures.
Please fix them before pushing/sending patches.
-------------------------------------------------------------------

i'll try to do a validate run with clang, the gcc-4.8 will barf on objective C.

comment:47 Changed 6 years ago by carter

I just did a ./configure --with-gcc=clang ; ./validate --no-dph and i got the following error at the end

= End post-install package check
===--- building phase 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=0 phase_0_builds
make[1]: Nothing to be done for `phase_0_builds'.
===--- building phase 1
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=1 phase_1_builds
make[1]: Nothing to be done for `phase_1_builds'.
===--- building final phase
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=final validate_build_xhtml
cd libraries/xhtml && "/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc" --make Setup
[1 of 1] Compiling Main             ( Setup.hs, Setup.o )
Linking Setup ...
cd libraries/xhtml && ./Setup configure --with-ghc="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc" --with-haddock="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/haddock" --enable-shared --disable-library-vanilla --disable-library-prof --global --builddir=dist-bindist --prefix="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir"
Configuring xhtml-3000.2.1...
cd libraries/xhtml && ./Setup build   --builddir=dist-bindist
Building xhtml-3000.2.1...
Preprocessing library xhtml-3000.2.1...
ghc: could not execute: 
make[1]: *** [validate_build_xhtml] Error 1
make: *** [validate_build_xhtml] Error 2

i'm not sure how to debug whats happening there!

Last edited 6 years ago by carter (previous) (diff)

comment:48 Changed 6 years ago by carter

Owner: set to carter

i just rebased my cpp-settings branch on top of todays ghc-7.8 tip, validate seems to be succeeding. Will paste test results shortly nb: im expect the objc tests to fail because my gcc is gcc-4.8

https://github.com/cartazio/ghc/compare/ghc:ghc-7.8...rb_cpp_settings

so the current patch set is rb_cpp_settings

Last edited 6 years ago by carter (previous) (diff)

comment:49 Changed 6 years ago by carter

== Start post-install package check
Timestamp 2014-03-23 17:32:46 UTC for /Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/package.conf.d/package.cache
Timestamp 2014-03-23 17:32:46 UTC for /Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/package.conf.d (same as cache)
using cache: /Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/package.conf.d/package.cache
== End post-install package check
===--- building phase 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=0 phase_0_builds
make[1]: Nothing to be done for `phase_0_builds'.
===--- building phase 1
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=1 phase_1_builds
make[1]: Nothing to be done for `phase_1_builds'.
===--- building final phase
/Applications/Xcode.app/Contents/Developer/usr/bin/make -r --no-print-directory -f ghc.mk phase=final validate_build_xhtml
cd libraries/xhtml && "/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc" --make Setup
[1 of 1] Compiling Main             ( Setup.hs, Setup.o )
Linking Setup ...
cd libraries/xhtml && ./Setup configure --with-ghc="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc" --with-haddock="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/haddock" --enable-shared --disable-library-vanilla --disable-library-prof --global --builddir=dist-bindist --prefix="/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir"
Configuring xhtml-3000.2.1...
cd libraries/xhtml && ./Setup build   --builddir=dist-bindist
Building xhtml-3000.2.1...
Preprocessing library xhtml-3000.2.1...
ghc: could not execute: 
make[1]: *** [validate_build_xhtml] Error 1
make: *** [validate_build_xhtml] Error 2

I'm thinking something relating to validate_build_xhtml in ghc/ghc.mk is the cuplrit

Last edited 6 years ago by carter (previous) (diff)

comment:50 Changed 6 years ago by carter

the results of the the test suite are

OVERALL SUMMARY for test run started at Sun Mar 23 13:57:15 2014 EDT
 0:50:45 spent to go through
    3930 total tests, which gave rise to
   15438 test cases, of which
   11846 were skipped

      28 had missing libraries
    3496 expected passes
      63 expected failures

       0 caused framework failures
       1 unexpected passes
       4 unexpected failures

Unexpected passes:
   codeGen/should_run  cgrun071 (normal)

Unexpected failures:
   driver/objc    objc-hi [exit code non-0] (normal)
   driver/objc    objcpp-hi [exit code non-0] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/haddock   haddock.base [stat not good enough] (normal)

this makes me think the issue lies with that xhtml validation step

comment:51 Changed 6 years ago by carter

PROGRESSS!

the relevant step of validate that was failing was was cd libraries/xhtml && ./Setup build --builddir=dist-bindist

I took the liberty of running this by hand:

carter libraries/xhtml ‹fb9e0bb*› » ./Setup build   --builddir=dist-bindist -v3                                                   1 ↵
Component build order: library
creating dist-bindist/build
creating dist-bindist/build/autogen
Building xhtml-3000.2.1...
Preprocessing library xhtml-3000.2.1...
Building library...
("/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc",["--info"])
("/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc",["--info"])
creating dist-bindist/build
Environment: [("Apple_PubSub_Socket_Render","/tmp/launch-IehmnE/Render"),("CAML_LD_LIBRARY_PATH","/Users/carter/.opam/system/lib/stublibs:/usr/local/lib/ocaml/stublibs"),("CLICOLOR","1"),("COLORFGBG","7;0"),("COMMAND_MODE","unix2003"),("DBUS_LAUNCHD_SESSION_BUS_SOCKET","/tmp/launch-kHPpAS/unix_domain_listener"),("DISABLE_AUTO_UPDATE","true"),("DISPLAY","/tmp/launch-7MeZK8/org.macosforge.xquartz:0"),("EDITOR","subl -w -n"),("GNUTERM","x11"),("GREP_COLOR","1;32"),("GREP_OPTIONS","--color=auto"),("HOME","/Users/carter"),("ITERM_SESSION_ID","w0t1p0"),("LANG","en_US.UTF-8"),("LC_CTYPE","en_US.UTF-8"),("LESS","-R"),("LOGNAME","carter"),("LSCOLORS","Gxfxcxdxbxegedabagacad"),("MANPATH","/Users/carter/.opam/system/man:"),("MYTARCACHE","/Users/carter/.tarsnap-cache/"),("MYTARKEY","/Users/carter/.secureme/tarsnap-2011MBAir13.key"),("NODE_PATH","/usr/local/lib/node_modules:/usr/local/lib/node"),("OCAML_TOPLEVEL_PATH","/Users/carter/.opam/system/lib/toplevel"),("OLDPWD","/Users/carter/Desktop/repoScratcher/ghc"),("PAGER","less"),("PATH","/Users/carter/.opam/system/bin:/opt/local/bin:/Users/carter/bin:/usr/local/Cellar/smlnj/110.74/libexec/bin:/Applications/Postgres.app/Contents/MacOS/bin:/Users/carter/.scripts:/Users/carter/.cabal/bin:/Users/carter/Library/Haskell/bin:/Users/carter/docTemplates:/usr/local/share/npm/bin:/usr/local/share/python:/usr/local/Cellar/ruby/1.9.3-p194/bin:/usr/local/bin:/usr/local/sbin:/usr/local/cuda/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/texbin:/usr/texbin"),("PKG_CONFIG_PATH","/opt/X11/lib/pkgconfig:"),("PWD","/Users/carter/Desktop/repoScratcher/ghc/libraries/xhtml"),("PYTHONPATH","/usr/local/share/python:/usr/local/lib/python:"),("SECURITYSESSIONID","186a4"),("SHELL","/bin/zsh"),("SHLVL","1"),("SSH_AUTH_SOCK","/tmp/launch-iHEzRI/Listeners"),("TERM","xterm-256color"),("TERM_PROGRAM","iTerm.app"),("TMPDIR","/var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/"),("USER","carter"),("ZSH","/Users/carter/.oh-my-zsh"),("ZSH_THEME","carter"),("_","/Users/carter/Desktop/repoScratcher/ghc/libraries/xhtml/./Setup"),("__CF_USER_TEXT_ENCODING","0x1F5:0:0"),("__CHECKFIX1436934","1")]
("/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/bin/ghc",["--make","-v","-fbuilding-cabal-package","-O","-dynamic","-fPIC","-osuf","dyn_o","-hisuf","dyn_hi","-outputdir","dist-bindist/build","-odir","dist-bindist/build","-hidir","dist-bindist/build","-stubdir","dist-bindist/build","-i","-idist-bindist/build","-i.","-idist-bindist/build/autogen","-Idist-bindist/build/autogen","-Idist-bindist/build","-optP-include","-optPdist-bindist/build/autogen/cabal_macros.h","-package-name","xhtml-3000.2.1","-hide-all-packages","-no-user-package-db","-package-db","dist-bindist/package.conf.inplace","-package-id","base-4.7.0.0-db909df045884d25c2593604bc4b7fad","-XHaskell98","-XCPP","Text.XHtml","Text.XHtml.Frameset","Text.XHtml.Strict","Text.XHtml.Transitional","Text.XHtml.Debug","Text.XHtml.Table","Text.XHtml.Strict.Attributes","Text.XHtml.Strict.Elements","Text.XHtml.Frameset.Attributes","Text.XHtml.Frameset.Elements","Text.XHtml.Transitional.Attributes","Text.XHtml.Transitional.Elements","Text.XHtml.BlockTable","Text.XHtml.Extras","Text.XHtml.Internals","-Wall"])
Glasgow Haskell Compiler, Version 7.8.0.20140323, stage 2 booted by GHC version 7.8.0.20140311
Using binary package database: /Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/package.conf.d/package.cache
Using package config file: dist-bindist/package.conf.inplace
wired-in package ghc-prim mapped to ghc-prim-0.3.1.0-ed30589de10ea47e88c54710213a68e0
wired-in package integer-gmp mapped to integer-gmp-0.5.1.0-5e6bd420a1279049d72fe1ad55e57662
wired-in package base mapped to base-4.7.0.0-db909df045884d25c2593604bc4b7fad
wired-in package rts mapped to builtin_rts
wired-in package template-haskell mapped to template-haskell-2.9.0.0-0b1690f10983f893b9e2b88305e78762
wired-in package dph-seq not found.
wired-in package dph-par not found.
Hsc static flags: 
wired-in package ghc-prim mapped to ghc-prim-0.3.1.0-ed30589de10ea47e88c54710213a68e0
wired-in package integer-gmp mapped to integer-gmp-0.5.1.0-5e6bd420a1279049d72fe1ad55e57662
wired-in package base mapped to base-4.7.0.0-db909df045884d25c2593604bc4b7fad
wired-in package rts mapped to builtin_rts
wired-in package template-haskell mapped to template-haskell-2.9.0.0-0b1690f10983f893b9e2b88305e78762
wired-in package dph-seq not found.
wired-in package dph-par not found.
*** Chasing dependencies:
Chasing modules from: *Text.XHtml,*Text.XHtml.Frameset,*Text.XHtml.Strict,*Text.XHtml.Transitional,*Text.XHtml.Debug,*Text.XHtml.Table,*Text.XHtml.Strict.Attributes,*Text.XHtml.Strict.Elements,*Text.XHtml.Frameset.Attributes,*Text.XHtml.Frameset.Elements,*Text.XHtml.Transitional.Attributes,*Text.XHtml.Transitional.Elements,*Text.XHtml.BlockTable,*Text.XHtml.Extras,*Text.XHtml.Internals
Created temporary directory: /var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/ghc15554_0
*** C pre-processor:
'' -include dist-bindist/build/autogen/cabal_macros.h -I dist-bindist/build -I dist-bindist/build -I dist-bindist/build/autogen -I dist-bindist/build -I '/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/base-4.7.0.0/include' -I '/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/integer-gmp-0.5.1.0/include' -I '/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install   dir/lib/ghc-7.8.0.20140323/include' '-D__GLASGOW_HASKELL__=708' '-Ddarwin_BUILD_OS=1' '-Dx86_64_BUILD_ARCH=1' '-Ddarwin_HOST_OS=1' '-Dx86_64_HOST_ARCH=1' '-U __PIC__' -D__PIC__ '-D__SSE__=1' '-D__SSE2__=1' -x assembler-with-cpp ./Text/XHtml.hs -o /var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/ghc15554_0/ghc15554_1.hscpp
*** Deleting temp files:
Deleting: /var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/ghc15554_0/ghc15554_1.hscpp
Warning: deleting non-existent /var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/ghc15554_0/ghc15554_1.hscpp
*** Deleting temp dirs:
Deleting: /var/folders/py/wgp_hj9d2rl3cx48yym_ynj00000gn/T/ghc15554_0
ghc: could not execute: 
/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install dir/bin/ghc
returned ExitFailure 1

Still not sure whats happening, but its a bit more explicit whatever it is

comment:52 Changed 6 years ago by carter

ghc: could not execute: 
/Users/carter/Desktop/repoScratcher/ghc/bindisttest/install dir/bin/ghc
returned ExitFailure 1

being the key bit. Though i don't claim to understand whats happening here (yet)

Last edited 6 years ago by carter (previous) (diff)

comment:53 Changed 6 years ago by carter

Owner: carter deleted

i've narrowed down whats happening a bit, but I still dont understand this last bit, and i don't have time this week to do that last piece of undirected debugging.

comment:54 Changed 6 years ago by thoughtpolice

Milestone: 7.8.17.8.2

I've looked over the patch more but can't quite see what's wrong yet. Unfortunately time is out, and I'm bumping this to 7.8.2

comment:55 Changed 6 years ago by thoughtpolice

Priority: highesthigh

comment:56 Changed 6 years ago by carter

sounds good to me. Especially since the patch is more future proofing than "needed today". Malcom's remark on devs

This looks like the key information:

> Building xhtml-3000.2.1...
> Preprocessing library xhtml-3000.2.1...
> ghc: could not execute:

Ghc is reporting that it could not execute "".  This suggests the commandline for external preprocessing is incorrectly formed.

is suggestive. I may have to revisit herbert's patch perhaps.

comment:57 Changed 5 years ago by carter

figured out what i was missing (thanks @archblob on irc for helping!), theres another configure.ac, distrib/configure.ac.in thats used for configure at install time!

http://github.com/cartazio/ghc/tree/cpp_silly-contrib-configure-in is the current patch set with that fix, will clean it up a bit, and it probably needs some refactoring and such.

seems to validate

with a few test failures

Unexpected results from:
TEST="objc-hi objcpp-hi haddock.base T3064 T5837 T6048"

OVERALL SUMMARY for test run started at Fri Mar 28 01:00:07 2014 EDT
 0:28:20 spent to go through
    3913 total tests, which gave rise to
   12832 test cases, of which
    9262 were skipped

      26 had missing libraries
    3477 expected passes
      61 expected failures

       0 caused framework failures
       0 unexpected passes
       6 unexpected failures

Unexpected failures:
   driver/objc    objc-hi [exit code non-0] (normal)
   driver/objc    objcpp-hi [exit code non-0] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/compiler  T5837 [stat not good enough] (normal)
   perf/compiler  T6048 [stat not good enough] (optasm)
   perf/haddock   haddock.base [stat not good enough] (normal)
Last edited 5 years ago by carter (previous) (diff)

comment:58 Changed 5 years ago by carter

Owner: set to carter

comment:59 Changed 5 years ago by thoughtpolice

Milestone: 7.8.27.8.3

comment:60 Changed 5 years ago by carter

i've rebased on top the 7.8.2 tip of 7.8 branch https://github.com/cartazio/ghc/compare/ghc:ghc-7.8...rb_cpp_settings

the failure matchup with using real GCC on OS X, wherein real GCC doesn't understand objective c, (plus the perf stat issues that are generally afoot on several platform)

Unexpected results from:
TEST="objc-hi objcpp-hi T5837 T3064 T4801 T6048"

OVERALL SUMMARY for test run started at Thu Apr 10 19:10:37 2014 EDT
 0:23:53 spent to go through
    3920 total tests, which gave rise to
   12853 test cases, of which
    9276 were skipped

      26 had missing libraries
    3484 expected passes
      61 expected failures

       0 caused framework failures
       0 unexpected passes
       6 unexpected failures

Unexpected failures:
   driver/objc    objc-hi [exit code non-0] (normal)
   driver/objc    objcpp-hi [exit code non-0] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/compiler  T4801 [stat not good enough] (normal)
   perf/compiler  T5837 [stat not good enough] (normal)
   perf/compiler  T6048 [stat not good enough] (optasm)

comment:61 Changed 5 years ago by carter

Blocking: 9094 added

comment:62 Changed 5 years ago by dagit

Mixing clang and gcc in the same build may lead to weird issues in some cases. I don't think it will be common, but it's something to watch out for.

Imagine this scenario: You're making an FFI binding and clang and gcc use different definitions in their respective libraries. Or stranger yet, someone has written their library to behave differently depending on whether it's compiled with clang or gcc (let's hope no one does that).

Thinking about this lead to realizing that cabal + hsc2hs exhibits a related problem: https://github.com/haskell/cabal/issues/1858

comment:63 Changed 5 years ago by carter

actually mixing clang and gcc is the only way to fix certain issues on OS X pre Xcode 5. Its imposible to correctly compile SIMD codes with xcode 4s GCC 4.2, so i've had to have custom SETUP.hs files for OS X code that invokes clang on c codes, even if ghc is using gcc for its own needs.

comment:64 Changed 5 years ago by carter

point being, similar issues arise on supported platforms even if you fix to using only clang or gcc. my pesonal GHC setup actually uses the hybrid of clang + gcc (which IS NOT the default for this patch, merely in my personal config after i built my ghc)

eg, you can't built hfsevents on mac unless you use Clang as the CC for ghc.

comment:65 Changed 5 years ago by carter

there was some further discussion on https://phabricator.haskell.org/P5 and https://github.com/cartazio/ghc/compare/ghc:ghc-7.8...fixed-cpp-7.8 has the fixed up current patch on top of current ghc 7.8 head

comment:66 Changed 5 years ago by carter

validates on linux 64bit intel with gcc and OS X 64bit with clang

Unexpected results from: TEST="T3064 T5837 T6048"

OVERALL SUMMARY for test run started at Mon Jun 23 19:20:11 2014 EDT

0:36:14 spent to go through

3963 total tests, which gave rise to

13079 test cases, of which

9461 were skipped

26 had missing libraries

3528 expected passes

61 expected failures

0 caused framework failures 0 unexpected passes 3 unexpected failures

Unexpected failures:

perf/compiler T3064 [stat not good enough] (normal) perf/compiler T5837 [stat not good enough] (normal) perf/compiler T6048 [stat not good enough] (optasm)

comment:67 Changed 5 years ago by MtnViewMark

As mentioned on phabricator, I've tested the patch and it works. This enables the Mac build to have a single build, which can be re-configured for the target machine after it has been copied onto the target machine.

This is important as otherwise the HP would have to be distributed either in two forms (one for clang, one for non-clang), or with the ghc-clang-wrapper hack, which works but is brittle.

Assuming this goes in, the next HP will have a script, activate-hs, which will allow the user to switch between installactions of the HP, and will also fix up these settings in cases where the users tool chain has switched from non-clang to clang. (This can happen on systems where the user has been using an old Xcode and then updates Xcode.)

comment:68 Changed 5 years ago by Austin Seipp <austin@…>

In 34f7e9a3c99850859901ca74370f55f1d4e2279a/ghc:

Control CPP through settings file (#8683)

Summary:
Allow the CPP program and flag choices for GHC
be configured via the the ghc settings file

Test Plan: ran validate yesterday

Reviewers: hvr, austin, mzero, simonmar

Reviewed By: austin, mzero, simonmar

Subscribers: mzero, simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D26

comment:69 Changed 5 years ago by thoughtpolice

Resolution: fixed
Status: patchclosed

comment:70 Changed 5 years ago by Austin Seipp <austin@…>

In 460eebec65811c6a7bbe11645df322dda868e80d/ghc:

Remove RAWCPP_FLAGS

Summary:
#9094 mentions to "remove the RAW_CPP bits from the ghc build system because
they're not longer needed", "once the CPP settings ticket is merged #8683"

#8683 was merged with 34f7e9a3c99850859901ca74370f55f1d4e2279a, Phab:D26.

Test Plan: harbormaster

Reviewers: carter, austin

Reviewed By: austin

Subscribers: simonmar, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D240

GHC Trac Issues: #9094

comment:71 Changed 5 years ago by mietek

Does this mean it is now possible to bootstrap cabal-install without specifying --no-doc? (#9174)

comment:72 Changed 5 years ago by thomie

Blocking: 9094 removed
Note: See TracTickets for help on using tickets.