Opened 4 years ago

Closed 4 years ago

Last modified 11 months ago

#10726 closed task (fixed)

Upgrade MingW-w64 distributions for windows

Reported by: Phyx- Owned by:
Priority: high Milestone: 8.0.1
Component: Build System (make) Version: 7.11
Keywords: Cc: RyanGlScott, snoyberg, joozek78, bgamari, thomie, Elieux
Operating System: Windows Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: #9218 #9014 #10435 #11223 Differential Rev(s): Phab:D1123
Wiki Page:

Description

This is the second part of splitting up #9218.

The goal is to move GHC x86 from MingW to MingW-w64 and to upgrade both the x86 and the x86_64 toolchains shipped with GHC to newer versions.

Attachments (3)

add_missing_symbols.patch (821 bytes) - added by lukexi 4 years ago.
logfloat_output.txt (4.5 KB) - added by RyanGlScott 4 years ago.
runghc -v LogFloat.hs
audioexample_output.txt (7.9 KB) - added by RyanGlScott 4 years ago.
runghc -v -no-hs-main AudioExample.hs

Download all attachments as: .zip

Change History (68)

comment:1 Changed 4 years ago by Phyx-

Chosen MinGW-w64 build for this are the dongsheng daily builds from the MinGW-w64 repository. Which avoids the issues mentioned in #9014 .

The GCC versions will be upgraded to 5.1.1.

To be complete the dongsheng builds contain:

binutils 2.25
gdb 7.9.1
make 4.1
gmp 6.0.0
mpfr 3.1.3
mpc 1.0.3
isl 0.14.1
mingw-w64 master
gcc 5.1.1

comment:2 Changed 4 years ago by Phyx-

Differential Rev(s): Phab:D1123
Status: newpatch

comment:3 Changed 4 years ago by awson

Could we, perhaps, use Msys2 builds for this? They are of a high quality and pretty well supported. Also they have gcc-5.2 and, most important, recently released binutils-2.25.1 which have a very important bug (wrong linking of 64-bit import libraries generated by MS linker) fixed.

As of today the transitive closure of packages required is (I believe) this:

mingw-w64-x86_64-crt-git-5.0.0.4531.49c7046-1
mingw-w64-x86_64-winpthreads-git-5.0.0.4538.78dca70-1
mingw-w64-x86_64-headers-git-5.0.0.4531.49c7046-1
mingw-w64-x86_64-libwinpthread-git-5.0.0.4538.78dca70-1
mingw-w64-x86_64-zlib-1.2.8-8
mingw-w64-x86_64-isl-0.14.1-2
mingw-w64-x86_64-mpc-1.0.3-2
mingw-w64-x86_64-mpfr-3.1.3.p0-2
mingw-w64-x86_64-gmp-6.0.0-3
mingw-w64-x86_64-gcc-libs-5.2.0-3
mingw-w64-x86_64-binutils-2.25.1-1
mingw-w64-x86_64-libiconv-1.14-5
mingw-w64-x86_64-gcc-5.2.0-3

The new repo of Msys2 packages is http://repo.msys2.org/mingw.

comment:4 Changed 4 years ago by Phyx-

I have no problems with that. I'll try to modify the script tomorrow and let validate run over night.

I will keep them all separate packages and download each individually. I think that gives us a bit more control over updating the packages. Unless you think we should group them in one large tar file?

Last edited 4 years ago by Phyx- (previous) (diff)

comment:5 Changed 4 years ago by RyanGlScott

Cc: RyanGlScott added

comment:6 Changed 4 years ago by Phyx-

I've updated the toolchains to 5.2.0. Validate on x86_64 looks good again, still fixing a few tests on x86. See Phabricator.

comment:7 Changed 4 years ago by Ben Gamari <ben@…>

In 7b211b4/ghc:

Upgrade GCC to 5.2.0 for Windows x86 and x86_64

This patch does a few things

- Moved GHC x86 to MinGW-w64 (Using Awson's patch)
- Moves Both GHCs to MSYS2 toolchains
- Completely removes the dependencies on the git tarball repo
  - Downloads only the required tarball for the architecture for
    which we are building
  - Downloads the perl tarball is missing as well
  - Fixed a few bugs in the linker to fix tests on Windows

The links currently point to repo.msys2.org and GitHub, it might be
more desirable to mirror them on
http://downloads.haskell.org/~ghc/mingw/ as with the previous patch
attempt.

For more details on what the MSYS2 packages I include see #10726
(Awson's comment). but it should contain all we need
and no python or fortran, which makes the uncompressed tar a 1-2
hundreds mb smaller.

The `GCC 5.2.0` in the package supports `libgcc` as a shared library,
this is a problem since
when compiling with -shared the produced dll now has a dependency on
`libgcc_s_sjlj-1.dll`.
To solve this the flag `-static-libgcc` is now being used for all GCC
calls on windows.

Test Plan:
./validate was ran both on x86 and x86_64 windows and compared against
the baseline.

A few test were failing due to Ld no longer being noisy. These were
updated.

The changes to the configure script *should* be validated by the build
bots for the other platforms before landing

Reviewers: simonmar, awson, bgamari, austin, thomie

Reviewed By: thomie

Subscribers: #ghc_windows_task_force, thomie, awson

Differential Revision: https://phabricator.haskell.org/D1123

GHC Trac Issues: #10726, #9014, #9218, #10435

comment:8 in reply to:  3 Changed 4 years ago by awson

Replying to awson:

recently released binutils-2.25.1 which have a very important bug (wrong linking of 64-bit import libraries generated by MS linker) fixed.

Strictly speaking, binutils-2.25.1 doesn't have this bug fixed, but its MSys2 build *has* the relevant patch (and a couple of other useful patches) applied.

comment:9 Changed 4 years ago by Phyx-

Resolution: fixed
Status: patchclosed

comment:10 Changed 4 years ago by thomie

Status: closedmerge

comment:11 Changed 4 years ago by thomie

Milestone: 7.12.17.10.3

In #10795, user joozek78 asks for a merge to 7.10 of the above patch.

comment:12 Changed 4 years ago by snoyberg

Cc: snoyberg added

comment:13 Changed 4 years ago by joozek78

Cc: joozek78 added

comment:14 Changed 4 years ago by snoyberg

Just adding a clarifying comment. The reason for the requested merge to 7.10 is so that gcc can use response files internally, fixing an issue with long linker command line options. This was originally raised in https://ghc.haskell.org/trac/ghc/ticket/8596. The discussion leading up to the requested merge is on Github at https://github.com/commercialhaskell/stack/issues/795#issuecomment-134758338

comment:15 Changed 4 years ago by joozek78

@thomie Is there any estimate for this will be completed? When should I start to worry about this issue not being looked at?

comment:16 Changed 4 years ago by thomie

It's not up to me to decide, but rather the 7.10 release manager's: bgamari or thoughtpolice.

It's a rather big change though, and hasn't been tested much at all (!). Such large changes usually don't belong in a point release. But since it should make the GHC Windows experience much better, maybe we should make an exception. I just have no idea if there are any serious programs being compiled with GHC on Windows that might break because of this patch, and neither do bgamari or thoughtpolice I think, so maybe that would explain their hesitance to include it (correct me if I'm wrong).

Maybe you or snoyberg could send out an email to the various mailinglists, asking if any Windows user is against including this patch.

Note: last I heard, it's not even clear there will be a 7.10.3 (I'm not sure why not).

comment:17 Changed 4 years ago by simonpj

It is not that we have decided not to produce 7.10.3. It's just that we'll only produce it if it's actually needed!

More specifically, we will produce 7.10.3 if (but only if)

  • there is a significant body of users
  • for whom 7.10.2 simply doesn't work
  • and there is no decent workaround they can use
  • and for which we can be reasonably sure that the fix isn't going to break anything else (necessitating 7.10.4 to mitigate).

I don't know whether or not this ticket is in this category.

Simon

comment:18 Changed 4 years ago by snoyberg

I can report that at least three separate users have reported to me the complete inability to build their projects on Windows with 7.10.2 or 7.8.4 without this patch (and the related response file fix).

comment:19 Changed 4 years ago by hanjoosten

I would also VERY much see this fix released early. My project is bumping on this issue too. (see this ticket: https://github.com/fpco/minghc/issues/91)

comment:20 Changed 4 years ago by hanjoosten

Thinking even more about this: Sandboxing is done more and more, resulting in longer paths, thus triggering this bug sooner. Hopefully there will be a 7.10.3 soon. If there is anything I could do to test, please let me know.

comment:21 Changed 4 years ago by tejon

Just found this via @hanjoosten in a MinGHC issue thread. Haven't tested yet, but there's a very good chance this will fix building the GTK+ bindings on Windows: current failures mostly revolve around attempts to use -pthreads, which is not available in GCC 4.6.3 but is as of 4.9.x.

comment:22 Changed 4 years ago by simonpj

Can someone be specific about what you'd like to see in 7.10.3. I assume the patch in comment:7. Anything else? There is some muttering above about "and the related response file fix" but what is that fix? Anything else?

comment:24 Changed 4 years ago by Phyx-

If this is merged back, it's important to merge e415369e91347d23f149a1a750b267da2ee5d74c as well, as that switches the URL to a Haskell.org location from the msys2.org site in this patch.

comment:25 Changed 4 years ago by bgamari

Status: mergeclosed

Is there any reason why we *need* the toolchain upgrade for 7.10.3? Austin has pointed out to me that this sort of change carries with it substantial risk. Given that we want to ensure that 7.10.3 is stable, I think it is reasonable to be a bit conservative here.

I'm going to close this for now as 296bc70b5ff6c853f2782e9ec5aa47a52110345e has been merged. If someone has a compelling reason why we need to update the toolchain then feel free to reopen.

comment:26 Changed 4 years ago by bgamari

I've spoken to Phyx- about this issue. It seems there may be compelling reasons to merge (and thomie agrees),

  • The current GCC contains a bug where it can’t call itself recursively with response files: this means that Michael Snoyman’s response file patch (#8596) is essentially useless without a newer toolchain
  • The upgrade enables programs to run on Windows Server. As Phyx- said in #ghc,

    since starting with Server 2008 SEHOP is enabled by default and the 32bit version of GHC (which was using a very old mingw GCC) did not correctly handle the exceptions. So windows would block the executable

  • Phyx- also said,

    under the misc category we get a lot of different bug fixes for binutils. But I am not aware of any specific reported issues reported.

Perhaps we could try merging and rope some Windows users into thorough testing pre-release?

comment:27 Changed 4 years ago by hanjoosten

There are several reports of problems with using windows and ghc 7.10.*. (see https://github.com/commercialhaskell/stack/issues/466 ). When using sandboxes on not too small program, these problems arise. Currently, our project too cannot use ghc 7.10 because of this. I am prefectly willing to do some testing.

comment:28 Changed 4 years ago by hanjoosten

Owner: Phyx- deleted
Resolution: fixed
Status: closednew

comment:29 Changed 4 years ago by bgamari

Resolution: fixed
Status: newclosed

comment:30 Changed 4 years ago by bgamari

Merged to 7.10.

comment:31 Changed 4 years ago by lukexi

Cc: bgamari thomie added
Resolution: fixed
Status: closednew

I seem to be hitting a regression with 7.10.3 RC1:

I get this when trying to link my bindings to Bullet Physics https://github.com/lukexi/bullet-mini

Preprocessing test suite 'cubes' for bullet-mini-0.1.0.0...
[2 of 3] Compiling Types            ( test\shared\Types.hs, .stack-work\dist\d96ab9d9\build\cubes\cubes-tmp\Types.o )
ghc.exe: unable to load package `bullet-mini-0.1.0.0'
ghc.exe: C:\msys64\home\lukex_000\Projects\bullet-mini\.stack-work\dist\d96ab9d9\build\HSbullet-mini-0.1.0.0-2FOziegOC2l7eL6oW7Ffnm.o: unknown symbol `__mingw_vsscanf'

The Types module is the first one that uses Template Haskell.

I'd previously triumphantly gotten this package building with a modified version of 7.10.2 with the following patches applied: https://github.com/lukexi/ghc/commits/ghc-7.10.2-release-plus-rework-windows-pe-linker

So something broke between here and there.

So far I've tried updating my entire pacman database but that didn't seem to help.

comment:32 Changed 4 years ago by lukexi

I've pushed a commit to https://github.com/lukexi/bullet-mini that should make this easy to repro:

stack test :linking --stack-yaml stack-minimal.yaml

or

cabal test linking

should do the trick.

comment:33 Changed 4 years ago by awson

__mingw_vsscanf resides in libmingwex.a which is never mentioned in project's cabal file.

comment:34 Changed 4 years ago by lukexi

Thanks awson — I noticed that too and tried to add libmingwex but ran into https://ghc.haskell.org/trac/ghc/ticket/3242 (odd that it worked fine in 7.10.2 + patches, though...)

comment:35 Changed 4 years ago by awson

I think the problem is in that libmingwex.a is unavailable during project's prelinked object file (HSbullet-mini-0.1.0.0-2FOziegOC2l7eL6oW7Ffnm.o in your case) creation.

Thus instead of adding mingwex to extra-libraries try to add something like -lmingwex to ghc-options.

Or, perhaps, even better is to use ld-options field of project's cabal file.

Last edited 4 years ago by awson (previous) (diff)

comment:36 Changed 4 years ago by lukexi

Progress, sort of?

$ stack test :linking
bullet-mini-0.1.0.0: configure (lib + test)
Configuring bullet-mini-0.1.0.0...
Warning: Instead of 'ghc-options: -lmingwex' use 'extra-libraries: mingwex'
Warning: Instead of 'ghc-options:
-LC:\msys64\usr\local\ghc\mingw\x86_64-w64-mingw32\lib' use 'extra-lib-dirs:
C:\msys64\usr\local\ghc\mingw\x86_64-w64-mingw32\lib'
bullet-mini-0.1.0.0: build (lib + test)
Preprocessing library bullet-mini-0.1.0.0...
[1 of 1] Compiling Physics.Bullet   ( src\Physics\Bullet.hs, .stack-work\dist\d96ab9d9\build\Physics\Bullet.o )
GHC runtime linker: fatal error: I found a duplicate definition for symbol
   coshf
whilst processing object file
   C:\msys64\usr\local\ghc\mingw\x86_64-w64-mingw32\lib\libmingwex.a
This could be caused by:
   * Loading two different object files which export the same symbol
   * Specifying the same object file twice on the GHCi command line
   * An incorrect `package.conf' entry, causing some object to be
     loaded twice.
ghc.exe: panic! (the 'impossible' happened)
  (GHC version 7.10.2.20151030 for x86_64-unknown-mingw32):
        loadArchive "C:\\msys64\\usr\\local\\ghc\\mingw\\x86_64-w64-mingw32\\lib\\libmingwex.a": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I used

ghc-options: -LC:\msys64\usr\local\ghc\mingw\x86_64-w64-mingw32\lib -lmingwex

comment:37 Changed 4 years ago by Phyx-

I don't really know how stack works, and I am sort of confused why --interactive is involved here. Never the less, I don't think anything broke here, interactive was just never fixed. Previously when it built cabal run was used as a test. This wouldn't use the runtime linker and should still be working.

I think the duplicate symbols error is coming from the fact that the Rts re-exports some symbols from mingw but not all. it includes SymI_HasProto(coshf) under the comment:

/* These are statically linked from the mingw libraries into the ghc
   executable, so we have to employ this hack. */

Does run still produce a working binary?

comment:38 Changed 4 years ago by lukexi

It's the use of Template Haskell that's triggering the runtime linker, but perhaps I'm confused in assuming it's the same as --interactive?

(I assume you're asking about run because you thought I was using --interactive — the error happens during compilation so I never get a working binary)

OK! I'm able to compile again if I add SymI_HasProto(__mingw_vsscanf) to Linker.c — is that the right way to go?

(The issue you fixed in #10672 happened with both cabal and stack — since stack uses Cabal-as-a-library internally I'm pretty certain cabal will do the same thing but I can give it a try!

And to restate: the ghc-7.10.2 tag with #10672's patches applied doesn't manifest the unknown symbol issue mentioned in comment 31 — only 7.10.3 does which is why I'm looking for what broke)

comment:39 Changed 4 years ago by Phyx-

Yes, sorry, I was confused as to why the runtime linker was involved during a normal compilation. But TH makes sense.

I guess a temporary solution to get 7.10.3 out the door is to just re-export everything from mingwex since you can't actually specify -lmingwex current.

But I have wondered and still do why the RTS is re-exporting symbols like this instead of us just linking against -lmingwex. Do you have an idea why @awson?

comment:40 Changed 4 years ago by lukexi

Looks like bindings-GLFW is also affected, with __ms_vsnprintf being the missing symbol this time.

(No idea why we're re-exporting symbols but that sounds like a fine solution for getting a release out!)

comment:41 Changed 4 years ago by Elieux

Wouldn't this help? I'm sorry if it's not relevant, but it's one of the changes I did when I was testing GHC with MSYS2 mingw-w64 toolchains.

diff --git a/rts/package.conf.in b/rts/package.conf.in
index 5c6d240..db1d281 100644
--- a/rts/package.conf.in
+++ b/rts/package.conf.in
@@ -57,9 +57,7 @@ unresolved symbols. */
                               ,"bfd", "iberty"  /* for debugging */
 #endif
 #ifdef HAVE_LIBMINGWEX
-# ifndef INSTALLING                             /* Bundled Mingw is behind */
                               ,"mingwex"
-# endif
 #endif
 #ifdef USE_LIBDW
                              , "elf"

comment:42 Changed 4 years ago by Elieux

Cc: Elieux added

comment:43 Changed 4 years ago by Phyx-

Thanks Elieux! I didn't know I was there, I'll give it a try, would be an easy fix.

@lukexi your bullet-mini example doesn't compile:

Preprocessing library bullet-mini-0.1.0.0...
[1 of 1] Compiling Physics.Bullet   ( src\Physics\Bullet.hs, dist\build\Physics\Bullet.o )

src\Physics\Bullet.cpp:2:36:
     fatal error: btBulletDynamicsCommon.h: No such file or directory
compilation terminated.

Did you miss a file?

comment:44 Changed 4 years ago by lukexi

Oh I think you need to do git submodule init && git submodule update!

comment:45 Changed 4 years ago by lukexi

(oops, and I forgot to push a fix to the linking test I made a few days ago, so be sure to pull again now!)

comment:46 in reply to:  39 ; Changed 4 years ago by Matt

Replying to Phyx-:

I guess a temporary solution to get 7.10.3 out the door is to just re-export everything from mingwex since you can't actually specify -lmingwex current.

But I have wondered and still do why the RTS is re-exporting symbols like this instead of us just linking against -lmingwex.

I think the main issue here is that not all symbols from libmingwex.a get linked into final ghc executable so some they can't be resolved when running GHC in interactive. Therefore i believe proper solution should be linking libmingwex.aas a whole archive when building ghc.exe, instead of just letting GCC to link only used symbols as it does by default.

Reexporting those symbols seems like a bad idea, because, if I'm not mistaken, RTS gets linked into ever executable produced by GHC, so this way we are actually forcing possibly unnecessary symbols into every executable?

comment:47 in reply to:  46 Changed 4 years ago by Phyx-

@lukexi thanks, can reproduce it now :)

@Elieux unfortunately it seems it's not that easy. The rts seems to be a special case again. The linker seems to ignore the extra-libraries from the package.conf, which explains why you can put anything you want in there and it'll still compile.

128 emptyPLS :: DynFlags -> PersistentLinkerState 
129 emptyPLS _ = PersistentLinkerState { 
130                         closure_env = emptyNameEnv, 
131                         itbl_env    = emptyNameEnv, 
132                         pkgs_loaded = init_pkgs, 
133                         bcos_loaded = [], 
134                         objs_loaded = [], 
135                         temp_sos = [] } 
136 
 
137   -- Packages that don't need loading, because the compiler 
138   -- shares them with the interpreted program. 
139   -- 
140   -- The linker's symbol table is populated with RTS symbols using an 
141   -- explicit list.  See rts/Linker.c for details. 
142   where init_pkgs = [rtsUnitId] 

Therefore i believe proper solution should be linking libmingwex.aas a whole archive when building ghc.exe, instead of just letting GCC to link only used symbols as it does by default.

Hence the temporary solution. The package.conf for the rts ignores the extra-libraries section in the linker. Windows isn't alone here, Mac also re-exports some symbols. And before ripping this out I want to know why. Also I think it's riskier then just adding some extra symbols. As it is right now, there is no way to use libmingwex.a on Windows.

Reexporting those symbols seems like a bad idea, because, if I'm not mistaken, RTS gets linked into ever executable produced by GHC, so this way we are actually forcing possibly unnecessary symbols into every executable?

Yes, I don't know why it was done this way. However we are already re-exporting some of the symbols. Quite a few in fact. Yes the proper solution is to rip the symbols out and just link against libmingwex.a which may get in to 8.0, but for 7.10.3 I doubt it. It's already at RC3.

Changed 4 years ago by lukexi

Attachment: add_missing_symbols.patch added

comment:48 Changed 4 years ago by lukexi

Priority: normalhigh
Status: newpatch

OK, with these 5 symbols added all of my libraries are working perfectly again.

diff --git a/rts/Linker.c b/rts/Linker.c
index 68f1f59..a939e35 100644
--- a/rts/Linker.c
+++ b/rts/Linker.c
@@ -525,6 +525,11 @@ typedef struct _RtsSymbolVal {
       SymI_HasProto(isalnum)                             \
       SymI_HasProto(isascii)                             \
       RTS___MINGW_VFPRINTF_SYM                           \
+      SymI_HasProto(__mingw_vsscanf)                     \
+      SymI_HasProto(__mingw_vprintf)                     \
+      SymI_HasProto(__mingw_vsprintf)                    \
+      SymI_HasProto(__ms_vsnprintf)                      \
+      SymI_HasProto(strdup)                              \
       SymI_HasProto(strcmp)                              \
       SymI_HasProto(memmove)                             \
       SymI_HasProto(realloc)                             \

After fully upgrading my MSYS2 installation I was also able to remove the brittle extra-lib-dirs hacks in my libraries that needed libstdc++-6.dll, which is great!

comment:50 Changed 4 years ago by RyanGlScott

I'm joining the conversation pretty late here, but how exactly did you know which symbols to export in Linker.c? I ask since I'm getting some pretty similar errors to what you've been reporting with different packages (after #3242 was fixed).

For example, when loading logfloat in GHCi HEAD:

$ ghc-stage2 --interactive
GHCi, version 7.11.20151123: http://www.haskell.org/ghc/  :? for help
> import Data.Number.LogFloat
> log1p 1.0
<interactive>: C:\Users\ryanscot\AppData\Roaming\cabal\x86_64-windows-ghc-7.11.20151123\logfloat-0.13.3.3-4JZYNCXKwghOD60rvMUAcn\HSlogfloat-0.13.3.3-4JZYNCXKwghOD60rvMUAcn.o: unknown symbol `log1p'
ghc-stage2.exe: unable to load package `logfloat-0.13.3.3'

And when loading sdl2 in GHCi HEAD:

$ ghc-stage2 --interactive
GHCi, version 7.11.20151123: http://www.haskell.org/ghc/  :? for help
> import SDL
> ticks
GHC runtime linker: fatal error: I found a duplicate definition for symbol
   wmain
whilst processing object file
   C:/Users/ryanscot/Documents/Software/ghc/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/5.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libmingw32.a
This could be caused by:
   * Loading two different object files which export the same symbol
   * Specifying the same object file twice on the GHCi command line
   * An incorrect `package.conf' entry, causing some object to be
     loaded twice.
ghc-stage2.exe: panic! (the 'impossible' happened)
  (GHC version 7.11.20151123 for x86_64-unknown-mingw32):
        loadArchive "C:/Users/ryanscot/Documents/Software/ghc/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/5.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libmingw32.a": failed

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Is this because libmingw32/libmingwex aren't being linked against when building GHC? Also, how can I figure out which symbols to reexport as a temporary fix?

Last edited 4 years ago by RyanGlScott (previous) (diff)

comment:51 Changed 4 years ago by Phyx-

@Lukexi thanks, I'll put the patch up soon.

@RyanGIScott hmm can you add -v to the first one and paste the output again? For the second one, I think libmingw32 has a main function of its own. Try adding -no-hs-main but that might be broken based on #2459

Changed 4 years ago by RyanGlScott

Attachment: logfloat_output.txt added

runghc -v LogFloat.hs

Changed 4 years ago by RyanGlScott

Attachment: audioexample_output.txt added

runghc -v -no-hs-main AudioExample.hs

comment:52 Changed 4 years ago by RyanGlScott

I've attached the results of invoking runghc -v on two different files. LogFloat.hs is this:

module Main (main) where

import Data.Number.LogFloat (log1p)

main :: IO ()
main = print $ log1p 1.0

AudioExample.hs is this file, just with the module name Main instead of AudioExample. Adding -no-hs-main didn't appear to change the error message.

comment:53 Changed 4 years ago by Phyx-

@RyanGIScott

The first one is indeed because you can't like against libmingwex

$ dlltool.exe --output-def libmingwex.def --export-all-symbols libmingwex.a
$ grep "log1p" libmingwex.def
        log1p @ 494
        log1pf @ 495
        log1pl @ 496

So re-exporting log1p should fix it. But this feels like a moving target. Try adding SymI_HasProto(log1p) to the list. If you're on HEAD, these symbols have been moved to RtsSymbols.c.

comment:54 Changed 4 years ago by Phyx-

I'm currently trying to track down an issue in HEAD so I can't test this myself atm, but

@RyanGIScott can you try something for me, since you're linking against libm.a try just adding -lmsvcr120 and see if that links. You may have an older version of the library so check the output of dir c:\Windows\System32\msvcr*.dll.

This should work since libm.a is a stub just to satisfy linking because the symbols are In the Microsoft runtime.

@Lukexi instead of re-exporting the symbols, try using the stock RC3, can you rebuild your C parts with -D__USE_MINGW_ANSI_STDIO=0. This will make MingW-w64 drop the __mingw prefixes and then the symbols should be found in the msvcr*.dll as well. So then you can just link against that as well.

This could be a workaround if the patch doesn't make it in 7.10.3. I haven't gotten any response to whether it would.

comment:55 Changed 4 years ago by RyanGlScott

Wow, I ran runghc -lmsvcr120 LogFloat.hs and it works perfectly.

It turns out there's another quirk: if I go to the logfloat source and run ghc-stage2 --interactive -isrc src/Data/Number/LogFloat.hs, I can then invoke log1p 1.0 without any linker issues. If I run cabal repl instead and then call log1p 1.0, it can't find the log1p symbol. Hm.

comment:56 Changed 4 years ago by Phyx-

LogFloat seems to have a fallback method to a less accurate version depending on if FFI is used:

#ifdef __USE_FFI__
foreign import ccall unsafe "math.h log1p"
    log1p :: Double -> Double
#else
-- See statistics:Statistics.Math for a more accurate Haskell
-- implementation.
log1p :: Double -> Double
{-# INLINE [0] log1p #-}
log1p x = log (1 + x)
#endif

__USE_FFI__ is set in the cabal file but of course won't be set using --interactive. Adding -D__USE_FFI__ to the command should trigger the same error.

comment:57 Changed 4 years ago by bgamari

Alright, I've applied Luke's patch from comment:48 to ghc-7.10 as bbbf79b91747dbb5ab284f075927d1e1e27aba95./ This will hopefully serve as a sufficient workaround for 7.10.3, although I do hope we can do better for 8.0. Thanks for the testing, everyone!

comment:58 Changed 4 years ago by bgamari

Milestone: 7.10.38.0.1

Bumping to 8.0.1.

comment:59 Changed 4 years ago by RyanGlScott

D'oh, you're right, I forgot about -D__USE_FFI__. Running ghc-stage2 --interactive -D__USE_FFI__ -isrc src/Data/Number/LogFloat.hs does indeed trigger the same error.

Another curiosity: I rebuilt GHC with the following change:

  • rts/RtsSymbols.c

    diff --git a/rts/RtsSymbols.c b/rts/RtsSymbols.c
    index 8413d31..31b190e 100644
    a b  
    123123      SymI_HasProto(isalnum)                             \
    124124      SymI_HasProto(isascii)                             \
    125125      RTS___MINGW_VFPRINTF_SYM                           \
     126      SymI_HasProto(log1p)                               \
     127      SymI_HasProto(expm1)                               \
    126128      SymI_HasProto(strcmp)                              \
    127129      SymI_HasProto(memmove)                             \
    128130      SymI_HasProto(realloc)                             \

And still experienced the same unknown symbol log1p error as before.

comment:60 Changed 4 years ago by Phyx-

Sorry, haven't had much time lately to look at GHC the past weeks. I should have free time again next week and I'll try to get as much as my tickets done!. I think I have a proper solution for this for 8.0.1 that doesn't require me to change the code for all platforms.

comment:61 Changed 4 years ago by bgamari

Any progress on this Phyx-?

comment:62 Changed 4 years ago by Phyx-

@bgamari I've tried 2 approaches which haven't worked completely:

1) I tried removing the symbols from the export list and adding libmingwex to the rts's package.confand have it just link against it. But turns out the rts's package.conf is ignored on all platforms. I didn't want to have to make an exception for Windows here and I don't know why the other platforms also ignore it so I abandoned this approach.

2) I tried marking the symbols we're re-exporting as weak symbols, so there wouldn't be a change to existing code and would allow you to link against libmingwex. But unfortunately because of when the other libraries specified by -l are linked in, some of the symbols have already been used and thus aren't weak anymore. So I still get duplicate link errors.

What I want to try now is leaving them as weak symbols, but loading libmingwex.a at rts initialization time. Much like how kernel32 is loaded. This is hopefully early enough that the symbols haven't been used yet. I'll give that a try tonight.

comment:63 Changed 4 years ago by bgamari

Status: patchnew

Phyx-, could you perhaps open a new ticket just to track this issue so we can close this ticket? I think most of the information needed has already been written down here in one place or another; it's just a matter of copying it to a new ticket, distilling a clear statement of the problem, and collecting the approaches that you have already tried.

comment:64 Changed 4 years ago by Phyx-

Resolution: fixed
Status: newclosed

Closing this issue as the error isn't directly related to the upgrade of GCC as it predated the upgrade. New issue is now #11223 .

comment:65 Changed 11 months ago by bgamari

Component: Build SystemBuild System (make)

The new Hadrian build system has been merged. Relabeling the tickets concerning the legacy make build system to prevent confusion.

Note: See TracTickets for help on using tickets.