Opened 12 years ago

Closed 10 years ago

Last modified 10 years ago

#1876 closed task (fixed)

Complete shared library support

Reported by: simonmar Owned by: duncan
Priority: high Milestone: 6.12.1
Component: Compiler Version: 6.11
Keywords: Cc: 1@…, batterseapower+GHC1876@…, mafo, alexander.dunlap@…, leather@…, ingmar@…, petersen@…, ndmitchell@…, Bulat.Ziganshin@…, ttuegel@…
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

This ticket is a placeholder for the shared libraries work, which is mostly working but needs to be completed (binary distributions, issues with -rpath etc.).

Attachments (4)

ghc-buildlog1 (51.4 KB) - added by juhpetersen 11 years ago.
end of above buildlog
buildlog-noprof (3.3 KB) - added by juhpetersen 11 years ago.
build error without prof
buildlog-head-noprof (17.0 KB) - added by juhpetersen 11 years ago.
same error with HEAD without prof (in English;)
ghc-6.11-ghc-pkg-install-inplace.patch (1.2 KB) - added by juhpetersen 10 years ago.
Use dist-inplace ghc-pkg for dist package.conf and cabal'ing

Download all attachments as: .zip

Change History (35)

comment:1 Changed 11 years ago by simonmar

Priority: normalhigh

comment:2 Changed 11 years ago by simonmar

difficulty: UnknownDifficult (1 week)
Milestone: 6.10 branch6.10.1

comment:3 Changed 11 years ago by batterseapower

Cc: batterseapower+GHC1876@… added

comment:4 Changed 11 years ago by mafo

Cc: mafo added

comment:5 Changed 11 years ago by simonmar

Milestone: 6.10.16.10 branch

I've made some progress on this, but it doesn't look like I'll have time to finish it for 6.10.1. Since it doesn't involve API changes, we could possibly get it into a later 6.10 release, though.

comment:6 Changed 11 years ago by ajd

Cc: alexander.dunlap@… added

comment:7 Changed 11 years ago by simonmar

Architecture: UnknownUnknown/Multiple

comment:8 Changed 11 years ago by simonmar

Operating System: UnknownUnknown/Multiple

comment:9 Changed 11 years ago by clemens

I fixed the following for Linux/x86_64 (probably also other archs) in today's HEAD "./configure --enable-shared; make" should compile error free. It should give you all libraries as shared objects, including a stage2 GHC package. The GHC package should be linkable into a working ghc binary (including a wrapper) that also works in interactive mode. Running the testsuits gives a few isolated regressions in ghci and template haskell I still have to dig into.

I won't go into details about the Windows status. At the moment, linking the RTS dies with unresolved RTS imp* symbols. Linking the RTS with its own import lib (libHSrts*.dll.a) should fix the problem, but I haven't got there yet.

However, for libgmp on Windows, we need to apply the same trick as with libffi, namely change it into a Haskell package. Most of the hacking was already done for libffi, so the changes from libffi/Makefile could be worked into gmp/Makefile without much trouble, I think.

In any case, I'd like to argue that we should give priority to Linux dynamic linking. With this I mean that we ship dynamic lib support even if it's just for this single architecture. This will help to build up a small user base that will hopefully report regressions as soon as they pop up. Also Linux is the most hassle free OS when it comes to dynamic library linking, so it is easier to establish that as base camp for further climbing.

Except for libffi change, most of the changes are quite minimal. We could probably target 6.10.2 as milestone for merging this into stable.

On Linux x86-64 dynamic at the moment:

Unexpected failures:
   2014(normal)
   T2386(normal)
   T2597a(normal)
   T2597b(normal)
   TH_genEx(normal)
   TH_spliceD1(normal)
   TH_spliceDecl3(normal)
   TH_spliceDecl4(normal)
   TH_spliceE5(normal)
   prog001(ghci)
   prog002(ghci)
   prog003(ghci)

comment:10 Changed 11 years ago by clemens

Cc: clemens@… added
Owner: set to clemens
Status: newassigned

comment:11 Changed 11 years ago by simonmar

Some issues to resolve before we can deploy shared libraries. Some of these are probable just questions to be answered, but hint at documentation that we need to write.

  • Do GHC distributions contain both shared and static libraries?
  • Will Cabal build both shared and static libraries by default?
  • Can a module built with -dynamic link against static libraries, if the dynamic versions are not available?
  • If modules are compiled with -fPIC and/or -dynamic, can they then be linked statically? On which platforms? If not, what goes wrong if you try this?
  • On x86_64 we should have -dynamic turned on by default, for loading modules into GHCi, otherwise data references to shared libraries will only have 32-bit relocations and thus will break.
  • Using -static at compile-time on x86_64 should therefore come with a warning that the modules cannot be loaded into GHCi. Can we detect this and report an error in GHCi?
  • What is the performance overhead of turning on -dynamic by default on x86_64?
  • What is the performance overhead (and code-size overhead) of shared libraries in general?

comment:12 Changed 11 years ago by spl

Cc: leather@… added

comment:13 Changed 11 years ago by ingmar

Cc: ingmar@… added

comment:14 Changed 11 years ago by juhpetersen

Cc: petersen@… added

Is there anything one can do to help with testing this?

comment:15 Changed 11 years ago by simonmar

Grab the GHC sources, configure with --enable-shared, build as normal and report problems that you encounter.

comment:16 Changed 11 years ago by NeilMitchell

Cc: ndmitchell@… added

comment:17 Changed 11 years ago by guest

Cc: Bulat.Ziganshin@… added

comment:18 Changed 11 years ago by clemens

We need a buildbot for regression testing. I asked a few times on #ghc without much response. Probably I wasn't persistant enough to chase after those responsible.

I haven't look at most recent HEAD but frankly, I would be surprised if --enable-shared still works out of the box, where I left it a few months ago. Also I would be surpised if the number of failing test cases given above would have remained constant. Honestly, there is more exciting stuff around than hacking on shared libs.

If someone wants to take the job, this one should feel free to reassign the bug to himself.

comment:19 Changed 11 years ago by igloo

We'll make it possible to build it in parallel with the non-shared GHC in the new build system, and then we won't need a dedicated buildbot for it.

comment:20 Changed 11 years ago by simonmar

To my mind this is stalled not just due to lack of effort, but there's some design work to be done - mainly around how distributions, and how Cabal and cabal-install will work with shared libraries. See my list of questions above (http://hackage.haskell.org/trac/ghc/ticket/1876#comment:11) for a start.

The other thing that worries me is that without having support for fixed ABIs, we could have more problems with shared libraries, since when someone reinstalls a library they will break not only dependent packages, but also existing dynamically-linked binaries.

comment:21 Changed 11 years ago by clemens

Cc: clemens@… removed
Owner: changed from clemens to simonmar
Status: assignednew

Simon, I didn't see any serious attempts to get buildbots going. Iirc I fixed "make install" to that extent where regression tests could have been run.

Also I do not see any reason, how establishing base line quality is related to deployment upgrading and installation lifetime issues. If GHC doesn't have a way of regression testing shared libs in whatever conceptual broken they may be now than GHC will fail to deliver. That's simply because shared libs will bitrot in the ongoing development.

As I can't put this bug into an unassigned status, I hope simon you don't mind if I reassign it to you.

comment:22 in reply to:  21 Changed 11 years ago by simonmar

Owner: simonmar deleted

Replying to clemens:

Simon, I didn't see any serious attempts to get buildbots going. Iirc I fixed "make install" to that extent where regression tests could have been run.

Also I do not see any reason, how establishing base line quality is related to deployment upgrading and installation lifetime issues. If GHC doesn't have a way of regression testing shared libs in whatever conceptual broken they may be now than GHC will fail to deliver. That's simply because shared libs will bitrot in the ongoing development.

You're right, nothing prevents us from setting up buildbots (except lack of time). But I would really like to have a clear plan of action for getting us to the goal of having shared libs deployed. It's probably not difficult, but someone/people need to sit down and seriously think about how all this is going to work.

comment:23 in reply to:  15 Changed 11 years ago by juhpetersen

Replying to simonmar:

Grab the GHC sources, configure with --enable-shared, build as normal and report problems that you encounter.

Ok, sorry for the delay. I installed libffi-devel on Fedora rawhide and got quite far this time with stable:

: : /home/petersen/ghc-6.10.1.20090303/utils/installPackage/install-inplace/bin/installPackage install '/home/petersen/ghc-6.10.1.20090303/utils/ghc-pkg/install-inplace/bin/ghc-pkg' 'XXX/package.conf' "" \

/home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace \ /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace \ '$prefix' \ '/home/petersen/ghc-6.10.1.20090303/inplace-datadir' \ '$prefix/libexec' \ '$prefix/dynlib' \ '/home/petersen/ghc-6.10.1.20090303/inplace-datadir' \ '$prefix/doc' \ '$prefix/html' \ '$prefix/haddock' \ --distpref dist-stage2 \ --disable-executable-stripping \ --enable-shell-wrappers

Installing executable(s) in /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace make[3]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/ghc' make[2]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/compiler' make -C utils with-stage-2 make[2]: Entering directory `/home/petersen/ghc-6.10.1.20090303/utils' make -C installPackage with-stage-2 make[3]: Entering directory `/home/petersen/ghc-6.10.1.20090303/utils/installPackage' /home/petersen/ghc-6.10.1.20090303/libraries/cabal-bin /usr/bin/ghc /home/petersen/ghc-6.10.1.20090303/libraries/bootstrapping.conf 1.6.0.2 configure --distpref dist-install \

--prefix=/NONEXISTENT --bindir=/NONEXISTENT --libdir=/NONEXISTENT --libexecdir=/NONEXISTENT --datadir=/NONEXISTENT --docdir=/NONEXISTENT --haddockdir=/NONEXISTENT --htmldir=/NONEXISTENT \ --with-compiler=/home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace/ghc --with-hc-pkg=/home/petersen/ghc-6.10.1.20090303/utils/ghc-pkg/install-inplace/bin/ghc-pkg --package-db /home/petersen/ghc-6.10.1.20090303/stage3.package.conf \ --libsubdir='$pkgid' --with-gcc=gcc --with-ld=/usr/bin/ld --with-happy="/usr/bin/happy" --configure-option='--enable-shared' --configure-option=--with-cc="gcc" --with-hsc2hs=/home/petersen/ghc-6.10.1.20090303/utils/hsc2hs/install-inplace/bin/hsc2hs \ --constraint="Cabal == 1.6.0.2"

Configuring installPackage-1.0... cabal-bin: ghc version >=6.4 is required but the version of /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace/ghc could not be determined. make[3]: * [with-stage-2] Error 1 make[3]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/utils/installPackage' make[2]: * [with-stage-2.installPackage] Error 2 make[2]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/utils' make[1]: * [stage2] Error 2 make[1]: Leaving directory `/home/petersen/ghc-6.10.1.20090303' make: * [bootstrap2] Error 2

Changed 11 years ago by juhpetersen

Attachment: ghc-buildlog1 added

end of above buildlog

comment:24 Changed 11 years ago by simonmar

see also #3072

Changed 11 years ago by juhpetersen

Attachment: buildlog-noprof added

build error without prof

Changed 11 years ago by juhpetersen

Attachment: buildlog-head-noprof added

same error with HEAD without prof (in English;)

comment:25 Changed 10 years ago by igloo

Milestone: 6.10 branch6.12.1

comment:26 Changed 10 years ago by ttuegel

Cc: ttuegel@… added

Since there hasn't been any chatter on this ticket in more than a month, I thought I would share some observations I made today testing shared library support on x86_64 Linux (I also have an x86 machine to test on later). I apologize in advance if this information is redundant; keep in mind it comes from an only moderately well-informed outsider.

  1. ./configure --enable-shared && make successfully compiles a compiler! I nearly fell out of my chair in awe!
  1. make -jN does not work for N > 1, but this may be a symptom of the build system in general, and not of shared library support.
  1. Because we need binaries bootstrapped as part of the compile process to install everything, make install fails because the operating system linker cannot find the shared libraries.

I was able to use some LD_LIBRARY_PATH magic to include the relevant build directories and install anyway, but this only exposed the following:

  1. We put shared libraries for most packages into ${prefix}/lib but the rts and ffi shared-libraries go into ${prefix}/lib/ghc-6.11.20090409 where the dynamic linker won't find them by default. Although we could use LD_LIBRARY_PATH to include the appropriate directory, that feels like a dirty hack: the standard Linux practice seems to be to put any shared libraries the linker needs to find where it will find them by default.

comment:27 in reply to:  26 Changed 10 years ago by juhpetersen

Version: 6.8.16.11

Replying to ttuegel:

  1. ./configure --enable-shared && make successfully compiles a compiler! I nearly fell out of my chair in awe!

Cool - that is indeed great news. Wow, wow! :-)

I managed to "make install" ghc-6.11.20090421 with fedora ghc-6.10.2.x86_64 with a small patch I am attaching next: install needs to use dist-inplace/ghc-pkg not dist-install, which might be what ttuegel also ran into in the proceeding comment.

Changed 10 years ago by juhpetersen

Use dist-inplace ghc-pkg for dist package.conf and cabal'ing

comment:28 Changed 10 years ago by simonmar

Owner: set to duncan

comment:29 Changed 10 years ago by Pete

Cc: 1@… added

"Although we could use LD_LIBRARY_PATH to include the appropriate directory, that feels like a dirty hack: the standard Linux practice seems to be to put any shared libraries the linker needs to find where it will find them by default."

Most packages I've seen install shared libraries in $PREFIX/lib. If you accept the default $PREFIX, this means they will go in /usr/local/lib, and the linker is normally set so it will find them there. If a package (deb, rpm or whatever) is being built, then $PREFIX will usually be set to /usr. In this case, the libraries will go in /usr/lib and again the linker will find them.

If you decide to install the package in a private directory, the linker won't find the libraries. In this case you can set the executable's rpath to the directory where the libraries are to be found. The downside is that this makes the executables non-portable. If Fred installs the libraries in ~fred/lib and Jim installs them in ~jim/lib, they won't be able to run each other's executables. The rpath will be set correctly for the person who compiled the executable, but no one else. At this point you have to set LD_LIBRARY_PATH and work round it that way.

The Debian package of ALSA takes a different approach. It installs its libraries in /usr/lib/alsa-lib/ and installs /etc/ld.so.conf.d/libasound2.conf which points to them. (This file contains a single line, '/usr/lib/alsa-lib'.) This approach makes sense, I imagine, if you are installing a lot of libraries and don't want to clutter up /usr/lib.

I was just doing some experiments to make sure I'd got this right. I created foo.c:

#include <stdio.h>

void blah() {

printf("Hello World!\n");

}

and bar.c:

extern void blah();

main() {

blah();

}

First of all I build foo.c into a shared library:

$ gcc -fPIC -shared foo.c -o libfoo.so

I can then link bar.c in two ways. Here is the first option:

$ gcc bar.c -o bar -L. -lfoo $ ./bar ./bar: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory $ LD_LIBRARY_PATH=. ./bar Hello World!

Second option:

$ gcc bar.c -o bar -Wl,--rpath=/home/pc -L. -lfoo $ ./bar Hello World!

I hope this is helpful. I'm not a functional programming expert, so I can't implement this myself. On the other hand, I'm happy to help with any of these operating system issues. My email is 1@… if you want anything from me.

comment:30 Changed 10 years ago by simonmar

Resolution: fixed
Status: newclosed

Shared libs are working on Linux in 6.12.1. We should open separate tickets for other tasks remaining to be done.

A big thanks to everyone who has worked on getting shared library support into GHC over the years, particularly Wolfgang Thaller, Clemens Fruhwirth and Duncan Coutts.

comment:31 Changed 10 years ago by simonmar

difficulty: Difficult (1 week)Difficult (2-5 days)
Note: See TracTickets for help on using tickets.