Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#10904 closed bug (fixed)

C finalizer may be called on re-used memory

Reported by: bherzog Owned by:
Priority: normal Milestone: 7.10.3
Component: Runtime System Version: 7.4.1
Keywords: Cc: simonmar
Operating System: Linux Architecture: Unknown/Multiple
Type of failure: Runtime crash Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D1275
Wiki Page:

Description

It seems that the runtime system sometimes reuses memory referred to by ForeignPtr values before the associated C finalizers have been run. At that's what it looks like to me. Maybe I'm interpreting the behavior of GHC's runtime system incorrectly, or it's a defect in my test program.

To reproduce the defect, compile finalizertest.hs and finalizerlib.c, e.g. with

ghc finalizertest.hs finalizerlib.c -threaded

Run the resulting program as

./finalizertest +RTS -N2

After a while it prints a message like

finalize_value: 80f69dc != 11223344 after 47393 calls
Aborted

The C code prints this if the pointer passed to the finalizer does not point to the expected value.

It's not necessary to link with the threaded runtime, nor is it necessary to run with more than one CPU. Doing so increases the likelihood of the defect to occur substantially, though.

I've observed this with several different combinations of GHC and host-systems:

  • GHC 7.10.2 on a 64 bit Linux (Debian jessie)
  • GHC 7.10.1 on a 32 bit Linux (Debian wheezy)
  • GHC 7.4.1 on a 32 bit Linux (Debian wheezy) This GHC is the one packaged by Debian

It's crucial for the defect that the memory is allocated with mallocForeignPtrBytes. Using mallocBytes instead and building a ForeignPtr with finalizerFree avoids the defect.

I came across this defect while trying to debug a segmentation fault in the zlib package. This defect was reported on the libraries mailing list https://mail.haskell.org/pipermail/libraries/2015-June/025829.html (corresponding Agda ticket: https://github.com/agda/agda/issues/1518). I came across it in one of my own projects last week.

My test program basically does what zlib does when allocating and initializing the z_stream value: it allocates memory with mallocForeignPtrBytes and later adds a finalizer with addForeignPtrFinalizer.

Attachments (3)

finalizertest.hs (513 bytes) - added by bherzog 4 years ago.
finalizerlib.c (446 bytes) - added by bherzog 4 years ago.
finalizerlib.h (106 bytes) - added by bherzog 4 years ago.

Download all attachments as: .zip

Change History (11)

Changed 4 years ago by bherzog

Attachment: finalizertest.hs added

Changed 4 years ago by bherzog

Attachment: finalizerlib.c added

Changed 4 years ago by bherzog

Attachment: finalizerlib.h added

comment:1 Changed 4 years ago by simonmar

I looked at the code, and I think you're right, there does seem to be a possibility that the finalizer will run after the memory has been reclaimed. As a workaround you can follow the suggestion you made, namely use mallocBytes and then newForeignPtr.

I'll think about how this can be fixed. It's tricky, and we already have too much overhead for C finalizers (things got a lot more complicated during the recent changes to ensure ordering for finalizers, I think that's when the bug may have crept in).

comment:2 Changed 4 years ago by simonmar

Differential Rev(s): Phab:D1275
Status: newpatch

comment:3 Changed 4 years ago by Simon Marlow <marlowsd@…>

In 2440e3c6/ghc:

Fix a bug with mallocForeignPtr and finalizers (#10904)

Summary: See Note [MallocPtr finalizers]

Test Plan: validate; new test T10904

Reviewers: ezyang, bgamari, austin, hvr, rwbarton

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1275

comment:4 Changed 4 years ago by simonmar

Status: patchmerge

comment:5 Changed 4 years ago by simonmar

Milestone: 7.10.3

comment:6 Changed 4 years ago by bgamari

Merged to ghc-7.10.

comment:7 Changed 4 years ago by bgamari

Resolution: fixed
Status: mergeclosed

comment:8 Changed 4 years ago by asr

I couldn't reproduce the Agda issue pointed out in the above description, i.e. Agda issue 1518, after this bug was fixed. Thanks!

Note: See TracTickets for help on using tickets.