Opened 5 years ago

Last modified 4 years ago

#9447 patch feature request

Add support for resizing `MutableByteArray#`s

Reported by: hvr Owned by:
Priority: normal Milestone:
Component: Compiler Version: 7.8.2
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Runtime performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D133, Phab:D1139
Wiki Page:

Description

This is motivated by #9281 which changes how memory is allocated, and gives rise to the need to be able to efficiently shrink or grow MutableByteArray#s before they're frozen into ByteArray# while retaining their original byte content to avoid a major bottle-neck.

This ticket is for tracking/documenting all changes related to this new facility.

Here's the current road-map:

  1. Implement
    shrinkMutableByteArray# :: MutableByteArray# s -> Int# -> State# s -> State# s
    resizeMutableByteArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, MutableByteArray# s #)
    
    (see Phab:D133)
  1. As suggested by Johan,
    getSizeofMutableByteArray# :: MutableByteArray# s -> State# s -> (# State# s Int# #)
    
    This is similar in spirit to numCapabilities and getNumCapabilities. Add a deprecate pragma (or equivalent) for sizeofMutableByteArray#. Add a note to sizeofMutableByteArray# stating that it's unsafe in the presence of calls to resize-operations on the same MBA.
  1. Submit patches to upstream libraries replacing calls to sizeofMutableByteArray# with getSizeofMutableByteArray#
  1. Investigate how to provide in-place (zero-copy) growing of MBAs (step 1. only implements in-place shrinking).

Change History (6)

comment:1 Changed 5 years ago by Herbert Valerio Riedel <hvr@…>

In 246436f13739593d2a211ceb830393338118ca4d/ghc:

Implement {resize,shrink}MutableByteArray# primops

The two new primops with the type-signatures

  resizeMutableByteArray# :: MutableByteArray# s -> Int#
                          -> State# s -> (# State# s, MutableByteArray# s #)

  shrinkMutableByteArray# :: MutableByteArray# s -> Int#
                          -> State# s -> State# s

allow to resize MutableByteArray#s in-place (when possible), and are useful
for algorithms where memory is temporarily over-allocated. The motivating
use-case is for implementing integer backends, where the final target size of
the result is either N or N+1, and only known after the operation has been
performed.

A future commit will implement a stateful variant of the
`sizeofMutableByteArray#` operation (see #9447 for details), since now the
size of a `MutableByteArray#` may change over its lifetime (i.e before
it gets frozen or GCed).

Test Plan: ./validate --slow

Reviewers: ezyang, austin, simonmar

Reviewed By: austin, simonmar

Differential Revision: https://phabricator.haskell.org/D133

comment:2 Changed 4 years ago by thomie

Type: bugfeature request
Type of failure: None/UnknownRuntime performance bug

comment:3 Changed 4 years ago by bgamari

See Phab:D1139 for an implementation of getSizeofMutableByteArray#.

comment:4 Changed 4 years ago by bgamari

Differential Rev(s): Phab:D133Phab:D133, Phab:D1139
Status: newpatch

comment:5 Changed 4 years ago by Ben Gamari <ben@…>

In 9e8562a/ghc:

Implement getSizeofMutableByteArrayOp primop

Now since ByteArrays are mutable we need to be more explicit about when
the size is queried.

Test Plan: Add testcase and validate

Reviewers: goldfire, hvr, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1139

GHC Trac Issues: #9447

comment:6 Changed 4 years ago by bgamari

hvr, I discussed this briefly with Simon Marlow in the meeting last week and he indicated that it may not be too tricky to implement efficient growing of ByteArrays. Perhaps you'd like to discuss this with him (assuming you still believe this is a worthwhile optimization)?

Note: See TracTickets for help on using tickets.