Opened 4 years ago

Last modified 14 months ago

#10822 upstream bug

GHC inconsistently handles \\?\ for long paths on Windows

Reported by: snoyberg Owned by:
Priority: normal Milestone: 8.8.1
Component: Compiler Version: 7.10.2
Keywords: Cc:
Operating System: Windows Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D4416
Wiki Page:

Description

To demonstrate what I mean first:

>ghc-pkg init \\?\c:\Users\michael\Desktop\some-pkg-db
>dir \\?\c:\Users\Michael\Desktop\some-pkg-db
 Volume in drive \\?\c: is Windows8_OS
 Volume Serial Number is CEF7-874D

 Directory of \\?\c:\Users\Michael\Desktop\some-pkg-db

09/01/2015  09:38 AM    <DIR>          .
09/01/2015  09:38 AM    <DIR>          ..
09/01/2015  09:38 AM                40 package.cache
               1 File(s)             40 bytes
               2 Dir(s)   5,907,509,248 bytes free
>ghc -package-db \\?\c:\Users\Michael\Desktop\some-pkg-db
ghc: can't find a package database at \\?\c:\Users\Michael\Desktop\some-pkg-db

The problem: on Windows by default, paths are limited to 260 characters, which can become a real problem in practice when dealing with the standard library path hierarchy the Cabal library uses and any package with a decent name length. This was a real problem in practice, see:

https://github.com/haskell/cabal/issues/2502

Windows has a workaround for this: you can prefix an absolute path with
?\, see:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath

It would be very useful if such paths worked with GHC, but currently they do not. I don't yet know how deep the problem is.

Change History (7)

comment:1 Changed 4 years ago by dobenour

CoreCLR ran into the same problem and fixed it by adding the \\?\ prefix to any overlong path, after making the path absolute with the GetFullPathNameW function in the Windows API. For UNC paths (that begin with \\ but not with \\?\), the solution is to replace \\ with \\?\UNC\.

comment:2 Changed 3 years ago by Phyx-

The problem is that we rely on ports of POSIX applications to work. e.g. GCC, binutils etc. So even if GHC supports UNC paths. It probably wouldn't help much unfortunately.

comment:3 Changed 3 years ago by ezyang

But then the best thing to do is figure out exactly what the state of short path support is in the toolchain, lobby projects to fix this, and eventually get this fixed. upstream is broken is not an excuse to throw ones hands up and give up.

comment:4 Changed 18 months ago by Phyx-

Differential Rev(s): Phab:D4416
Milestone: 8.6.1
Status: newpatch

GCC/libiberty still needs a patch but this takes care of the Haskell side.

comment:5 Changed 17 months ago by Tamar Christina <tamar@…>

In 4de585a5/ghc:

Remove MAX_PATH restrictions from RTS, I/O manager and various utilities

Summary:
This shims out fopen and sopen so that they use modern APIs under the hood
along with namespaced paths.

This lifts the MAX_PATH restrictions from Haskell programs and makes the new
limit ~32k.

There are only some slight caveats that have been documented.

Some utilities have not been upgraded such as lndir, since all these things are
different cabal packages I have been forced to copy the source in different places
which is less than ideal. But it's the only way to keep sdist working.

Test Plan: ./validate

Reviewers: hvr, bgamari, erikd, simonmar

Reviewed By: bgamari

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #10822

Differential Revision: https://phabricator.haskell.org/D4416

comment:6 Changed 17 months ago by Phyx-

Status: patchupstream

underlying toolchain needs fixing now.

comment:7 Changed 14 months ago by bgamari

Milestone: 8.6.18.8.1

These won't be fixed for 8.6, bumping to 8.8.

Note: See TracTickets for help on using tickets.