Opened 4 years ago

Closed 4 years ago

#10205 closed bug (fixed)

On Windows ghc-pkg always reports cache out of date

Reported by: hgolden Owned by:
Priority: normal Milestone: 7.10.2
Component: ghc-pkg Version: 7.10.1
Keywords: Cc: RyanGlScott
Operating System: Windows Architecture: Unknown/Multiple
Type of failure: Incorrect result at runtime Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D990
Wiki Page:

Description

When I ran ghc-pkg list on Windows, it reported that the global and user caches were out of date. As instructed, I recached both caches. Despite this, I continued to get the same error message. It appears that the timestamp on the package.conf.d directory is always a few milliseconds later than the timestamp of the package cache, even after recaching, as shown by the log excerpt below.

Original error:

C:\Users\hgolden\AppData\Roaming>ghc-pkg check
WARNING: cache is out of date: C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
WARNING: cache is out of date: C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
[snip...]

Checking timestamps:

C:\Users\hgolden\AppData\Roaming>ghc-pkg -v2 list
Timestamp 2015-03-27 21:18:29.5247984 UTC for C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\pa
ckage.cache
Timestamp 2015-03-27 21:18:29.5277984 UTC for C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d (N
EWER than cache)
WARNING: cache is out of date: C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
[snip...]
Timestamp 2015-03-27 21:15:48.6285984 UTC for C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache
Timestamp 2015-03-27 21:15:48.6835984 UTC for C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d (NEWER than c
ache)
WARNING: cache is out of date: C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
[snip...]

Recaching as instructed:

C:\Users\hgolden\AppData\Roaming>ghc-pkg -v2 recache --user
[snip...]
modifying: Just "C:\\Users\\hgolden\\AppData\\Roaming\\ghc\\i386-mingw32-7.10.1\\package.conf.d"
flag db stack: ["C:\\Users\\hgolden\\AppData\\Roaming\\ghc\\i386-mingw32-7.10.1\\package.conf.d"]
writing cache C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\package.cache
C:\Users\hgolden\AppData\Roaming>ghc-pkg -v2 recache --global
[snip...]
modifying: Just "C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\\lib\\package.conf.d"
flag db stack: ["C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\\lib\\package.conf.d"]
writing cache C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache

Looking at timestamps after recaching:

C:\Users\hgolden\AppData\Roaming>ghc-pkg -v2 list
Timestamp 2015-03-27 21:21:50.3341984 UTC for C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\pa
ckage.cache
Timestamp 2015-03-27 21:21:50.3371984 UTC for C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d (N
EWER than cache)
WARNING: cache is out of date: C:\Users\hgolden\AppData\Roaming\ghc\i386-mingw32-7.10.1\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
[snip...]
Timestamp 2015-03-27 21:23:05.4329984 UTC for C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache
Timestamp 2015-03-27 21:23:05.4359984 UTC for C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d (NEWER than c
ache)
WARNING: cache is out of date: C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
C:\Users\hgolden\AppData\Roaming>

Timestamps have changed, but each directory's timestamp is still a few milliseconds after each cache file's timestamp.

Attachments (1)

ghc-pkg-list.log (25.4 KB) - added by svenpanne 4 years ago.
Output of "ghc-pkg list -v" with 7.10rc2 on Windows

Download all attachments as: .zip

Change History (22)

comment:1 Changed 4 years ago by dmcclean

This happens to me too, on Windows 7 Home Premium, SP1, x64.

comment:2 Changed 4 years ago by thomie

Architecture: x86Unknown/Multiple
Milestone: 7.10.2

comment:3 Changed 4 years ago by Kludgy

NTFS modified dates are somewhat unreliable. The results also vary depending on how a particular filesystem is configured.

It is possible that Windows users seeing this issue persist after a recache have sub-second precision enabled, making ghc-pkg much more sensitive to the errors.

EDIT: Not sure if it helps, but certain operations (like renaming a file with a directory) cause the directory modified time to be updated, usually leading to the directory modified time exceeding the file modified time.

Last edited 4 years ago by Kludgy (previous) (diff)

comment:4 Changed 4 years ago by rpaladugu1

You can remove the following message by opening the command prompt as administrator and doing the "ghc-pkg recache" in the C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib folder. It worked for me and the message was gone.

WARNING: cache is out of date: C:/Program Files/MinGHC-7.10.1/ghc-7.10.1\lib\package.conf.d\package.cache ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.

comment:5 Changed 4 years ago by thoughtpolice

Yes, to be clear, was this with the binary distribution, MinGW, Haskell Platform, etc? We shouldn't ever need administrative privileges in the regular binary distribution for anything, so this may not be our bug if that fixes it?

comment:6 Changed 4 years ago by thoughtpolice

Status: newinfoneeded

comment:7 in reply to:  5 Changed 4 years ago by hgolden

Replying to thoughtpolice:

Yes, to be clear, was this with the binary distribution, MinGW, Haskell Platform, etc? We shouldn't ever need administrative privileges in the regular binary distribution for anything, so this may not be our bug if that fixes it?

In my original report, there are two separate sets of messages: One is for the global directory and the other is for my personal directory. As a non-administrator user, I don't have write access to the global directory, since it is a subdirectory of C:\Program Files (under Windows 7 or later). However, I do have access to my own directory. I am getting the out-of-date messages for both files.

As I recall, when I ran into this situation, I had administrative privileges on my PC. In my company's wisdom, I no longer have administrative privileges, so I can't test the proposed fix in comment #4, but I believe I already did that in the original report.

Storing the global cache (or any other part of GHC) in a subdirectory of C:\Program Files will not work if the user doesn't have administrative privileges, at least under Windows 7 or later.

comment:8 Changed 4 years ago by rpaladugu1

Without admin privileges, you can not even install it in c:\Program Files folder or anywhere else. The installer did ask admin privileges before starting the installation process if you logged in as standard user. The default location "i.e.C:\Program Files" is optional for MingGW or Haskell Platform during the install time. You can choose to override and install it somewhere. I have tested both MingGW and Haskell Platform and indeed was the case. As such there is no issue with the platform. However, doucmentation could be updated to highlight this issue so that there is a workaround.

comment:9 Changed 4 years ago by thomie

Status: infoneedednew

I am looking into this.

comment:10 Changed 4 years ago by thomie

Owner: set to thomie

comment:11 Changed 4 years ago by thoughtpolice

Milestone: 7.10.27.10.3

Moving to 7.10.3 milestone - if you think this is an error (or a showstopping bug), please remilestone it to 7.10.2 and let us know why.

comment:12 Changed 4 years ago by thomie

Differential Rev(s): Phab:D990
Milestone: 7.10.37.10.2
Status: newpatch

I think we can fix this still for 7.10.2.

comment:13 in reply to:  3 Changed 4 years ago by thomie

Replying to Kludgy:

EDIT: Not sure if it helps, but certain operations (like renaming a file with a directory) cause the directory modified time to be updated, usually leading to the directory modified time exceeding the file modified time.

It was exactly this!

comment:14 Changed 4 years ago by Austin Seipp <austin@…>

In f0636562908236f6ce9bf91796bc952534074a61/ghc:

Fix ghc-pkg reports cache out date (#10205)

See Note [writeAtomic leaky abstraction].

GHC on Linux already received a patch for this bug in
e0801a0fb342eea9a312906eab72874d631271cf. On Windows several cabal tests
were hitting the bug, causing validate failures, but we never noticed
because of all the other tests that were failing on Windows. And it
didn't start happening till `getModificationTime` received sub-second
resolution support on Windows in
5cf76186d373842bf64d49cecb09e0a9ddce3203.

Since there are regression tests already, I am not adding another one.
But for good measure, here is a script that shows the bug without
needing to do a full validate run:

  DB=/tmp/package.conf.d.test
  GHC_PKG=ghc-pkg #utils/ghc-pkg/dist/build/tmp/ghc-pkg
  LOCAL_GHC_PKG="${GHC_PKG} --no-user-package-db --global-package-db=${DB}"
  while true; do
    rm -rf ${DB}
    ${LOCAL_GHC_PKG} init "${DB}"
    ${LOCAL_GHC_PKG} list
  done

If you see "WARNING: cache is out of date" after a few seconds, the bug
is not fixed.

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D990

GHC Trac Issues: #10205

comment:15 Changed 4 years ago by thomie

Status: patchmerge

comment:16 Changed 4 years ago by thoughtpolice

Resolution: fixed
Status: mergeclosed

Merged to ghc-7.10.

comment:17 Changed 4 years ago by svenpanne

Owner: thomie deleted
Resolution: fixed
Status: closednew

This is still broken in the 7.10rc2 on Windows.

$ which ghc-pkg
/c/Program Files/Haskell Platform/7.10.2/bin/ghc-pkg

Changed 4 years ago by svenpanne

Attachment: ghc-pkg-list.log added

Output of "ghc-pkg list -v" with 7.10rc2 on Windows

comment:18 Changed 4 years ago by thomie

Timestamp 2015-07-06 19:15:52 UTC for c:/Program Files/Haskell Platform/7.10.2\lib\package.conf.d\package.cache
Timestamp 2015-07-07 07:51:56.8491787 UTC for c:/Program Files/Haskell Platform/7.10.2\lib\package.conf.d (NEWER than cache)
WARNING: cache is out of date: c:/Program Files/Haskell Platform/7.10.2\lib\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.

Try running 'ghc-pkg recache' as the message says. Does it fix the problem?

comment:19 Changed 4 years ago by RyanGlScott

Cc: RyanGlScott added

comment:20 Changed 4 years ago by svenpanne

OK, running 'ghc-pkg recache' fixes the problem, but:

a) You have to do it as an Administrator, which might not be an option for all users.

b) The Windows installer for 7.10rc2 shouldn't install the package DB like this, forcing users to recache as an Administrator directly after installation.

comment:21 Changed 4 years ago by thomie

Resolution: fixed
Status: newclosed

Ok, then it is not a reappearance of this bug. Sorry for leading you this way.

I have opened the following ticket in the Haskell platform issue tracker for you: https://github.com/haskell/haskell-platform/issues/189.

Note: See TracTickets for help on using tickets.