Opened 6 years ago

Closed 5 years ago

#8251 closed task (fixed)

Validate submodule references during pre-receive hook

Reported by: hvr Owned by: hvr
Priority: normal Milestone:
Component: Trac & Git Version: 7.6.3
Keywords: admin git Cc: thoughtpolice
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: #8545 #8544 Differential Rev(s):
Wiki Page:

Description

Right now, you can push a commit introducing a not-yet existing Git submodule commit reference.

However, such commits can be rejected for submodule repositories hosted at git.haskell.org, thus forcing the user to push the submodule first, and only after that push the submodule-affecting commit in ghc.git

This kind of sanity checking ghc.git commits will become more important, if more repositories are converted into Git submodules.

Change History (7)

comment:1 Changed 6 years ago by hvr

Moreover, when adding a new submodule ref to a repository hosted at git.haskell.org another hook script could add a hidden ref or a Git note to the referenced commit, effectively providing a back-ref, and ensuring the commit doesn't become dangling and subsequently garbage collected.

comment:2 Changed 6 years ago by hvr

Component: NoneTrac & Git

comment:3 Changed 6 years ago by hvr

Another idea: to avoid updating submodule refs by accident (which has happened a few times already), the respective commit message ought contain the word "submodule" or maybe "update(s) submodule" as a safe-guard.

comment:4 Changed 6 years ago by hvr

I've implemented a server-side hook and enabled it for git-sandbox.git and ghc.git The hacked-up implementation can be found in this gist

Some details:

  • only commits becoming reachable via non-wip/-branches ("persistent branches") are validated
  • if a non-merge commit results in a modified gitlink-type, or a merge commit results in a gitlink-type entry which is different from all its ancestors, then
    • the git commit message body must contain the string submodule
    • each touched gitlink is
      • looked up in the .gitmodules file and resolved to the respective git repo hosted git.haskell.org,
      • the referenced submodule commit must be reachable from a "persistent branch" (i.e. a non-wip/-branch)

This scheme relies on "persistent branches" not being allowed to be deleted or non-fast-fwd updated.

comment:5 Changed 6 years ago by hvr

Here's a sample session demonstrating the new server-side validation checks, when trying to update the Win32 package to version 2.3.0.2 (from previously 2.3.0.1);

After having synced up libraries/Win32 to point version 2.3.0.2 of Win32 we have the following change to commit to ghc.git (git add libaries/Win32):

  • libraries/Win32

    diff --git a/libraries/Win32 b/libraries/Win32
    index 1e909ad..c51e81a 160000
    a b  
    1 Subproject commit 1e909adb06b766e107148b8b37a4a9f9e50baf74
     1Subproject commit c51e81a43cd5e9540453bd5ca6da8992245a4774
  1. Pushing a commit touching a submodule ref w/o including the "submodule" safe-word:
    $ git commit -s -m "Update Win32-2.3.0.2"
    $ git push
    ...
    remote: performing tab-check...
    remote: performing submodule-ref update validations...
    remote: Submodule update(s) detected in ff458fc260fd05ebcb7db3294cbb1ec623e6525e:
    remote: *FAIL* commit message does not contain magic 'submodule' word
    remote: hooklet hooks/update.secondary.d/check-submodule-refs failed
    remote: hooks/update.secondary died
    remote: error: hook declined to update refs/heads/master
    To ssh://git@git.haskell.org/ghc.git
     ! [remote rejected] master -> master (hook declined)
    error: failed to push some refs to 'ssh://git@git.haskell.org/ghc.git'
    
  1. Rewrite the commit message to include the magic word, and try pusing again:
    $ git commit --amend -s -m "Update submodule to Win32-2.3.0.2"
    $ git push
    ...
    remote: performing tab-check...
    remote: performing submodule-ref update validations...
    remote: Submodule update(s) detected in a24ef02140dabb42dfbd2b649434017af6bb6fca:
    remote:  libraries/Win32 => c51e81a43cd5e9540453bd5ca6da8992245a4774
    remote: *FAIL* commit not found in submodule repo ('../packages/Win32.git')
    remote:        or not reachable from persistent branches
    remote: hooklet hooks/update.secondary.d/check-submodule-refs failed
    remote: hooks/update.secondary died
    remote: error: hook declined to update refs/heads/master
    To ssh://git@git.haskell.org/ghc.git
     ! [remote rejected] master -> master (hook declined)
    error: failed to push some refs to 'ssh://git@git.haskell.org/ghc.git'
    
  1. Making sure the Win32 sub-repo actually contains the new commit ghc.git is going to refer to, and try pushing again:
    $ (cd libraries/Win32 ; git push origin HEAD:ghc-head)
    ...
    To ssh://git@git.haskell.org/packages/Win32.git
       1e909ad..c51e81a  ghc-head -> ghc-head
    
    $ git push
    ...
    remote: performing tab-check...
    remote: performing submodule-ref update validations...
    remote: Submodule update(s) detected in 696bfc4ba5fce6b75cc91bcb67c5d0a3c9f29bd2:
    remote:  libraries/Win32 => c51e81a43cd5e9540453bd5ca6da8992245a4774
    remote:  OK
    remote: mirroring ssh://git@git.haskell.org/ghc to ssh://git@github.com/ghc/ghc ...
    remote: To ssh://git@github.com/ghc/ghc
    remote:    3099e40..696bfc4  master -> master
    remote: running notifier
    To ssh://git@git.haskell.org/ghc.git
       3099e40..696bfc4  master -> master
    

The last git push finally succeeded!

comment:6 Changed 6 years ago by hvr

comment:7 in reply to:  1 Changed 5 years ago by thomie

Resolution: fixed
Status: newclosed

Please reopen if needed:

Moreover, when adding a new submodule ref to a repository hosted at git.haskell.org another hook script could add a hidden ref or a Git note to the referenced commit, effectively providing a back-ref, and ensuring the commit doesn't become dangling and subsequently garbage collected.

Note: See TracTickets for help on using tickets.