Request via proxy server doesn't send absolute URI

[bringert]

When a request is sent via a proxy server, the absolute URI is not sent as the request URI, even though it should. This is because we always send just the absolute path, and have no way of knowing that this is a proxy server request. See the emails in the comments for more information.

[bringert]

---------- Forwarded message ----------
From: Daniel McAllansmith <dm.maillists@gmail.com>
Date: Mon, Mar 31, 2008 at 11:13 PM
Subject: Re: Haskell HTTP and proxy servers
To: Eric Kow <eric.kow@gmail.com>
Cc: Björn Bringert <bringert@cs.chalmers.se>, Satoshi Nakamura <snak@snak.org>


On Tue, 01 Apr 2008 03:34:07 you wrote:
> Hi,
>
> I was briefly in contact with Bjorn Bringert about the http package
> because it (and anything that uses it, e.g. cabal and now darcs) does
> not play nice with our University wide proxy server.

Yeah, that was a known risk... I thought that I'd stated that in the patch
description but it looks like I misremembered.
>
> Now I don't have much experience with reading RFCs, but I do know that
> unpulling your patch 'Send abs_path as Request-URI to comply with HTTP
> 1.1 spec.' "fixes" the problem (presumably because when talking to a
> proxy, we indeed do want to send absoluteURI).

Correct.  The relevant spec is RFC2626 Section 5.1.2.
In short:
Request-URI    = "*" | absoluteURI | abs_path | authority
Requests of server information need "*"
Requests via a proxy need absoluteURI
Requests to the origin server or a gateway need abs_path
authority is used for the CONNECT method (which the library doesn't support).

>
> So I was wondering if you might consider having a second look at your
> patch and trying to work out a good way of making http play nice with
> proxy servers?

You need to add a mechanism to discriminate between the different types of
requests.
I've attached a patch (against current darcs) to add a RequestURI that allows
that discrimination and generates the appropriate Request-URI...
unfortunately I'm short on time so
it's not plumbed in.

It should be fairly straight forward to get it going, looks like you'll need
to remove the call to fixReq.  Otherwise I might be able to finish it off...
can't promise when though.

Of course the problem is that it's a major change to the interface so your
darcs/cabal etc are still going to be knackered until they upgrade.  I
believe Requests are made directly via the data constructor so I don't think
there is a convenient place to put in a compatibility adaptor.  Björn will
probably have better idea of whether it's possible to making a change in a
compatible
manner.

I've also CC'd Satoshi who, along with me, hit upon origin servers that failed
due to http not conforming to the spec and prompted the abs_path patch in the
first place.

>
> Many thanks!

No problemo.  Any questions, ask away.

Cheers
Dan

[bringert]

Request URI patch from Daniel McAllansmith?

[bringert]

Eric Kow worked a bit more on this, but he still doesn't seem to think that it is the 'correct' way of doing it.

---------- Forwarded message ----------
From:  <E.Y.Kow@brighton.ac.uk>
Date: Wed, Jun 18, 2008 at 4:46 PM
Subject: darcs patch: Provisional code for creating request-ur... (and 2 more)
To: bjorn@bringert.net
Cc: dm.maillists@gmail.com


I have no idea if this is the right way to do this or not.

But with these changes (and some changes to cabal-install), I was able
to get a cabal-install that worked behind my university proxy server.
Anyway, hopefully this will prod somebody into doing the equivalent
change the 'correct' way?

(btw, Daniel, you might want to remove the extra single quotes from your darcs
author preferences)

Mon Mar 31 21:24:00 BST 2008  'Daniel McAllansmith <dm.maillists@gmail.com>'
 * Provisional code for creating request-uris that conform with RFC2616.
 Currently only abs_path request-uris are used.  This means that you can't get
 resources via a proxy, which requires the use of absoluteURI request-uris.
 This provisional code adds a RequestURI datatype that allows you to
 discriminate between requests to an origin server, a proxy and requesting
 server info.  Further changes to actually use this code are required, at
 the least replacing the URI in Request with RequestURI and removing the use
 of fixReq in simpleHTTP.

Wed Jun 18 15:41:46 BST 2008  Eric Kow <E.Y.Kow@brighton.ac.uk>
 * Tidy up RequestURI.

Wed Jun 18 15:42:25 BST 2008  Eric Kow <E.Y.Kow@brighton.ac.uk>
 * Plug RequestURI in (for better proxy support).

[bringert]

Eric Kow's provisional patch

[sof]

The HTTP-4000.x line of releases fixes this longer-standing issue,

http://projects.haskell.org/http/