HTTP

Apache IP ACLs and SourceForge

I was fiddling with some Apache ACLs for one of my projects that I host on SourceForge. They seem to be using Nginx in from of their Apache farms, so ACLs based on the src IP did not want to work at all. Since all I do all day is reverse proxying (at Y!), it was pretty obvious that the src IP was the IP of the Nginx box, and not the client. So, I modified my rules to check the X-Remote-Addr header instead (hopefully their Nginx is smart enough to set this somewhat safely...). I ended up with an Apache rule like

SetEnvIf X-Remote-Addr 1.2.3.4 is_trusted_ip
SetEnvIf X-Remote-Addr 4.3.2.1 is_trusted_ip

<Files somefile.php>
  Order Deny,Allow
  Deny from all
  Allow from env=is_trusted_ip
</Files>

Yes, I know, I could probably use X-Forwarded-For as well, but then I'd risk having multiple IPs in the header.

Drupal Update

To hack around this problem for my SourceForge site (where all src IPs now show up as at 127.0.0.1), I added the following ugly hack to the beginning of my includes/bootstrap.inc file:

if ($_SERVER['HTTP_X_REMOTE_ADDR']) {
    $_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_X_REMOTE_ADDR'];
}

Python httplib.py problems with Akamai

I spent a long night debugging a problem in my current project (Image/content spam prevention), and had problems retrieving URLs that resolved into the Akamai distributed proxy mesh. And yes, I did file this as a SourceForge bug

I don't know who's at fault here, but httplib.py will interpret the HTTP response from an Akamai server (erroneously) as if the socket is closed when the request is finished. This does not happen, because Akamai implements a Connection: keep-alive feature. The following diff to httplib.py (Python 2.3.2) does solve the problem, although I'm not sure if it's the right solution:

--- /usr/lib/python2.3/httplib.py    2003-10-06 09:11:52.000000000 -0700
+++ httplib.py    2004-01-11 03:10:18.000000000 -0800
@@ -355,6 +355,12 @@
         # An HTTP/1.0 response with a Connection header is probably
         # the result of a confused proxy.  Ignore it.
 
+        # Akamai returns HTTP 1.0 headers, with connection: keep-alive, so
+        # the socket will not close.
+        conn = self.msg.getheader('connection')
+        if conn and conn.lower().find("keep-alive") >= 0:
+            return False
+
         # For older HTTP, Keep-Alive indiciates persistent connection.
         if self.msg.getheader('keep-alive'):
             return False

As an alternative, you can subclass the HTTPResponse class, to override the _check_close() method:

class HTTPResponse(httplib.HTTPResponse):
    def _check_close(self):
        if self.version == 11:
            # An HTTP/1.1 proxy is assumed to stay open unless
            # explicitly closed.
            conn = self.msg.getheader('connection')
            if conn and conn.lower().find("close") >= 0:
                return True
            return False

        # Akamai returns HTTP 1.0 headers, with connection: keep-alive, so
        # the socket will not close.
        conn = self.msg.getheader('connection')
        if conn and conn.lower().find("keep-alive") >= 0:
            return False

        # For older HTTP, Keep-Alive indiciates persistent connection.
        if self.msg.getheader('keep-alive'):
            return False

        # Proxy-Connection is a netscape hack.
        pconn = self.msg.getheader('proxy-connection')
        if pconn and pconn.lower().find("keep-alive") >= 0:
            return False

        # otherwise, assume it will close
        return True

httplib.HTTPConnection.response_class = HTTPResponse