Apache 2.0.x + Mongrel + mod_proxy + mod_rewrite configuration

When I decided to try the switch from fcgid to mod_proxy + mongrel for this blog (and now the majority of my projects at work), I discovered there was no example of such configuration or at least not all in one place.
This post will try to fill this lack of a complete example. The distro i’m using here is Debian, but the configuration should be easily ported to yours.

Thereafter I assume you already have a working configuration with :

- Apache 2.0.x - mod_proxy_http (thanks AMicky) - mod_proxy - mod_rewrite

mod_proxy configuration :

There’s not so much to change here, just allow proxying to anyone for local domains.

/etc/apache2/mods-available/proxy.conf

  <Proxy *>
     Order deny,allow
     # next line was Deny from all
     Allow from all
  </Proxy>

A sample vhost file

This configuration is intended to be used with a RubyOnRails application served by Mongrel and deployed with Capistrano.

So you should notice some nice things, like auto-disabling website with a personalized message when there’s a maintenance.html page found, serving static contents directly by apache (no uneeded hit on mongrel) and conserving access to cgi-bin directory (see my previous post about awstats).

/etc/apache2/sites-available/zone.mydomain.tld.conf

<VirtualHost *>
    # Set the domain
    ServerName mydomain.tld
    # Set domain aliases
    ServerAlias www.mydomain.tld 
    ServerAlias web.mydomain.tld

    # Set the DocumentRoot, this is where apache will look for static files
    # This is the public directory of your rails application 
    # As previously stated, my app is deployed with Capistrano, so I set DocumentRoot to the current/public directory
    DocumentRoot /var/www/mydomain.tld/current/public/

    # Set where you want apache put its logs
    # note I use deflate as log format, this is because I use mod_deflate to compress all outputs
    CustomLog /var/www/mydomain.tld/logs/access.log deflate
    ErrorLog /var/www/mydomain.tld/logs/error.log
    
    # if you want to have server-wide statistics,
    # you can also tell apache to log in a common file shared by all your domains 
    CustomLog /var/log/apache2/access.log deflate
    ErrorLog /var/log/apache2/error.log

    # I use utf-8 for all my projects, so I force apache to send the good charset by default.
    # This is needed if you use page caching and want apache serves these with the good charset.
    AddDefaultCharset utf-8

    # Allow limited access to your public directory
    # dont allow user to list directories
    # allow apache to FollowSymlinks
    <Directory /var/www/mydomain.tld/current/public/>
      Options -Indexes FollowSymLinks

      AllowOverride All
      Order allow,deny
      Allow from all
    </Directory>

    # Be sure mod_rewrite is activated for you vhost
    RewriteEngine On

    # If you use applications which resides in cgi-bin, like awstats
    # the following mod_rewrite RewriteRule will allow apache to know it should not proxy these request and the PT switch will allow the ScriptAlias directive to work
    RewriteRule "^/cgi-bin/.*" "$0" [QSA,PT,L]
    RewriteRule "^/awstatsicon/.*" "$0" [PT,L]

    # Set a ScriptAlias for /cgi-bin/ url
    # All url begining with /cgi-bin/ will be served from the specified cgi-bin deirectory
    ScriptAlias /cgi-bin/ /var/www/mydomain.tld/cgi-bin/  
    <Directory /var/www/mydomain.tld/cgi-bin/>
      AllowOverride All
      Options ExecCGI FollowSymLinks
      Order allow,deny
      Allow from all
    </Directory>

   # Do not allow open proxying, allow only requests starting with a /
   <LocationMatch "^[^/]">
      Deny from all
   </LocationMatch>
   
   # Avoid open you server to proxying
   ProxyRequests Off

   # Let apache correctly rewrite redirect
   ProxyPassReverse / http://localhost:8001/

   # Let apache pass the original host not the ProxyPass one
   ProxyPreserveHost On

   # As I mentioned earlier, Capistrano disable_web task allows you to disable your application
   # In fact it only create a simple page named maintenance.html in the directory public/system/ of your application
   # The following mod_rewrite rules will tell apache to directly serve the maintenance.html pages if it find it out.
   RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
   RewriteCond %{SCRIPT_FILENAME} !maintenance.html
   RewriteRule ^.*$ /system/maintenance.html [L]
   
   # For static files it's good to avoid hitting your mongrel process
   # so let apache knows it should serve it directly
   # for a rails application it means, files in images / stylesheets / javascripts
   RewriteRule "^/(images|stylesheets|javascripts)/?(.*)" "$0" [L]
   # Try to match a cached pages
   RewriteRule ^([^.]+)$ $1.html [QSA]
   # if the cached page does not exists
   RewriteCond %{REQUEST_FILENAME} !-f
   # proxy requests to your mongrel instance
   RewriteRule "^/(.*)" "http://localhost:8001/$1" [P,QSA,L]
</VirtualHost>

Wed, 26 Jul 2006 23:28 Posted in

  1. By http://blog.pomozov.info/tags/english/ 27/07/2006 at 12h39


    Small addition to your configuration.

    On Apache 2.2 Mongrel start reject requests after some time (~12 hours).

    Solution add 2 SetEnv options

    Read here
    http://blog.pomozov.info/posts/apache-mongrel-and-proxy-error.html

  2. By Tron Jonathan 27/07/2006 at 14h47


    Thanks for this information, I’ll keep until I can switch to Apache 2.2.

  3. By matthias subik 18/11/2006 at 21h46


    would your code example:

    <Proxy *> Order deny,allow
    1. next line was Deny from all
      Allow from all

    not create an open proxy for spammers??
    how do you restrict proxy calls?

    just my two cents …

  4. By Tron Jonathan 19/11/2006 at 11h15


    Matthias : open proxying is disabled because in the same configuration file (proxy.conf and in my vhost configuration) ProxyRequest is set to Off (which disabe proxy forwarding).

  5. By Tron Jonathan 19/11/2006 at 11h19


    Matthias : I forget to mention proxying to domains on the same apache will be allowed.. which I don’t see as a big deal.

  6. By rizwan 10/02/2007 at 13h08


    thanks for mod rewrite…

  7. By Tom 30/03/2007 at 09h26


    There is something here I’m finding I don’t understand. If I comment out the line for the proxy server, stop the proxy server, and stop Mongrel, my cached pages show up fine (makes sense).

    However, if I uncomment the proxy server line, all of a sudden Apache starts sending those cached-page requests to it. I don’t understand that. Shouldn’t it only send those requests if it doesn’t match the file name on disk?

  8. By Tom 30/03/2007 at 12h04


    If it’s helpful for anyone, I needed to use this to have mod_rewrite match my images:

    %{DOCUMENT_ROOT}/%{REQUEST_FILENAME}

  9. By Lee 04/04/2007 at 23h50


    What do you do if you want to put some of your actions behind SSL using this setup? I’ve got Apache running SSL and I can either secure the entire site or none of the site. I can’t figure out how to secure certain actions. The SslRequirement plugin would be perfect but I can’t get it to work as mongrel seems to think that you are always behind SSL even if you aren’t because of passing in RequestHeader set X_FORWARDED_PROTO ‘https’ Does anyone have any experience getting SSL working?

  10. By Jonathan Tron 05/04/2007 at 10h57


    Hi Lee,
    you have to do 2 vhosts for this to work, one with ssl activated and one without ssl (yes those will share most of their configurations, so use the “Include” directive of apache put those shared configs in an external file), then the SslRequirement plugin will be helpful to force certain actions to go through the ssl enabled site.

    Hope I correctly understand your problem.

  11. By BruceH 26/09/2007 at 17h45


    I’ve been futzing with Mongrel/Apache2.0/PROXYPASS/ModRewrite for over 2 weeks, and have not been able to get it to work. I saw your blog on Goggle, and decided to give your config a try.

    I’ve taken it step-by-step, and this is how far I’ve gotten:

    > ServerName myapp.com
    > ServerAlias www.myapp.com
    > ServerAdmin webmaster@localhost
    > DocumentRoot /var/www/rails/myapp/public
    > ErrorLog /var/www/myapp/log/apache2.log
    > RewriteMap railservers rnd:/var/www/rails/map.txt
    > <Directory /var/www/rails/myapp/public/>
    > Options FollowSymLinks +Includes +ExecCGI
    > AllowOverride None
    > Order allow,deny
    > Allow from all
    > RewriteEngine On
    > RewriteRule $ index.shtml [QSA]
    > RewriteRule "
    /(images|stylesheets|javascripts|pictures|graphics)/?(.)" “$0” [L]
    > RewriteCond %{REQUEST_FILENAME} !-f
    > RewriteRule ^/(.
    )$ http://localhost:${railservers:ports)/$1 [P,L]
    >
    > RewriteLog “/var/www/myapp/log/rewrite.log”
    > RewriteLogLevel 13
    >
    > # Avoid open server to proxying
    > ProxyRequests Off
    > # Pass other requests to mongrel instance
    > #ProxyPass / http://localhost:8000/
    > # Enable reverse proxying
    > ProxyPassReverse / http://localhost:8000/
    > # Let apache pass the original host not the ProxyPass one
    > ProxyPreserveHost On
    >
    > AddType text/html .shtml
    > AddOutputFilter INCLUDES .shtml
    >
    > LogLevel debug
    >
    > CustomLog /var/log/apache2/access.log “combined”
    > ServerSignature On

    When looking at the rewritelog, I get the following:
    > (3) [per-dir /var/www/rails/myapp/public/] add path info postfix: /var/www/rails/myapp/public/entity → /var/www/rails/myapp/public/entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] strip per-dir prefix: /var/www/rails/myapp/public/entity/list → entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] applying pattern ‘^$’ to uri ‘entity/list’
    > (3) [per-dir /var/www/rails/myapp/public/] add path info postfix: /var/www/rails/myapp/public/entity → /var/www/rails/myapp/public/entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] strip per-dir prefix: /var/www/rails/myapp/public/entity/list → entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] applying pattern ‘^/(images|stylesheets|javascripts|pictures|graphics)/?(.*)’ to uri ‘entity/list’
    > (3) [per-dir /var/www/rails/myapp/public/] add path info postfix: /var/www/rails/myapp/public/entity → /var/www/rails/myapp/public/entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] strip per-dir prefix: /var/www/rails/myapp/public/entity/list → entity/list
    > (3) [per-dir /var/www/rails/myapp/public/] applying pattern ‘^/(.*)$’ to uri ‘entity/list’
    > (1) [per-dir /var/www/rails/myapp/public/] pass through /var/www/rails/myapp/public/entity

    It seems to go alright until I check for a valid file, then it passes through anything.

    Any Idea what I’m doing wrong?

  12. By Jonathan Tron 26/09/2007 at 21h24


    BruceH > I’m not sure what exactly your problem is, but if I decrypt correctly your config, you miss a * in :

    
    RewriteRule “/(images|stylesheets|javascripts|pictures|graphics)/?(.)” ”$0” [L]
    
    

    it should be

    
    RewriteRule “/(images|stylesheets|javascripts|pictures|graphics)/?(.*)” ”$0” [L]
    
    

    with an * near the end of matching regexp.

    Hope this help.

  13. By BruceH 26/09/2007 at 23h40


    Sorry, that was a cut’n Paste error on my part.

    What I am trying to do is have the Apache server handle the following:

    1)if someone navigates to the site with a “www.myapp.com”;, they get the index.shtml page (I’m using SSI).

    2)all static content (css,js,images,pdf’s

    3)Anything that doesn’t fall into those 2 catagories, gets redirected to Mongrel at localhost:8000.

    What’s happening is:

    1) all urls with no additional path info get index.shtml passed.

    I arrived at this point by adding 1 directive at a time. Putting the RewriteEngine On anywhere but inside the doesn’t work (I get a compile error )

    2)static files are served by apache.

    3)but any files that fall thru are not routed to Mongrel. Apache tries to serve them. Not happening, I get a “object not found” 404 error. The ReWriteLog shows that the url in the RewriteCond gets parsed, but no results are displayed. (I wish rewritelog codes were defined somewhere so that I can find them! Lines in my response above are delineated by a “>”)

    Or, maybe my proxy is not working.

    Any help is appreciated.
    Thanks.

  14. By BruceH 27/09/2007 at 22h51


    Let me go about this in a different way.

    If you had to take your config above, and add to it the ability of Apache to process server-side Includes(*.shtml files), how would you do it? The solution above does almost everything I need except for that.

    I discovered that I could not put the directive to handle no specific file being requested (www.myapp.com instead of www.myapp.com/index.shtml) being redirected to the index file outside of the directory block. Apache would stop serving the site unless a specific page was requested.

  15. By Jonathan Tron 29/09/2007 at 10h13


    BruceH : Sorry about the delay between responses, but it was a very busy week for me.

    About serving *.shtml file I would try something like :

     
     
    RewriteRule ([^.]+\.shtml) /your/shtml/dir/path/$1 [QSA,L]  
    

    If you want to be more specific about what files should be picked up as shtml :

     
     
    # Only shtml at root of your site 
    RewriteRule ^([^/.]+\.shtml) /your/shtml/dir/path/$1 [QSA,L] 
    # Only index.shtml at root of your site 
    RewriteRule ^index.shtml$ /your/shtml/dir/path/index.shtml [QSA,L] 
     
    

    I didn’t try them extensively, but it should work. You should put them before any other Rewrites that can have a match.

  16. By BruceH 01/10/2007 at 16h42


    After much trial and tribulation I finally figured out why my rewritecond wasn’t working.

    In the rewritecond, the regex to extract the Ruby string (i.e. “/state/
    list”) MUST include the leading slash. In other words, "^(.)$ not "^/
    (.
    )$.

    The littlest things…

  17. By Jonathan Tron 02/10/2007 at 14h00


    BruceH > glad you finally find the problem.

  18. By AMicky 20/03/2008 at 18h30


    hi jonathan !
    your tuto helped me a lot to deploy my application… but a spent one day finding out that i should install and enable proxy_http_module in addition to proxy_module.
    whithout enabling this module apache server will return 403 error (forbiden).
    i think it will help the futur reader of this tuto if you add this information at the begining.
    i fixed this out by reading this post: http://www.parsed.org/tip/146/

    Thanks for this tuto, it’s working well :D

  19. By WarrenWeiss 24/06/2008 at 23h07


    I’m experiencing the same issue, I believe, that “Lee” was having with SSL.


    I’m adding a maintenance page to an existing configuration that uses a statement to rewrite all requests to a secure connection:



    RewriteCond %{SERVER_PORT} !^443$
    RewriteCond %{REQUEST_URI} !^/index.html$
    RewriteRule ^/(.*) https://%{SERVER_NAME}/$1[R,L]

    When I add the condition statements for a maintenance page:



    RewriteCond %{DOCUMENT_ROOT}/maintenance.html -f
    RewriteCond %{SCRIPT_FILENAME} !maintenance.html
    RewriteRule ^.*$ %{DOCUMENT_ROOT}/maintenance.html [L]


    The URL is rewritten to the index page, but on the default port of 80 (non-secure).


    What I want apache to do is rewrite to the maintenance page if it exists and then stop.

  20. By piyush 07/10/2008 at 14h43


    hi all,

    i am new to new to apache rewriting rules.

    my requirement is some thing like this.

    I am working for a rails web application that have upload functionality in it.

    I am using apache as web server and 4 mongrel cluster as application server for load balancing.

    now i want to delegate the task(feature) for uploading to specific mongrel instances to handle for better performance and for other tasks remaining mongrel clusters should be used.

    for that i need to write the rewrite rule in my apache configuration.

    can any one help me to get it done..

    Thanks in advance..

  21. By exdee 04/09/2009 at 08h03


    Thank u very much!

  22. By Siva 10/08/2010 at 15h55


    Hi,

    Nice explanation.. This works for me. But how can i add a wordpress blog rewrite rule with in this configuration. Can u please help me on this ?

    thanks,
    Siva

Comment Apache 2.0.x + Mongrel + mod_proxy + mod_rewrite configuration


RSS