UPDATE: Ivan Ristic implemented SecDisableBackendCompression in trunk. With this directive, you don’t need the hack below. See my post to the list.

One of cryptic warnings found in ModSecurity audit.log is

  [msg "ModSecurity does not support content encodings"]

Clearly, it says ModSecurity doesn’t support content encodings. That is, it cannot inspect gzipped content. This is problematic especially when ModSecurity is running on reverse proxy. The question is, “how can I inspect, then?” There are some very useful advice from the developer in the list archive. Bad news is, it doesn’t work. Some claims it worked, but they skip deflate process by SetEnvIfNoCase, which is not an option for me.

My goal is:

• Compress by Content-Type, not by file extension because sometime file extension is not available (/foo.php?imageid=1234). You cannot use SetEnvIfNocase because it cannot access to response headers.
• Do not compress images, video and other pre-compressed files (compress all except a few others).
• Do not compress before ModSecurity inspect the content (the goal).

After hours and days to find why it doesn’t work, I finally found the problem. In modextfilter.c, it looks for ext filter by f->frec->name, which is not the name of ext filter but FilterProvider’s name when it should have looked for it by “nodeflate” in my case.

583     /* look for the user-defined filter */
584     ctx->filter = find_filter_def(f->r->server, f->frec->name);

I’m not an apache guru in any way, I might be wrong.

Three tricks:

• Even though configtest doesn’t complain, ExtFilterDefine is only valid in server config context
• DO NOT “SetOutputFilter DEFLATE”. It compresses all contents, ignoring nodeflate
• Define two ext filters

Here is the WORKING config.

    LoadModule ext_filter_module libexec/apache22/mod_ext_filter.so
    LoadModule filter_module libexec/apache22/mod_filter.so
    ...
    # Netscape 4.x has some problems...
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    # IE is ok, but looked like Netscape, so we reset it
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

    # We need to force compression since we will remove the
    # request Content-Encoding/TE headers which mod_deflate
    # looks for.
    SetEnvIfNoCase Accept-Encoding gzip force-gzip
    SetEnvIfNoCase TE gzip force-gzip

    # Disable compression from backend
    RequestHeader unset Accept-Encoding
    RequestHeader unset TE

    # Make sure caching still works
    Header append Vary User-Agent env=!dont-vary

    # XXX be sure to define ExtFilterDefine in server config context!
    # not in virtual host, not directory, etc
    ExtFilterDefine nodeflate mode=output \
        cmd=/usr/bin/true \
        enableenv=SomeVarThatWillNeverBeSet
    # XXX workaround a bug in mod_ext_filter.
    # it looks up extfilter by FilterProvider's name, not user defined ext filter.
    # it complains "couldn't find definition of filter 'compress'"
    # Define the same name of FilterProvider.
    ExtFilterDefine compress mode=output \
        cmd=/usr/bin/true \
        enableenv=SomeVarThatWillNeverBeSet

    # Setup the filter to compress the response if we saw
    # an Accept-Encoding/TE header and marked it as force-gzip
    FilterDeclare compress CONTENT_SET
    FilterProvider compress DEFLATE env=force-gzip =1

    # Overwrite the "deflate" filter with the "nodeflate"
    # for various content types so they will not be compressed.
    # This is done as a hack because there is no way to
    # use more than one condition with FilterProvider above.
    FilterProvider compress nodeflate Content-Type $image/
    FilterProvider compress nodeflate Content-Type $application/
    FilterProvider compress nodeflate Content-Type $video/
    FilterProvider compress deflate Content-Type $application/javascript
    FilterProvider compress deflate Content-Type $application/json
    FilterProtocol compress "change=yes"

    FilterDeclare  modsec CONTENT_SET
    FilterProvider modsec modsecurity_out env=modsec-ignore !=1

    FilterChain modsec compress

I tested this on FreeBSD, so, use “cmd=/bin/true” on Linux and others (“/bin/sh” should work and is more portable, if you prefer).

When Apache 2.4 is released, it will be much simpler. See “Expression parser for Apache”.