One of cryptic warnings found in ModSecurity audit.log is
[msg "ModSecurity does not support content encodings"]
Clearly, it says ModSecurity doesn’t support content encodings. That is, it cannot inspect gzipped content. This is problematic especially when ModSecurity is running on reverse proxy. The question is, “how can I inspect, then?” There are some very useful advice from the developer in the list archive. Bad news is, it doesn’t work. Some claims it worked, but they skip deflate process by SetEnvIfNoCase, which is not an option for me.
My goal is:
- Compress by Content-Type, not by file extension because sometime file extension is not available (/foo.php?imageid=1234). You cannot use SetEnvIfNocase because it cannot access to response headers.
- Do not compress images, video and other pre-compressed files (compress all except a few others).
- Do not compress before ModSecurity inspect the content (the goal).
After hours and days to find why it doesn’t work, I finally found the problem. In modextfilter.c, it looks for ext filter by f->frec->name, which is not the name of ext filter but FilterProvider’s name when it should have looked for it by “nodeflate” in my case.
583 /* look for the user-defined filter */
584 ctx->filter = find_filter_def(f->r->server, f->frec->name);
I’m not an apache guru in any way, I might be wrong.
Three tricks:
- Even though configtest doesn’t complain, ExtFilterDefine is only valid in server config context
- DO NOT “SetOutputFilter DEFLATE”. It compresses all contents, ignoring nodeflate
- Define two ext filters
Here is the WORKING config.
LoadModule ext_filter_module libexec/apache22/mod_ext_filter.so
LoadModule filter_module libexec/apache22/mod_filter.so
...
# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html
# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip
# IE is ok, but looked like Netscape, so we reset it
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# We need to force compression since we will remove the
# request Content-Encoding/TE headers which mod_deflate
# looks for.
SetEnvIfNoCase Accept-Encoding gzip force-gzip
SetEnvIfNoCase TE gzip force-gzip
# Disable compression from backend
RequestHeader unset Accept-Encoding
RequestHeader unset TE
# Make sure caching still works
Header append Vary User-Agent env=!dont-vary
# XXX be sure to define ExtFilterDefine in server config context!
# not in virtual host, not directory, etc
ExtFilterDefine nodeflate mode=output \
cmd=/usr/bin/true \
enableenv=SomeVarThatWillNeverBeSet
# XXX workaround a bug in mod_ext_filter.
# it looks up extfilter by FilterProvider's name, not user defined ext filter.
# it complains "couldn't find definition of filter 'compress'"
# Define the same name of FilterProvider.
ExtFilterDefine compress mode=output \
cmd=/usr/bin/true \
enableenv=SomeVarThatWillNeverBeSet
# Setup the filter to compress the response if we saw
# an Accept-Encoding/TE header and marked it as force-gzip
FilterDeclare compress CONTENT_SET
FilterProvider compress DEFLATE env=force-gzip =1
# Overwrite the "deflate" filter with the "nodeflate"
# for various content types so they will not be compressed.
# This is done as a hack because there is no way to
# use more than one condition with FilterProvider above.
FilterProvider compress nodeflate Content-Type $image/
FilterProvider compress nodeflate Content-Type $application/
FilterProvider compress nodeflate Content-Type $video/
FilterProvider compress deflate Content-Type $application/javascript
FilterProvider compress deflate Content-Type $application/json
FilterProtocol compress "change=yes"
FilterDeclare modsec CONTENT_SET
FilterProvider modsec modsecurity_out env=modsec-ignore !=1
FilterChain modsec compress
I tested this on FreeBSD, so, use “cmd=/bin/true” on Linux and others (“/bin/sh” should work and is more portable, if you prefer).
When Apache 2.4 is released, it will be much simpler. See “Expression parser for Apache”
Recent Comments