About the Site

This weblog is edited and run by members of reallyenglish, a company offering a total English learning solution based in London, Beijing, Shanghai and Tokyo. Visit our corporate site to know more about what we do.

Notes are posted by members from various cultural and geographical backgrounds, and the topics range from education, business and international communication to software development, the internet culture, and more.


Masatomo Nakano http://twitter.com/masatomon simonl davida jeremyw Go Kameda gavin b No name tomoyukis


Recent Comments

  • Tomoyuki Sakurai: we've been using vether(4) successfully since 4.7. http://old.nabble.com/kernel-6318:-arp-cache-problem-bridge%284%29-with-vether%284%29-td27635451.html (the bug read more
  • RofR: Hi everyone Where about to setup serveral OBSD 4.9 or read more
  • Tomoyuki Sakurai: the above configuration has been working fine since 4.6. read more
  • Jean Aumont: Hi Tomoyuki, If you capture the packets with tcpdump and read more
  • Tomoyuki Sakurai: you will not see OSPF packets on physical interface. > read more
  • Jean Aumont: Hi everyone, I have been trying to pass OSPF in read more
  • Anonymous: without any other information, all i can say is doing read more
  • Jean Aumont: I have a similar set-up, but my 2 end points read more
  • illouca: thank you for the patch !! read more
  • Go Kameda: Glad to hear that - im afraid i cannot really read more

Recently in Security Category

iPhone 4 is able to make VPN connection and send all traffic through it, which is useful when you use public WiFi and/or need intranet resources.

Installing OpenBSD 4.9

See the FAQ. You need to install the kernel and userland sources as well to build npppd and PIPEX kernel.

Building PIPEX kernel and installing npppd

At the time of this writing, npppd is not installed by default. npppd requires "option PIPEX" in kernel config file, which is not enabled by default. Follow the instruction in /usr/src/usr.sbin/npppd/HOWTO_PIPEX_NPPPD.txt after you extract the source code. Note that amd64 users need to patch npppd at the moment. The patch below was from the author. Patch at your own risk.

> cd && fetch http://journal.reallyenglish.com/2011/05/13/privsep.diff.txt
> cd /usr/src/usr.sbin/npppd/npppd
> sudo patch < ~/privsep.diff.txt



interface_list: tun1
interface.tun1.ip4addr: $ip.add.re.ss
pool.dyna_pool: $ip.add.re.ss/$subnet
lcp.mru: 1400
ipcp.dns_primary: $dns.add.re.ss


Choose authentication method. CSV file is good for testing.

auth.method: mschapv2 chap
auth.local.realm_list: local
auth.local.realm.acctlist: /etc/npppd/npppd-users.csv
realm.local.concentrate: tun1

Create a CSV file, /etc/npppd/npppd-users.csv which looks like:

user,secret,$ip.add.re.ss,,memo for user

If you have working RADIUS server,

auth.radius.realm_list: radius
auth.radius.realm.server.secret: $radius_secret
realm.radius.concentrate: tun1

Note that you need clear text password or NT password in the RADIUS backend when mschapv2 is used.

VPN protocol

Choose VPN protocol.

pptpd.enabled: true

Connecting to npppd

Run npppd in debug mode.

# npppd -d
2011-05-21 17:54:03:NOTICE: Starting npppd pid=30068 version=5.0.0                                               
2011-05-21 17:54:03:NOTICE: Load configuration from='/etc/npppd/npppd.conf' successfully.
2011-05-21 17:54:03:WARNING: write() failed in in_route0 on RTM_ADD : File exists
2011-05-21 17:54:03:INFO: tun1 Started ip4addr=
2011-05-21 17:54:03:INFO: pool name=default dyn_pool=[] 
2011-05-21 17:54:03:INFO: Added 1 routes for new pool addresses
2011-05-21 17:54:03:INFO: Loading pool config successfully.
2011-05-21 17:54:03:INFO: realm name=radius(radius) Loaded configuration timeout=9 nserver=1
2011-05-21 17:54:03:INFO: Listening /var/run/npppd_ctl (npppd_ctl)
2011-05-21 17:54:03:INFO: l2tpd Listening (L2TP LNS) [L2TP]
2011-05-21 17:54:03:INFO: l2tpd Listening [::]:1701/udp (L2TP LNS) [L2TP]
2011-05-21 17:54:03:INFO: pptpd Listening (PPTP PAC) [PPTP]
2011-05-21 17:54:03:INFO: pptpd Listening (PPTP PAC)
2011-05-21 17:54:03:INFO: tun1 is using ipcp=default(1 pools).

When everything works, you will see:

2011-05-21 17:58:00:NOTICE: ppp id=1 layer=base logtype=TUNNELSTART user="user" duration=3sec layer2=PPTP layer2from= auth=MS-CHAP-V2  ip= iface=tun1
2011-05-21 17:58:00:NOTICE: ppp id=1 layer=base Using pipex=yes

UPDATE: Ivan Ristic implemented SecDisableBackendCompression in trunk. With this directive, you don’t need the hack below. See my post to the list.

One of cryptic warnings found in ModSecurity audit.log is

  [msg "ModSecurity does not support content encodings"]

Clearly, it says ModSecurity doesn’t support content encodings. That is, it cannot inspect gzipped content. This is problematic especially when ModSecurity is running on reverse proxy. The question is, “how can I inspect, then?” There are some very useful advice from the developer in the list archive. Bad news is, it doesn’t work. Some claims it worked, but they skip deflate process by SetEnvIfNoCase, which is not an option for me.

My goal is:

  • Compress by Content-Type, not by file extension because sometime file extension is not available (/foo.php?imageid=1234). You cannot use SetEnvIfNocase because it cannot access to response headers.
  • Do not compress images, video and other pre-compressed files (compress all except a few others).
  • Do not compress before ModSecurity inspect the content (the goal).

After hours and days to find why it doesn’t work, I finally found the problem. In modextfilter.c, it looks for ext filter by f->frec->name, which is not the name of ext filter but FilterProvider’s name when it should have looked for it by “nodeflate” in my case.

583     /* look for the user-defined filter */
584     ctx->filter = find_filter_def(f->r->server, f->frec->name);

I’m not an apache guru in any way, I might be wrong.

Three tricks:

  • Even though configtest doesn’t complain, ExtFilterDefine is only valid in server config context
  • DO NOT “SetOutputFilter DEFLATE”. It compresses all contents, ignoring nodeflate
  • Define two ext filters

Here is the WORKING config.

    LoadModule ext_filter_module libexec/apache22/mod_ext_filter.so
    LoadModule filter_module libexec/apache22/mod_filter.so
    # Netscape 4.x has some problems...
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    # IE is ok, but looked like Netscape, so we reset it
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

    # We need to force compression since we will remove the
    # request Content-Encoding/TE headers which mod_deflate
    # looks for.
    SetEnvIfNoCase Accept-Encoding gzip force-gzip
    SetEnvIfNoCase TE gzip force-gzip

    # Disable compression from backend
    RequestHeader unset Accept-Encoding
    RequestHeader unset TE

    # Make sure caching still works
    Header append Vary User-Agent env=!dont-vary

    # XXX be sure to define ExtFilterDefine in server config context!
    # not in virtual host, not directory, etc
    ExtFilterDefine nodeflate mode=output \
        cmd=/usr/bin/true \
    # XXX workaround a bug in mod_ext_filter.
    # it looks up extfilter by FilterProvider's name, not user defined ext filter.
    # it complains "couldn't find definition of filter 'compress'"
    # Define the same name of FilterProvider.
    ExtFilterDefine compress mode=output \
        cmd=/usr/bin/true \

    # Setup the filter to compress the response if we saw
    # an Accept-Encoding/TE header and marked it as force-gzip
    FilterDeclare compress CONTENT_SET
    FilterProvider compress DEFLATE env=force-gzip =1

    # Overwrite the "deflate" filter with the "nodeflate"
    # for various content types so they will not be compressed.
    # This is done as a hack because there is no way to
    # use more than one condition with FilterProvider above.
    FilterProvider compress nodeflate Content-Type $image/
    FilterProvider compress nodeflate Content-Type $application/
    FilterProvider compress nodeflate Content-Type $video/
    FilterProvider compress deflate Content-Type $application/javascript
    FilterProvider compress deflate Content-Type $application/json
    FilterProtocol compress "change=yes"

    FilterDeclare  modsec CONTENT_SET
    FilterProvider modsec modsecurity_out env=modsec-ignore !=1

    FilterChain modsec compress

I tested this on FreeBSD, so, use “cmd=/bin/true” on Linux and others (“/bin/sh” should work and is more portable, if you prefer).

When Apache 2.4 is released, it will be much simpler. See “Expression parser for Apache”.