by Brandon Rhodes • Home

Fixing OpenConnect’s VPN Search Domains on Ubuntu

Date: 3 August 2017

A quick technical note about VPN hostnames on Ubuntu Linux, since otherwise I will forget:

If other users of your VPN can refer to hosts by an unqualified hostname, but an Ubuntu user like you receives a not found error for the same hostname, then try creating the following file (you will need to create the directory by hand).

/etc/vpnc/connect.d/cisco_split_dns

# Take each "Cisco split domain" defined by the VPN and add
# it to the "search" line in resolv.conf, so unqualified
# hostnames are searched for in all the subdomains that
# the network architects intended.

if [ -n "$CISCO_SPLIT_DNS" ]; then
    for domain in $(echo "$CISCO_SPLIT_DNS" | sed "s/,/ /g")
    do
        CISCO_DEF_DOMAIN="$CISCO_DEF_DOMAIN
domain $domain"
    done
fi

After creating the above file, disconnect and reconnect to the VPN. All of the unqualified hostnames that your co-workers enjoy should now start working for you!

How did I develop this fix?

I suspected that the problem with hostname resolution involved how openconnect sets the search line in /etc/resolv.conf. For example, connecting to a VPN might configure /etc/resolv.conf with a line like:

search example.com

When a network program is given a hostname that does not include a period — an “unqualified” hostname that doesn’t specify its full domain — it uses this search line to suggest possible domains for the host. The command ssh foo, for example, would check whether foo.example.com exists. My guess was that the search line was actually supposed to be longer after connecting to the VPN, like:

search example.com corp.example.com dev.example.com

This would make ssh foo also check for foo.corp.example.com and foo.dev.example.com if the name foo.example.com turned out to not exist. But for some reason my search line was listing only a single subdomain.

I was daunted at the thought of trying to find a fix. How could I possibly affect something as intricate as how VPN software decides to configure my network, without iterating through the time-consuming process of patching the source and recompiling its binary?

It turns out, as I was delighted to discover, that openconnect defers the step of configuring the local host’s network to a plain-text shell script! This is one of the glories of the UNIX tradition: solving even system configuration problems, where possible, with standard and transparent tooling.

When openconnect finishes negotiating the secure channel, it calls a shell script named vpnc-script, which is briefly described on its manual page:

$ man openconnect

...

-s,--script=SCRIPT
       Invoke  SCRIPT  to  configure  the network after
       connection. Without this, routing and name  ser‐
       vice  are unlikely to work correctly. The script
       is  expected   to   be   compatible   with   the
       vpnc-script which is shipped with the "vpnc" VPN
       client.  See   http://www.infradead.org/opencon‐
       nect/vpnc-script.html for more information. This
       version of  OpenConnect  is  configured  to  use
       /usr/share/vpnc-scripts/vpnc-script by default.

I opened the default version of the script and was happy to see that it was profusely commented. Right at the top, it featured a long list of the environment variables that openconnect sets before calling it. Did there exist a setting on this particular VPN that the script was not equipped to use? I tried running openconnect with the verbose -v option that was mentioned on the manual page, and whole screenfulls of configuration data from the VPN poured across my terminal window. I perused them and quickly found:

...
X-CSTP-Default-Domain: example.com
...
X-CSTP-Split-DNS: corp.example.com
X-CSTP-Split-DNS: dev.example.com
...

These “Split DNS” entries were the ones missing from the search clause of my resolv.conf! But how were these values delivered to the shell script? I made a temporary edit atop the vpnc-script to save all of its environment variables to a file:

set > /tmp/my-environment

Reconnecting to the VPN and then reading through this temporary file, I saw that the environment variable CISCO_SPLIT_DNS was the one I was looking for: a comma-separated list of the missing subdomains. How could I arrange for them to be added to resolv.conf? One possibility would be to maintain my own version of vpnc-script — but, happily, I found that the script’s behavior could be manipulated from a separate file!

Using a few temporary echo here >> /tmp/log statements to make sure I understood which decisions the script was making, I learned that it assembled a small configuration file in a variable $NEW_RESOLVCONF that it then passed as input to a program I had never heard of named /sbin/resolvconf, that apparently is how modern Linuxes like Ubuntu automate updates to their resolv.conf file. Expecting $CISCO_DEF_DOMAIN to be a lone domain name, the shell script adds it to the input it is building for resolvconf:

domain $CISCO_DEF_DOMAIN

But there is no rule that $CISCO_DEF_DOMAIN has to contain only one line — if I supplemented it to include several additional lines of text, then they would also be included in the input to resolvconf! Is there any way I could get in and tweak the value of $CISCO_DEF_DOMAIN before the script uses it?

I kept reading and, happily, discovered that the script calls an internal routine run_hooks connect before performing an actual connection. The routine runs every file of shell commands that it finds in the directory /etc/vpnc/connect.d.

And so the problem was solved!

  1. I could add a file of my own to /etc/vpnc/connect.d to supply code to run before vpnc-script did the rest of its work.

  2. The environment variable $CISCO_DEF_DOMAIN would hold the hostnames I needed to add.

  3. The only impedance mismatch was that $CISCO_DEF_DOMAIN is comma separated, whereas the text I needed was several separate search … lines, but a quick for loop could easily generate one from the other.

And so I wrote the short shell script at the top of this post, disconnected from the VPN, reconnected, and for the first time was able to use the unqualified hostnames that had always worked for my coworkers on their Macs but had never worked for me.

And the fix was easy, thanks to the time-tested UNIX practice of plain-text shell scripts driving system configuration, making even the details of VPN network setup visible and extensible when they need to be.

©2021