Massoud Mazar

This problem took me a day to fix and it worth documenting here for others.

On my Mac laptop, I connect to a VPN, which has its own DNS server defined to resolve internal host names. For example, I could use 

nslookup host.foobar.net

and get a good response. So far so good. But when I tried 

nslookup host.sub.foobar.net

I did not get a response. After talking to IT team, it turned out this address was a CNAME record and not a A record. Still, I expected the DNS resolution to correctly return an IP address for this host, but it didn't.

First solution I found was to add the VPN DNS address as my primary DNS server on my primary network connection, but it meant all the DNS traffic from my laptop would flow through the VPN DNS server, and I thought that is not the right solution.

Then from this port I found out that if the CNAME resolves to a name in a different domain (e.g. amazonaws.com), it will not be resolved correctly, and that was exactly my situation. So the new solution was to add that external domain (amazonaws.com) to the list of domains resolved through the VPN DNS server. This can be done using the scutil tool on Mac.

First, you will need to find the name for your VPN connection:

sudo scutil
Password:
> list ".*DNS"
  subKey [0] = State:/Network/Global/DNS
  subKey [1] = State:/Network/MulticastDNS
  subKey [2] = State:/Network/PrivateDNS
  subKey [3] = State:/Network/Service/SOMETHING_SOMETHING_SOMETHING/DNS
  subKey [4] = State:/Network/Service/SOMETHING_SOMETHING_SOMETHING/DNS
  subKey [5] = State:/Network/Service/YourVPNConnecttion/DNS

Then you can see details of your VPN DNS settings:

> get State:/Network/Service/YourVPNConnecttion/DNS
> d.show
<dictionary> {
  ServerAddresses : <array> {
    0 : XXX.XX.X.X
  }
  SupplementalMatchDomains : <array> {
    0 : foobar.net
  }
  SupplementalMatchDomainsNoSearch : 1
}

Here is where you add your external domain (e.g. amazonaws.com):

> d.add SupplementalMatchDomains foobar.net amazonaws.com
> d.show
<dictionary> {
  ServerAddresses : <array> {
    0 : XXX.XX.X.X
  }
  SupplementalMatchDomains : <array> {
    0 : foobar.net
    1 : amazonaws.com
  }
  SupplementalMatchDomainsNoSearch : 1
}
> set State:/Network/Service/OpenVPNConnect/DNS
> exit

It worked all good until I disconnected and reconnected to VPN. My changes were gone!

It means the VPN server pushes down these settings, so if I need this to be permanent, I need to ask IT team to include amazonaws.com in the list of domains resolved by this VPN DNS server. After they made the change on the VPN server, it gets pushed down to all VPN clients and problem is solved!

Another gotcha in solving this problem was the nslookup trap on Mac. I thought it is safe to use this tool for DNS resolution troubleshooting, but it was not, as nslookup still returns:

server can't find host.sub.foobar.net: NXDOMAIN

But the Mac native tool resolves the address now:

% dscacheutil -q host -a name host.sub.foobar.net
name: something.something.something.something.amazonaws.com
alias: host.sub.foobar.net 
ip_address: XXX.XX.X.XXX