IPv6 and OpenBSD

I decided to try running with IPv6 only, but my job involves IPv4-only hosts. So I thought I’d try OpenBSD with NAT64 and DNS64. Writing this ended up taking far too much time as I faffed over whether to talk about DS-Lite, whether to focus on the OpenBSD aspects, whether to bother at all. So it’s all quite ramshackle. Maybe read this more like a set of field notes.

Clients

Before we can talk about routing, we need to talk about clients. Client behaviour in IPv6-only networks is not quite as uniform as with IPv4-only networks - solutions range from, well, doing nothing (single-stack) and letting things fail, to various forms of encapsulation and translation.

In terms of raw numbers, I mostly use BSDs and forms of MacOS (MacOS, iOS, tvOS, etc). But I do have a large GNU+Linux workstation. I also have an android phone I use for aNag. This is a reasonably decent representation of client behaviour in v6-only networks.

OpenBSD

So with that, I figured I’d run OpenBSD. It provides pf, which implements a necessary af-to function for converting between address families. I’ve used OpenBSD before, maybe twice. Once on a Sun UltraSPARC T1000, and again on a Raspberry Pi as a router. I think thought I recalled how to set things up, but I ended up being quite wrong and broke my home network for an hour or two.

In any case, after the initial setup, there were only a few things to configure. DHCP, DNS, RA, PPPoE and the firewall. No chemical-X. The configuration I’d wrote off-the-cuff didn’t work, I’d forgotten about unbound’s default behaviour when access control is left out (localhost only). I’d also forgotten to do MSS scrubbing in PF, which broke connectivity to a few odd sites.

With those things sorted, I was roughly where I started, but on OpenBSD instead of DragonflyBSD. Now to make the network IPv6-only. Ish.

DNS64 with Unbound

The first component to tackle was DNS. Some hosts only have A records. Some hosts have both. In an IPv6-only network, A records are rather useless to clients - they’re smart enough to know they can’t reach the address. DNS64 exists to mangle A records into AAAA records.

Take the example of ipv4.google.com, a host that only provides an A record. Currently that A record is 142.250.178.14. The purpose of DNS64 is to take that address and encapsulate it within an IPv6 address with a know prefix. What that prefix may be is left to the sysadmin, but 64:ff9b::/96 is the “well-known” prefix specified in the RFCs. Specifying your own is fine, provided everything else in the stack makes use of it.

Converted, 142.25.178.14 is equivalent to 8efa:b20e. With the prefix added, we have 64:ff9b::8efa:b20e. As far as any v6-only client should be concerned, that’s a routable, reachable IPv6 address.

NAT64 with pf

The problem we now face is that for all we’ve converted it, and the client is using it, we’ve really only changed how we represent the IPv4 address. Ultimately we need to speak IPv4 at some point. Well-known or not, 64:ff9b::/96 is for local use, and it generally isn’t meaningful outside your own network.

So, we need to convert our packet from a real IPv6 address to our synthesised IPv6 address into a packet from a real IPv4 address to a real IPv4 address. Because word-salad isn’t filling, here’s a diagram:

+---------+  IPv6  +----------+  IPv4  +-----------+  IPv4  +--------+
|public6  +------->|public6   |------->|public6    |------->|no ipv6 |
|no ipv4  |        |public4   |        |public4    |        |public4 |
|client   |<-------|my router |<-------|isp router |<-------|server  |
+---------+  IPv6  +----------+  IPv4  +-----------+  IPv4  +--------+

Simple, right? I’ve skipped out including the addresses, because I’m just talking about general process here. Just know there’s a thing in the middle between the client and the server translating IPv6 into IPv4, and clients think everything is available over IPv6.

In pf, this is done on one line. The only thing that caught me out was the inet from $public4. I’d tried using egress:0, but for whatever reason, nothing worked. My address is static, so specifying the IPv4 address to NAT to isn’t a big deal:

pass in quick on $lan_if inet6 from $public6 to $nat64 af-to inet from $public4 keep state

IPv4 Literals

All that said - there’s a problem here. While ipv4.google.com involves a DNS lookup to resolve it to an IP address, there’s a lot of cases where we don’t use DNS at all. VoIP is one of them - it’s perfectly sensible to throw around bare IP addresses in VoIP. Anything peer-to-peer is another critical example.

If we do nothing, IPv6-only devices will see they need to communicate with an IPv4 address, and fail. Probably with some form of unreachable host error. So we have two options:

  1. DS-Lite
  2. 464XLAT

DS-Lite

We outsource NAT64. Our clients on the local network are given private IPv4 addresses and public IPv6 addresses. They speak to our router as they would on a normal dual-stack network. Our router, which doesn’t have a public IPv4 address in this scenario, wraps the client’s IPv4 packets in IPv6 (like above), and fires them at our ISP who unpacks them and performs NAT64.

This is kind of popular because it allows ISPs to avoid allocating public addresses to every customer. But in return, it means customers can’t really do things like port-forwarding, as their public IP can change per-connection. Operating systems don’t need to do anything new.

Here’s the previous diagram adapted to show DS-Lite. Note that my router now has a private IPv4 address, rather than public:

+---------+  IPv4  +----------+  IPv6  +-----------+ IPv4 +--------+
|public6  |------->|public6   |------->|public6    |----->|public4 |
|private4 |        |private4  |        |public4    |      |        |
|client   |<-------|my router |<-------|isp router |<-----|server  |
+---------+  IPv4  +----------+  IPv6  +-----------+ IPv4 +--------+

I can’t do DS-Lite. My ISP would need to support it, and they currently have no need for it. I wouldn’t really want to do it either - I like being in control of this stuff. So we go for option 2.

464XLAT

We have the client give itself an IPv4 address and perform the IPv4-address-in-IPv6-address translation. Naturally this requires the operating system support it. On hand at the moment I have MacOS and iOS, which both do. The particular criteria that triggers MacOS/iOS to enable this behaviour is as follows:

  1. Our DHCPv4 server must make offers with Option 108 set. Option 108 means “we’d prefer you didn’t use IPv4”.
  2. Our router adverts must specify the NAT64/DNS64 prefix.
  3. Our router adverts must be perfect. MacOS will fail in strange ways if not. Examples of ‘not perfect’ include mismatching subnet size with interface (assigning ::1/48 to the interface sending adverts, but specifying /64 in rad.conf).

The first is configured in dhcpd.conf:

option ipv6-only-preferred 900;

The second in rad.conf:

nat64 prefix 64:ff9b::/96

The third is really just paying attention when setting things up. I’ve been allocated a /48 by my ISP, but the subnet used on my home network is in fact /64. Mix stuff up like that, and MacOS will throw vague messages in the console like “Failed to allocate address”, “failed to activate CLAT”.

Presuming you meet the criteria, you’ll end up with this:

en0: flags=88e3 mtu 1500
	options=6460
	ether 7b:f7:59:a6:b3:72
	inet6 fe80::cca:2bf0:3a18:d4ee%en0 prefixlen 64 secured scopeid 0xb
	inet6 2001:8b0:ca70:32b4:12f8:9397:3548:5c5 prefixlen 64 autoconf secured
	inet6 2001:8b0:ca70:32b4:86dd:c35b:3053:6a4 prefixlen 64 autoconf temporary
	inet 192.0.0.2 netmask 0xffffffff broadcast 192.0.0.2
	inet6 2001:8b0:ca70:32b4:1720:2bd6:32c:1fe prefixlen 64 clat46
	nat64 prefix 64:ff9b:: prefixlen 96
	nd6 options=201
	media: autoselect
        status: active

And will be able to do this:

patrick@nilgiri ~ % ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=88.738 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=90.751 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=91.646 ms

Our first diagram, taken and expanded to show the three distinct scenarios we can (technically four, but IPv6 works the same way whether or not DNS is involved), look like this:

IPv4 addresses (IPv4 "literals") where DNS is not involved (i.e. client to 1.1.1.1)

+---------+  IPv4  +-----------+  IPv6  +---------+  IPv4  +-----------+ IPv4
|public6  +------->|public6    +------->|public6  |------->|public6    |------> 1.1.1.1
|192.0.0.2|        |192.0.0.1  |        |public4  |        |public4    |<------
|client   |<-------+also client|<-------|my router|<-------|isp router |
+---------+  IPv4  +-----------+  IPv6  +---------+  IPv4  +-----------+
        (Internal CLAT)          (NAT64)

IPv4 addresses where DNS64 translates A to AAAA (i.e. client to ipv4.google.com):

+---------+  IPv6  +--------+  IPv4  +-----------+ IPv4
|public6  +------->|public6 |------->|public6    |------> ipv4.google.com
|         |        |public4 |        |public4    |<------
|client   |<-------|router  |<-------|isp router |
+---------+  IPv6  +--------+  IPv4  +-----------+
            (4in6)

IPv6 "literals" or AAAA records (this is just standard routing):

+---------+  IPv6  +--------+  IPv6  +-----------+ IPv6
|public6  +------->|public6 |------->|public6    |-----> ipv6.google.com
|         |        |public4 |        |public4    |<-----
|client   |<-------|router  |<-------|isp router |
+---------+  IPv6  +--------+  IPv6  +-----------+

Non-Fruit Operating Systems

I apologise - I lied to you. My home network is not IPv6-only. You probably noticed that MacOS wont even try to do 464XLAT/CLAT if you’re not running a DHCP(v4) service. So yeah, my network is actually dual-stack, but the Apple stuff is choosing to avoid taking v4 addresses.

This leads to the question of how other operating systems behave. Sadly, not that well. GNU/Linux doesn’t do CLAT by default, though there’s an issue open to try and get systemd doing it. Android/Linux does, in theory. But in practice the crappy HONOR tablet I have seems to respect Option 108, but can’t set up CLAT, and prevents me from seeing any details in the interface. God it’s shite.

BSD’s don’t seem to fare much better. There are working solutions, but nothing out of the box.

Windows is Windows. It might work, might not. Microsoft say they’re supporting it on non-WWAN interfaces at some unspecified point in the future. I don’t have any windows devices on my network to actually investigate.

In every case outside of the crappy Android tablet, everything still works with graceful fallback to dual-stack operation. I’m sure I’ll come back to this at some point from a Linux perspective.