Routing a Public IP Over a VPN Connection

Each industry has its weird little standards. When I worked in the pharmacy space, that weird little standard was how you had to route certain public IP addresses over a private VPN. When I first came across this, I scratched my head as to why they did it this way. Then I had to figure out how to do this in a virtual cloud environment. Then I balled up my fists and screamed. Then I figured it out and went and got a beer. It was a tumultuous, exhilarating time!

The Problem

Since this was required to process pharmacy payments, let me first offer the briefest background on how retail pharmacies process claims. When it’s time to pick up your pills (none of my business what it’s for), the pharmacy tech sends your prescription and insurance information to your insurance company. The insurance company responds back immediately with how much they will reimburse the pharmacy and how much the patient owes. It’s actually a pretty well-oiled machine.

But just like every other part of the medical industry, there is a middleman in this process. Each pharmacy doesn’t make a direct connection to each of the roughly 66 pharmacy insurers in the US. Instead, a pharmacy makes a connection to something known as a clearinghouse that relays the claim to the correct insurance company. The clearinghouse is like a hub in the middle of a wagon wheel. It connects to all the pharmacies and all the insurance companies.

Who would have guessed that any part of the American healthcare system is actually efficient?
house by barretr (Open Clip Art Library), licensed under CC0
office building by Vincent Le Moign, licensed under CC BY 4.0

My experience with this is only in the pharmacy space, but my understanding is that this model is the same for medical claims processing.

For stupid historical reasons, the pharmacy coverage part of your medical insurance is technically known as a pharmacy benefit manager or PBM. If you find yourself at a dinner party sitting next to someone in the retail pharmacy space, use the term "PBM" and watch their face light up and buckle in for a really boring conversation.

Here’s a rapid fire set of concepts to know for the rest of this article.

• The insurance company is known as the payer. Payers can be any entity that processes a claims (e.g., pharmacy discount cards).
• When the payer processes claims, it’s known as adjudication.
• The adjudication process has been around since before web services were popular, so this connection is made over TCP sockets (which is why we need the VPN connection in the first place). Some clearinghouses are developing REST services, so hopefully this whole networking model will be a thing of the past in a few years.

Public IP Addresses to the Rescue

On to the technical problem. Each payer that wants to communicate with the clearinghouse will need some type of network connection. This often will be a site-to-site VPN connection. But when you traditionally think about a corporate site-to-site VPN, you’re usually connecting two or more separate private networks that are controlled by the organization. This means you can ensure that each site has a separate private address space that doesn’t overlap. For instance, your small satellite office in San Francisco, CA may have the 192.168.0.0/24 address space whereas your gigantic headquarters in Boerne, TX has the 10.0.0.0/12 address space.

If you’re not a network person, there’s two types of IPv4 addresses: public and private. Private addresses can be assigned however you want in your private network but cannot be used on the public internet. Since private addresses can be assigned however you want, there’s no guarantee that the private address space that the clearinghouse has available will be available on your network. In a perfect world, the clearinghouse would have the same unused private subnet as your organization does. Also, in a perfect world, there would be no disease and we wouldn’t need pharmaceutical companies at all, but that’s not our reality.

Since public IP addresses are guaranteed to be unique, the clearinghouses have chosen to use that as their solution. So you assign a public IP address to your adjudication endpoint where the clearinghouse will establish a TCP connection. The whole thing looks something like this:

Public IP over VPN high-level network design
house by barretr (Open Clip Art Library), licensed under CC0
firewall by Uri Herrera and others, KDE Visual Design Group, licensed under LGPL

Using public IPs ensures that there are no IP address collisions like you potentially would have if you were only connecting private networks. There is nothing in the IPSec VPN specifications that say you can’t use public IP addresses, but it’s a departure from how you regularly do things so it takes some noodling to get it right.

If you’ve dealt with IP conflicts in the past, you may be thinking that VPN SNAT would be a solution that the clearinghouses could have used. I don’t know why they don’t offer that, but my guess is that their customers may find that managing SNAT introduces more complexities for both parties than the public IP solution.

Challenges in Azure

This can be done with native Azure services, but there are some challenges.

If you’re like me, the first thought you had was that this requires some NAT magic, like this:

This seems like the NATural solution
house by barretr (Open Clip Art Library), licensed under CC0
firewall by Uri Herrera and others, KDE Visual Design Group, licensed under LGPL

The problem is that Azure VPN Gateway does not support DNAT. As you recall, the clearinghouse will be sending packets over the VPN to a public IP address. We need to translate that destination public IP address to the private IP address associated with our application, but the VPN Gateway lacks any DNAT feature. If you Google it you may discover that all but the cheapest SKUs support NAT. But this is only source NAT. Even if the source is translated, the destination will still go to our public IP address.

But hmmm, you think. Maybe you could game the system and route to that public IP through Azure’s software-defined networking, you think. Since the packets to our application’s public IP can route over the VPN we could assign the public IP to a VM and let it just route that last little bit over the public internet, is your thought. After all, since this will be going from one Azure resource (VPN Gateway) to another Azure resource (VM), it will probably stay within the edge of the Microsoft global network (cold potato routing), such is your thinking.

Shame on you for such sinful thoughts! That network is still untrusted! But even if your CSO approved such a hairbrained scheme, the Azure VPN Gateway does not allow internet access, so you can give up on the idea.

The Easy Solution

Holy shit, there’s actually a super easy way to do this! Did you know that you can assign public IP addresses to your virtual network address space?! (If you did know that, shut up.)

The solutions looks something like this:

Public IP over VPN using VNet with public subnet
house by barretr (Open Clip Art Library), licensed under CC0

And can be done like this:

• Create an Azure VPN Gateway connection to your clearinghouse (easier said than done, I know).
• Create your own public IP prefix or bring your own. Either way, the prefix must be a /29 or larger.
• Create an address space on your virtual network using the public IP prefix from above and then create a new subnet using the same prefix, which we’ll call adjudication.
• Assign the adjudication subnet to the NIC for your adjudication application’s VM or load balancer and optionally assign a static IP to the NIC.
• Create a route table with a route to the clearinghouse public subnet specifying the Virtual network gateway as the next hop and assign it to the adjudication subnet.

The route for your route table associated with the adjudication subnet has these properties.

Destination type	IP Addresses
Destination IP addresses	198.51.100.0/25
Next hop type	Virtual network gateway
Next hop address	n/a

Easy, peasy, lemon squeezey, as we say in the cloud architecture world. Your virtual network will know to route traffic to your public IP prefix to your internal Azure resources and will know to route traffic to your clearinghouse back through the VPN Gateway. Even though these IPs are public, they won’t be available from the internet by default.

The only sorta, kinda downside is that if you only need a single IP address, you’ll still need to provision a /29, which equates to 8 IP addresses that you’ll be paying for. And actually, since you’re creating a subnet from the address space, you lose 3 IP addresses to networking infrastructure. That leaves 4 IP addresses left over which you probably don’t need.

Network Virtual Appliance

The way I’ve done this in the past, before I knew about the easy way, was with a network virtual appliance (NVA). An NVA is virtual router from a non-Microsoft vendor of your choosing (Cisco, Palo Alto, Fortinet, etc.) that you can purchase from the Azure marketplace. This is like plopping a physical router that you’re already familiar with in Azure (but it’s like, virtual, man) and grants you complete control over your networking infrastructure, including the VPN connection, routing, and NAT. If you already have experience using, say, a Cisco device, you can apply all of your existing knowledge to the NVA without having to learn all this Azure networking horseshit.

This can be done any number of ways depending on your network topology, but here’s one example:

Public IP over VPN NVA network design
house by barretr (Open Clip Art Library), licensed under CC0
physical router image by George Shuklin, licensed under Creative Commons Attribution-Share Alike 3.0 Unported

Since this can be any type of device, I’m not going to show you how to do it, but you’ll want to do the following in general on the NVA.

1. Create your NVA resource in Azure and with at least two NICs.
   • Assign a private static IP (10.0.16.8) to the first NIC on the routing subnet.
   • Assign a static public IP address (o.o.p.p) to the first NIC that will be used for the VPN peer. This is not the same as the public IP address for your application endpoint. (This public IP may also be used as your management access point, depending on the vendor.)
   • Assign a private static IP (10.0.20.8) to the second NIC on the adjudication subnet.
2. Establish a VPN connection to the clearinghouse’s IPSec peer IP (c.c.p.p).
3. Create a NAT rule to translate your application’s public IP address to its private address (203.0.113.83 10.0.20.32).
4. If necessary, create a NAT rule to translate your application’s private IP address to its public address (10.0.20.32 203.0.113.83). For some routers, this may have been done as part the previous step.
5. If necessary, create a route to the clearinghouse’s public IP over the VPN (198.51.100.0/25 VPN). For some routers, this route is inferred from the VPN.

You will also need to create an Azure Route Table with a route to the NVA and associate it with the adjudication subnet and any other subnet that needs to access the clearinghouse network. Without this, our adjudication app will assume that traffic to 198.51.100.0/25 should go through the internet. In our example, the route table would contain a route with these properties.

Destination type	IP Addresses
Destination IP addresses	198.51.100.0/25
Next hop type	Virtual appliance
Next hop address	10.0.20.8

Other Considerations

I didn’t talk about firewalls, but always make sure you’re restricting traffic through NSGs, Azure Firewall, or your NVA. Whenever I’m troubleshooting my network configuration I always remember two things: routing and firewalls. It almost comes down to one or both of those things at each hop.

Another consideration obviously is cost. The easy option is the lightest on the wallet. In 2024 dollars, the least expensive VPN Gateway costs $138.70 per month plus $10.80 for the site-to-site tunnel. The public IP prefix will cost you $4.32 per month. That comes to $153.82 per month at the cheapest before your traffic ingress/egress charges. Not too shabby.

As an example for the NVA, the lowest tier option for a Fortinet FortiGate firewall is $262.80 per month before the costs of the underlying VM. For that, they recommend an F4 VM which comes to $889.87. That totals $1,152.67 per month. That’s not terrible but your CFO may not understand why you need this purchase.