Redundant ISP’s – The Inbound Failover Problem Solved with DNS Made Easy

Adding second ISP to your firewall allows users in the office to access the Internet in case the primary ISP is unavailable. However, any inbound services that depend on public DNS records (typically ActiveSync, OWA, Citrix, VPN, and publicly accessible web servers) will be unavailable to remote users and clients until DNS records have been pointed to the new IP address. Kraft Kennedy has been testing DNS Made Easy, a product designed to address this quandary.

Traditionally this issue has been solved by setting up BGP (Border Gateway Protocol) routing between the ISP’s. This works because the public IP address never changes, just the path to get to the internal systems either through the primary or backup ISP. This requires the cooperation of both ISPs to support BGP (typically won’t work with business DSL or cable connections), a router/switch that supports BGP that connects to both ISPs, the purchase of an autonomous system number (ASN) from ARIN, and an expert to help configure the switch and coordinate the process with both ISP’s and configure the switch/router. Not a simple deployment, to say the least.

Less traditional were the load balancing appliances that could aggregate multiple ISP connections, providing increased bandwidth and failover. Appliances from FatPipe and F5 fit into this category. Inbound failover is handled by DNS manipulation on the public DNS server running on the appliance. The advantage of this is that it can work with any ISP and doesn’t require cooperation from the ISP to implement. The downsides of this solution can be costliness and additional network complexity in deploying the appliance.

Recently Kraft & Kennedy has implemented a third approach for inbound failover, using a hosted solution from DNS Made Easy. They host our public DNS zones on their server and manipulate DNS in case we have an ISP failure with our inbound services.

The basic process of migration to DNS Made Easy is as follows:

1. Migrate the public DNS zones from your existing public DNS provider to DNS Made Easy.

2. Identify particular DNS records to be setup for failover (e.g. Outlook Web Access).

3. Set the TTL (Time to Live) on the DNS record to 180 seconds (3 minutes).  This is how long a DNS server should cache the record before requesting an update from DNS Made Easy.

4. Enable firewall rules so that your inbound service has the required ports open on both the primary and backup ISP.

5. Setup the Monitoring and Failover rule.

DNS Made Easy Failover Record

6. Monitoring Notifications: Checked

7. Notification Contact: Account Owner

8. Number of Emails: 3

9. Sensitivity: Low / Medium (Default) / High

10. Protocol: HTTP / HTTPS / TCP / UDP / DNS.  Pick the appropriate protocol depending on the service/server you are monitoring.  For OWA we are just monitoring on HTTPS protocol.

11. FQDN: The fully qualified domain name of the server being monitored (e.g. webmail.kraftkennedy.com).

12. File to Query (particular to HTTP/HTTPS monitoring): If you are looking for a particular file on the web server.

13. String to Query For (particular to HTTP/HTTPS monitoring): If you are looking for particular contents in the file received from the web server.

14. DNS Failover: Checked.  This actually does the failover if the monitoring fails.

15. Turn off auto-failover after first failure: Typically unchecked.  We want the connection to fail back to the primary ISP when it is available.

16. Location 1 -5:  This is the IP address that you’d like to use for your primary ISP (Location 1) and then any failover IP addresses should follow after this. The locations do not need to be in the same office so this can also be used to fail over between data centers.

So if we had an issue with our primary ISP, DNS Made Easy will detect this in a matter of a few minutes, realize that it is unavailable, and then switch over the DNS record to the secondary ISP. The 180 second TTL on the DNS record means that clients should receive the change in approximately 3 minutes depending on the DNS provider used. While the failover isn’t instantaneous, it will typically happen faster than manually changing DNS records and IT staff can receive an automated email alert about the failover and failback.

The typical costs for most clients of the DNS Made Easy Business account are $59.99 per year for up to 25 domains and 3 failover records. Additional failover records can be purchased for $4.95/record/year (slightly cheaper, at $45.95, for a 10 pack of failover records).

Note: This article also posted to my work blog here.