Work

Redundant ISP’s – The Inbound Failover Problem Solved with DNS Made Easy

Leave a comment

Adding second ISP to your firewall allows users in the office to access the Internet in case the primary ISP is unavailable. However, any inbound services that depend on public DNS records (typically ActiveSync, OWA, Citrix, VPN, and publicly accessible web servers) will be unavailable to remote users and clients until DNS records have been pointed to the new IP address. Kraft Kennedy has been testing DNS Made Easy, a product designed to address this quandary.

Traditionally this issue has been solved by setting up BGP (Border Gateway Protocol) routing between the ISP’s. This works because the public IP address never changes, just the path to get to the internal systems either through the primary or backup ISP. This requires the cooperation of both ISPs to support BGP (typically won’t work with business DSL or cable connections), a router/switch that supports BGP that connects to both ISPs, the purchase of an autonomous system number (ASN) from ARIN, and an expert to help configure the switch and coordinate the process with both ISP’s and configure the switch/router. Not a simple deployment, to say the least.

Less traditional were the load balancing appliances that could aggregate multiple ISP connections, providing increased bandwidth and failover. Appliances from FatPipe and F5 fit into this category. Inbound failover is handled by DNS manipulation on the public DNS server running on the appliance. The advantage of this is that it can work with any ISP and doesn’t require cooperation from the ISP to implement. The downsides of this solution can be costliness and additional network complexity in deploying the appliance.

Recently Kraft & Kennedy has implemented a third approach for inbound failover, using a hosted solution from DNS Made Easy. They host our public DNS zones on their server and manipulate DNS in case we have an ISP failure with our inbound services.

The basic process of migration to DNS Made Easy is as follows:

1. Migrate the public DNS zones from your existing public DNS provider to DNS Made Easy.

2. Identify particular DNS records to be setup for failover (e.g. Outlook Web Access).

3. Set the TTL (Time to Live) on the DNS record to 180 seconds (3 minutes).  This is how long a DNS server should cache the record before requesting an update from DNS Made Easy.

4. Enable firewall rules so that your inbound service has the required ports open on both the primary and backup ISP.

5. Setup the Monitoring and Failover rule.

DNS Made Easy Failover Record

6. Monitoring Notifications: Checked

7. Notification Contact: Account Owner

8. Number of Emails: 3

9. Sensitivity: Low / Medium (Default) / High

10. Protocol: HTTP / HTTPS / TCP / UDP / DNS.  Pick the appropriate protocol depending on the service/server you are monitoring.  For OWA we are just monitoring on HTTPS protocol.

11. FQDN: The fully qualified domain name of the server being monitored (e.g. webmail.kraftkennedy.com).

12. File to Query (particular to HTTP/HTTPS monitoring): If you are looking for a particular file on the web server.

13. String to Query For (particular to HTTP/HTTPS monitoring): If you are looking for particular contents in the file received from the web server.

14. DNS Failover: Checked.  This actually does the failover if the monitoring fails.

15. Turn off auto-failover after first failure: Typically unchecked.  We want the connection to fail back to the primary ISP when it is available.

16. Location 1 -5:  This is the IP address that you’d like to use for your primary ISP (Location 1) and then any failover IP addresses should follow after this. The locations do not need to be in the same office so this can also be used to fail over between data centers.

So if we had an issue with our primary ISP, DNS Made Easy will detect this in a matter of a few minutes, realize that it is unavailable, and then switch over the DNS record to the secondary ISP. The 180 second TTL on the DNS record means that clients should receive the change in approximately 3 minutes depending on the DNS provider used. While the failover isn’t instantaneous, it will typically happen faster than manually changing DNS records and IT staff can receive an automated email alert about the failover and failback.

The typical costs for most clients of the DNS Made Easy Business account are $59.99 per year for up to 25 domains and 3 failover records. Additional failover records can be purchased for $4.95/record/year (slightly cheaper, at $45.95, for a 10 pack of failover records).

Note: This article also posted to my work blog here.

 

Work

Problems registering for Apple Developer Account and D-U-N-S Name

Leave a comment

My company is registering for an Apple Developer account to be able to deploy MDM (Mobile Device Management) solution to IOS devices.  As part of the registration process you have to get your D-U-N-S Number from D&B register the company.  However, even though the rest of the information was correct, it kept failing on the company name not matching and we couldn’t submit the application.

The company name is “Kraft & Kennedy, Inc.” and that is the name that we have registered with Dun & Bradstreet and shows up in the D-U-N-S verification form on Apple’s web site.  However, you can’t put ampersands or commas into the Apple form for the company name.   After trying multiple variants of the company name the only thing that worked was substituting the ampersand with the word “and”, so we were able to get registered with “Kraft and Kennedy Inc.” on the Apple form.

Work

Troubleshooting SCCM Client Installation Error 0x80041002

5 Comments

In trying to install the SCCM 2012 client on a Windows 7 workstation  it terminated with error 0x80041002.  Here’s the log snippet from CCMSETUP.LOG:

[LOG[Failed to open to WMI namespace ‘\\.\root\cimv2’ (80041002)]LOG]!<time=”05:24:55.747+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”3″ thread=”1300″ file=”wminamespace.cpp:305″
[LOG[CcmGetOSVersion failed with 0x80041002]LOG]!<time=”05:24:55.747+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”2″ thread=”1300″ file=”util.cpp:1474″
[LOG[Failed to open to WMI namespace ‘\\.\root\ccm’ (80041002)]LOG]!<time=”05:24:55.750+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”3″ thread=”2156″ file=”wminamespace.cpp:305″
[LOG[Failed to get client version for sending messages to FSP. Error 0x80041002]LOG]!<time=”05:24:55.750+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”2″ thread=”2156″ file=”ccmsetup.cpp:9838″
[LOG[Params to send FSP message ‘5.0.7958.1000 Deployment Error 0x80041002 : ‘]LOG]!<time=”05:24:55.750+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”0″ thread=”2156″ file=”ccmsetup.cpp:9887″
[LOG[Failed to open to WMI namespace ‘\\.\root\ccm’ (80041002)]LOG]!<time=”05:24:55.753+240″ date=”05-15-2015″ component=”FSPStateMessage” context=”” type=”3″ thread=”2156″ file=”wminamespace.cpp:305″
[LOG[State message with TopicType 800 and TopicId {A1967584-10FA-45E0-8790-1559DEF627B5} has been sent to the FSP]LOG]!<time=”05:24:55.760+240″ date=”05-15-2015″ component=”FSPStateMessage” context=”” type=”1″ thread=”2156″ file=”fsputillib.cpp:752″
[LOG[CcmSetup failed with error code 0x80041002]LOG]!<time=”05:24:56.180+240″ date=”05-15-2015″ component=”ccmsetup” context=”” type=”1″ thread=”2156″ file=”ccmsetup.cpp:10879″

It turned out that WMI on this workstation was completely unusable.  When using the WBEMTEST application you couldn’t even connect to the WMI at all.

With the help of the WMI: Rebuilding the WMI Repository article I was able to rebuild WMI on this workstation using the following commands:

Winmgmt /verifyrepository

Winmgmt /salvagerepository

Once WBEMTEST connected successfully the SCCM Client correctly installed.

Work

Cisco ASA Firewall Presents Only “ASA Temporary Self Signed Certificate”

8 Comments

Recently we started to get reports of untrusted certificates for AnyConnect and when accessing the ASDM web page. When you browse to the web site it was presenting the default “ASA Temporary Self Signed Certificate” rather than our public SSL certificate.

After opening a case with Cisco TAC about this they pointed us to the release notes issue in ASA 9.4(x):

Elliptic curve cryptography for SSL/TLS—When an elliptic curve-capable SSL VPN client connects to the ASA, the elliptic curve cipher suite will be negotiated, and the ASA will present the SSL VPN client with an elliptic curve certificate, even when the corresponding interface has been configured with an RSA-based trustpoint. To avoid having the ASA present a self-signed SSL certificate, the administrator needs to remove the corresponding cipher suites using the ssl cipher command. For example, for an interface configured with an RSA trustpoint, the administrator can execute the following command so that only RSA based ciphers are negotiated:

ssl cipher tlsv1.2 custom
“AES256-SHA:AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DES-CBC3-SHA:DES-CBC-SHA:RC4-SHA:RC4-MD5”

As soon as I added the command into our ASA it started working again. It sounds like this should be a default entry going forward for all ASA firewalls.

Want to learn more about elliptic curve cryptography  or look at this for a primer.

Work

Microsoft Intune and Conditional Access to Exchange On-Premise Configuration Problems

Leave a comment

I’m trying to setup the Microsoft Intune MDM solution with the Conditional Access policies to our Exchange On-Premise server.  The idea behind this is that users must enroll their device with Intune via the Company Portal app on their mobile device and then once they meet the requirements, they will be granted access to Exchange ActiveSync.

The basic setup is straightforward and easy to setup following this Technet article.  However we have run into two issues:

I received emails previously (last week) from the Intune Exchange Connector that looked like the following:

To access your organization’s email, you must enroll your device and make sure that it complies with your organization’s security policies. Follow the steps below for the relevant device.

Instructions for your Android with ID androidxxxxxxxxxxxxx
1. If you haven’t enrolled your device yet, enroll it now
2. After a couple of minutes activate your email
3. Check to see if your device is compliant

But after upgrading the latest version of the Microsoft Intune On-Premises Connect for On-Premises Exchange, I’m no longer receiving these emails but other users are.

The other issue is that the “activate your email” link doesn’t work.  The URL points to the following web site: https://enterpriseregistration.windows.net/manage/, but also includes options to POST the following:

Kraft&Kennedy,Inc.?.onmicrosoft.comeasactivation?easid=”androidxxxxxxxxxxxx&amp;traceid=xxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxx&quot;”

And the response to this URL and POST commands just looks like the following:

{“Message”:”No API matching request was found, verify URL and parameters are correct”,”TraceId”:”90baac77-3ac9-43e6-bb67-3b47a6c3726f”,”Time”:”04-28-2015 11:37:57Z”}

And the TraceId above has nothing to do with the obfuscated TraceId above.

I have opened a case with Microsoft Online Support about this and will update once I get more information.

Work

Cisco ASA Firewall ASDM Incompatibility with Java 7 Update 51

20 Comments

The latest version of Java 7 Update 51 that was deployed this week breaks access to Cisco ASA firewalls running ASDM.  When you connect with the ASDM you get the following error message: “Unable to launch device manager from X.X.X.X”

Unable to Launch Device Manager
“Unable to launch device manager from”

The symptoms are that the web page for the firewall will show up and display normally, but you can’t connect to the server with the ASDM launcher.  The log on the firewall shows

%ASA-6-302013: Built inbound TCP connection 112 for outside:X.X.X.X/64508 (X.X.X.X/64508) to identity:Y.Y.Y.Y/443 (Y.Y.Y.Y/443)
%ASA-6-725001: Starting SSL handshake with client outside:X.X.X.X/64508 for TLSv1 session.
%ASA-7-725010: Device supports the following 6 cipher(s).
%ASA-7-725011: Cipher[1] : RC4-SHA
%ASA-7-725011: Cipher[2] : DHE-RSA-AES128-SHA
%ASA-7-725011: Cipher[3] : DHE-RSA-AES256-SHA
%ASA-7-725011: Cipher[4] : AES128-SHA
%ASA-7-725011: Cipher[5] : AES256-SHA
%ASA-7-725011: Cipher[6] : DES-CBC3-SHA
%ASA-7-725008: SSL client outside:X.X.X.X/64508 proposes the following 8 cipher(s).
%ASA-7-725011: Cipher[1] : AES128-SHA
%ASA-7-725011: Cipher[2] : DHE-RSA-AES128-SHA
%ASA-7-725011: Cipher[3] : DHE-DSS-AES128-SHA
%ASA-7-725011: Cipher[4] : RC4-SHA
%ASA-7-725011: Cipher[5] : DES-CBC3-SHA
%ASA-7-725011: Cipher[6] : EDH-RSA-DES-CBC3-SHA
%ASA-7-725011: Cipher[7] : EDH-DSS-DES-CBC3-SHA
%ASA-7-725011: Cipher[8] : RC4-MD5
%ASA-7-725012: Device chooses cipher : RC4-SHA for the SSL session with client outside:X.X.X.X/64508
%ASA-7-725014: SSL lib error. Function: SSL3_READ_BYTES Reason: sslv3 alert certificate unknown
%ASA-6-725006: Device failed SSL handshake with client outside:X.X.X.X/64508
%ASA-6-302014: Teardown TCP connection 112 for outside:X.X.X.X/64508 to identity:Y.Y.Y.Y/443 duration 0:00:00 bytes 580 TCP Reset by appliance

Cisco has included this information in their latest release notes:

If you use Java 7 Update 51, you must upgrade ASDM to Version 7.1(5.100) or later, and you can only use the Java web start. The ASDM Launcher is not supported.

So the alternatives are to downgrade your Java on your workstation or upgrade to the latest ASDM version at this point to get the ASDM working again.

Fun

Pork Scratchings

Leave a comment

 

Pork ScratchingsWhile the United States celebrates Thanksgiving today (I do miss my brother’s Buffalo Fried Turkey) I picked up a couple bags of Pork Scratchings from the Saint Nicholas Fayre going on in York.  What are pork scratchings?  They are the UK equivalent of pork rinds in the US.  But while most of the pork rind products I’ve had in the US have been the equivalent of pork flavored puffs, the pork rinds from this particular vendor are MUCH better.

First, they are sold in paper bags.  The scratchings quickly soak through the bag turning it slightly translucent.   All fatty products should be sold this way, as putting them into plastic or mylar defeats the purpose.  Next a bite.  Hard and crunchy on the outside (some of the big ones can be almost teeth damaging), but rich and fatty on the inside.  The salty flavor isn’t over powering and almost has a slight season salt taste as well, just makes this pork scratching perfection.

Thankfully, for my health at least, the vendor is only in York on special market days, but it is the new tradition to pick up a bag (or three) when they are in town.

So while we aren’t celebrating Thanksgiving today in the UK I’m trying to keep up the tradition of over indulgence.   Happy Thanksgiving everyone!

Work

Slow TFTP Transfers to Cisco 3850 Switch

Leave a comment

I recently upgraded a couple of Cisco 3850 switches and noticed that the TFTP transfer rate to get the Cisco IOS files to the switch were horrible (approximately 200Kbps) which took a LONG time to transfer the 250MB file required for these switches.  After trying a couple different TFTP clients, and finding nothing that worked, I dug into the settings on my preferred TFTPD32 software and found that changing the TFTP setting for “Use anticipation window of” to 4092 gave me about a 4x to 5x improvement in transfer speed.  Still slower than I would have liked but definitely tolerable now.

TFTPD32 Settings showing "Use Anticipation Window"

Work

Sending Email from WordPress Hosted by GoDaddy

17 Comments

If you have just signed up for a newer GoDaddy hosting account (using either cPanel (Linux) or Plesk (Windows)) for a domain that hosts email externally from GoDaddy account you may have problems sending email.

I noticed this problem when trying to send email to new WordPress users (installed via GoDaddy) that I just created and they never got the introductory email from the server.

Mail sent from your web hosting account will be blackholed and never sent out to the external account.  You can confirm this problem by going into cPanel (sorry I don’t know what this looks like in Plesk) and use the Email Trace feature.  Just click Run Report (you don’t need any email address in the list) and look for recently sent email.  The message will show up as “Message Accepted”

GoDaddy Email Trace with Message Accepted

But click on the magnifying glass and the Delivery Event Details will show that this message is delivered to the :blackhole: address.GoDaddy Delivery Event Details showing "Delivered To: :blackhole:"

After an almost two hour phone call with GoDaddy support, their support person was able to contact a back end engineer who “changed something” on our account so that GoDaddy was no longer authoritative for email for our hosting domain and mail correctly started to flow out to the Internet.  I wasn’t able to get GoDaddy Support to clarify exactly what was changed on the back end but it is apparently a new “feature” of the newer hosting platform that hasn’t been reported previously.  There is currently no way to change this through the existing cPanel interface.

So please give your GoDaddy Support a call if you are running into this problem and push for them to escalate to the back end support as soon as possible.

Fun

From Rag & Bone to Boney M

1 Comment

As I’m continuing my British education (and hopefully yours too) after my previous posting about Rag & Bone Men, I only just found out about the musical group Boney M., a huge disco hit from the 1970’s and onwards, but apparently not a big hit in the United States.  Go ahead and click play below on the Rasputin song and read on…..

I originally heard a song from them while listening to BBC York during my workday.  When my wife came home that evening I asked if she had ever heard of Boney M. and couldn’t believe I had never heard of them.  Even my mother-in-law knew the name and remembered the band and music.

I spent most of the day listening to the various Boney M offerings on YouTube, so catch up on your 1970’s Euro disco music and get educated.