By Kirk Drake | February 11, 2013

Top Reasons Credit Unions Failover to a Hotsite

If you are responsible for your organizations Credit Union Business Continuity Plan or maybe you are in the Credit Union technology department – this post is for you.

Since Ongoing Operations was created as a Disaster Recovery CUSO in 2006 – we have helped hundreds of Credit Unions and other clients handle real disaster situations.  This post is a compiling of the top reasons Credit Unions failover to their hotsite in our experience.

Most Common

Failed server – The vast majority of our clients have multiple servers with some form of virtualization, SAN, or some considerations towards high availability design.  Even so, we help about two Credit Unions every week recover failed servers.  We have seen failed hardware, software, configuration issues, and many other problems and we are happy to help.  The beauty of a Business Continuity Solution provider is that we are here to help with the small and large events.  Usually the servers that get the most attention involve e-mail or Microsoft Exchange.  Long term we think this will slow as Credit Unions move to managed servers or to secure community clouds for their infrastructure requirements.

Down Connection – While many of our clients wouldn’t say any individual connection is the most important thing for the Credit Union – the failure frequency on these means that we recover them almost as much as servers.  Using our Connector product, Credit Unions regularly reroute third-party connections (ATM, Debit, Credit, Online Banking, Federal Reserve) etc. through the Ongoing Operations network.

Weirdest

Transfer Switch Explosion – One of our clients had a generator designed to run the Credit Union in power outages.  Along with a generator comes a transfer switch which is there to transfer the power from street power to the generator as needed.  In this case – the transfer switch exploded causing an evacuation and about a 36 hour power outage.  Of course – with no transfer switch the generator didn’t do much good.  In that case, the Credit Union relocated their call center to our Maryland hotsite and continued providing member service throughout the event.

Power Glitches – Perhaps one of the most annoying situations is when your power is on but unstable.  It could be that it is on for a few hours but then goes back off…or maybe it flickers or has spikes.  We have seen both and they are one of the most difficult situations for a Credit Unions decision making process.  Our opinion – if your power isn’t going to be stable for the next 24-48 hours or the power company can’t give you a straight answer – go ahead and declare, start recovering and get about 5 or 10 hours further.  By then, all of your Credit Union technology should be ready to go at your hotsite and you it is just a choice of waiting.  Plus the worst case – you got to do a solid disaster recovery exercise.

Weather Related

Hurricanes Katrina, Gustav, and Sandy – all resulted in multiple declarations at the same time.  Almost always these are our most common major declarations.  The advantage with hurricanes is that everyone can prepare.  Even so – we have seen some weird problems following a hurricane including forgetting tapes in time vaults (make sure you have your checklists ready).   Our experience says that power issues, flooding, failed telecom, and being able to access your buildings are the biggest problems caused by Hurricanes.

Derecho – Equally frustrating and impactful was a random freak storm called Derecho that hit the Washington, DC area in the summer of 2012.  This storm knocked out mostly legacy telecom infrastructure and caused multiple Ongoing Operations clients at the same time.  Our advice – move as much of your telecom infrastructure to Internet and SIP trunks as possible.  Those worked great.  Stay away from legacy DS3, SONET or other old telecom infrastructure that doesn’t have the redundancy or flexibility built in.

All in all – if you have a solid Contingency Plan and perform Business Continuity tests you should be fine.  Don’t pretend it is just an IT problem and involve your whole organization.

Top Related Other Articles

Ways to engage your staff in Disaster Recovery Testing

Ways to improve your Disaster Recovery Plan by outsourcing

Best ways to lower your RPO, Recovery Point Objective

FOR MORE INFO – FILL OUT THIS FORM