Redundant Datacenter Connectivity

Datacenter Connectivity

Goals

• Create redundant datacenters. For now, 2 of them.
• Redundant, diverse, physical circuits/paths
• Allow simple movement of services between datacenters => support the same IP address ranges in either location

An example:  http://connect.iltanet.org/ILTANET/Go.aspx?c=BlogViewer&BlogKey=d671b21d-b20a-4b07-84d5-3d4357723f0a

Issues/Discussion Items
Layer 2
to allow the same IP address ranges

• Circuits are different vendors taking very different paths with unknown infrastructure in between the datacenters. A failure could occur within the service provider, but all my ports show "up." 
• So, Spanning Tree and Port Channels will not work for "in between" failures
• UDLD, Unidirectional Link Detection, doesn't seem work on a "virtual circuit" over a provider network like Metro Ethernet.
• Cisco's OTV is supposed to accomplish a big part of this, but it seems that a much less overblown solution could at least provide the Layer 2 redundancy.

Routing Redundancy

• For routing to work, the (outbound)default route must be configured to go out through the same datacenter where the inbound traffic will be arriving. 
• So any kind of automatic redundancy that fails the inbound route to the secondary datacenter needs to also initiate a change to the default route of all machines in the "shared" networks.
• A compromise would be a "one button push" method to switch inbound and outbound routing to change between datacenters.
• Does OTV address this?

Nuts & Bolts

• A traditional solution would possibly involve 2 circuits each with a router at each end.  L3 redundancy between the 2 links using HSRP/GLBP, some kind of IP tracking to expose a service provider outage, and somehow create L2 tunnels over each one and use them via a port aggregation at the core switch.  The question remains whether this solution can be designed to show a link as down even if the failure is in between.
• There seems to be little in the way of configuration help or reference material for GRE/MGRE tunnels.  Even less for L2 tunneling.  I suppose the motivation for helping users do that is even less now that there is a nebulous thing called OTV that involves a big spend on monster Nexus 7K core switches.
• I haven't been able to get into the guts of this and do any testing.  Thinking it through on paper I always get bogged down with the L2 tunnel.  A possible example L2TP configuration is:

Router A:

pseudowire-class test
encapsulation l2tpv3
protocol none
ip local interface Loopback0
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
!
interface FastEthernet0/1
description LAN
no ip address
speed 100
full-duplex
xconnect 2.2.2.2 1 encapsulation l2tpv3 manual pw-class test
l2tp id 1 2


Router B:

pseudowire-class test
encapsulation l2tpv3
protocol none
ip local interface Loopback0
!
interface Loopback0
ip address 2.2.2.2 255.255.255.255
!interface FastEthernet0/1
no ip address
duplex auto
speed auto
xconnect 1.1.1.1 1 encapsulation l2tpv3 manual pw-class test
l2tp id 2 1

Data Center::Fire Suppression

FM200
Interesting information gleaned from overview from vendor:
- This system puts out a fire by quickly lowering the temperature of the room by 20 degrees or more. This also creates a vaccum in the room which, in addition to lower temp, puts out the fire. This change in pressure can displace ceiling tiles and stir up dust from the floor. Very shortly after the gas is deployed the room warms back up and the pressure in the room returns to normal.
- The gas is inert and not toxic to breath.
- The gas disperses sideways from a nosel that looks like a sprinkler head.
At our site this will be integrated with the same control system as our pre-action system.
So...it will work as follows:
- smoke alarm in the data center => the preaction system will release water control valve making water available to the system. The pipes remain pressurized so pipes still have only air in them until the heat from a fire causes a sprinkler to open.
- multiple smoke alarms in the data center => FM200 system will alarm, 30 second delay, the gas will be released. temp will go way down, ceiling tiles will be sucked down into the room--some will fall out, "hurricane" wind may blow more dust up from the floors, A/C system will be shut down to prevent air flow that would further feed fire. A few seconds later the room will warm up and the pressure will become normal. No cleanup procedure required. (just dust things off...)
Other interesting info from Q&A
- It is required to have a 4 foot square of ceiling around our sprinkler heads to allow for proper operation. If the sprinkler head is not at the top of the ceiling it will not heat up at the same rate as the rest of the room and not kick in soon enough.
- Sprinkler heads - bottom plate will melt off at 135 degrees F. That exposes an element that will melt at about 155 degrees F.
- in some cases locality may allow water fire suppression systems to be removed. This would likely require a backup system. Many times this is not allowed however either by the local statutes and/or building management.

data center::equipment rack::power

Equipment Rack power should not provide more outlets than a UPS circuit can service. A power strip the height of the rack will seldom or never be adequately supplied by one circuit of a UPS.
A suggestion would be to put 2-4 smaller PDU's (maybe the width of the rack.) Into separate circuits of one UPS to prevent overloading and tripping the circuit.