(Posted LinkedIn March 2013)
Last week I installed the RCFC cards in to our 2x P10400 3PARs. Cabled all up via our Brocades and zoned everything 1:1. We have a 3PAR in each DC. The RCFC cards are installed to slot 1 of each node, there is 4 ports per card, we are only utilising 2 ports. Once the cards are installed in the 3PAR you can see the ports in the InForm Console but they are not truly classified as RC ports until after zoning. Once that is done they drop into the RCFC tab in the InForm. At this point however you can still configure the port settings: i.e. it’s going to be a RC port, it’s going to run at 8Gb, we want interrupt coalescing ON etc.
We have 2 nodes per 3PAR. So for the zoning:
N0:S1:P1 on 3PAR#1 is zoned to N0:S1:P1 on 3PAR2 via Fabric1.
N1:S1:P1 on 3PAR#1 is zoned to N1:S1:P1 on 3PAR2 via Fabric1.
N0:S1:P2 on 3PAR#1 is zoned to N0:S1:P2 on 3PAR2 via Fabric2
N1:S1:P2 on 3PAR#1 is zoned to N1:S1:P2 on 3PAR2 via Fabric2
In the RCFC tab of the InForm Console is where you setup the RC configuration. In the GUI you drag the same ports (identical to your zoning) between your 3PARs to create the Remote Copy Pairs. As we have 4 ports, that means I have 4 Remote Copy Pairs. So now you have RC setup and ready to roll, what to replicate is next.
My RC setup is Bi-Directional. There are quite a few different supported configurations.
To replicate data you need to setup Remote Copy groups. A Remote Copy Group is just 1 or more volumes that you want to replicate. When you create a RC Group you give it a name, which 3PAR is the source, which is the destination, and what volumes you want to replicate, and if you want the replication to be SYNC or ASYNC. You can also set the group to auto-restart after a link failure is restored.
When adding volumes to the RC Group, the destination volumes can already exist or can be created on the fly as part of the RC Group setup. I preferred to create the destination volumes first as I had more flexibility and wanted to be fussy with their naming and CPG placement etc. All my volumes being replicated are being Thinned at the destination, and all running in SYNC mode. Once you have finished creating your RC Group the syncing automatically starts. You now have a RC Group created and the source volume is the Primary (RW), and the destination the Secondary (RO).
If you have a lot of data it would probably be best to do a block based backup-restore to the destination volume for the initial seed, then RC will sync the deltas. We don’t so RC copied it all – it flogged our 1GB/s pipe for most of the weekend doing the initial seed, but after that data throughput dropped. Haven’t experienced any host-write latency yet, but are monitoring. A host writes to its local 3PAR and the write needs to be sent across the link to the destination 3PAR before that acknowledges back to the source 3PAR and then acknowledges back to the host.
I have carried out some initial testing on a Windows 2008 VM with a script I had running on it to create 100,000 files in sequential order. I forced power-off of the VM while it was creating the files, and then failed-over the RC Group, which changes the Secondary (RO) to Primary (RW).
I then exported the new Primary volume to an ESXi Host at the destination DC, mounted the volume on the host, browsed the data store and added back in the VM. I had to manually add back in the Network is was on (VM- Edit Settings) and then powered it back on. I checked the files it created and as I had the RC Group for this volume set as SYNC, all the files were there up to the number I powered it off at. It WORKED. Never set anything like this up before so that was a WIN!
I then kicked off the script again on the host and forced power off again, and did the RC Group failover, this time back the other way to the other DC. Again after bringing up the VM on the other side, all the files were there – no issues.
Remote Copy – Simple, intuitive, and it works!
Musings from a past life 