your comment like "my box is bigger and faster than yours" above triggered me.
Before I started working in my current environment (Cisco/EMC)  I too thought that SAN switching was always/better faster than the IP kind.

Needles to say I was VERY supprised to see that this shop tends to do their synchronous replication over IP, and the asynchronous replication over FC. Both over 200KM links. The IP features like fast-write acceleration and compression seems to speed IP up quite a bit over this distance.
The FC latency is about 5ms over this distance, so the synchronous replication over FC is simply not feasible.

PS.  I always have problems with formatting if I cut and paste from Word into WordPress – the paragraphing always disappears.  So I always have tp type directly into wordpress.

Hope that helps

I love designing huge, complex environments.  But, when simple works I find it easier to go that route.

This was a largish core/edge Cisco solution.  I used a special non trunking VSAN for the management traffic.  It make my life quite easy.  Cisco were a bit coy on this setup and could only state one problem with it as far as non hardware issues and that was the chance of FCNS dying but it will automatically restart.  Considering, each VSAN has its own, I took that chance.  IF the principal switch died, I could have an issue or two but such is life.  The other core switch would take over and it still offered an alternative path for all the systems.

Troubleshooting the SAN was easy but they were only minor issues that made me even more aware of what I was doing.I had the odd ISL and some were quite big in size.  I quite liked my setup.

The network people never cared about the SAN switches until they saw the MDS 9513 and as it is big (in size and performance), I feel they felt a bit let down by their own equipment.  Some even tried to take over the SAN but I told them what I thought about THAT!!!

Mind you I had to sit down and talk about the Optical network when it came to our DWDM native SAN extension.  So, in a round about way, they are part of the SAN.

In SAN fabric services are there but no one seems to want to use them.  I was very keen on the Cisco Network Accelerated Serverless Backup but I think no one is game to use it due to the cost.  Sooner or later, fabric services will take off.

I tend to think that with the ever expanding IP performance,  FC SAN's  might  have a limited lifespan or they will decrease.  I doubt it will be in the next few years but eventually, FCoE and iSCSI will take off.So, in the interim, give me big and complex SAN's…. It justifies my pay.

Stephen