Someone on Twitter made a comment about a replication solution being expensive.  That particular solution is expensive, but I think that expense is relative to the problem you’re trying to solve. 

Something I always tell my customers when designing solutions is "You can have availability, performance, or an inexpensive solution.   Choose two."  That’s right, no matter how hard you try you can’t have all three.  Each slice of the pie requires sacrifice of another piece. 

I’d love to hear your thoughts…

I had an opportunity to sit in a day long meeting with EMC and a customer last week. We did a walkthrough of the customers current environment and how we can simplify the overall environment to enable the customer to move to a more nimble, simplistic architecture. The customer has been an EMC customer for a little more than three years now and they have asked me to come in and bring all of the silos of infrastructure into an overall solution architecture. It has been fun so far and I’m looking forward to working with them and EMC over the next few years as things come together.

A Little History

I used to be an EMC customer back in the day and had a few Clariions running some small storage environments. They worked well, but I wasn’t really impressed with the EMC service and the overall kluge that was the EMC machine at the time. Today I work for an EMC partner (amongst other vendors) and until recently I spent most of my time competing against them rather than working with them. Chuck, Barry and I have had several disagreements/discussions in the past over technologies (just look back in time on this blog) and I have nothing but respect for those guys.

Impressed with the new EMC

So back to the meeting we had last week. We had discussions around BURA, Unified Storage, Encryption, RSA, and the list goes on. Just the amount of resources that EMC was able to bring to the table, on short notice, was impressive. Each individual was knowledgeable about their individual product, but also was able to spout the overall EMC messaging well. They were able to show how their individual product meshed with the other EMC products that were relevant to the customer.

Most impressive about the day were the products in the pipeline slated for the near future. In the past EMC was a company with a lot of products that didn’t really fit well together. Now they are a company that has brought things together and it feels like they have a true vision on where they want to go. Not just a vision but a plan on how to get to where they want to go. The end game is something that I always felt they lacked in the past. You can actually see how the differing products are going to come together to create an entire solution. Beautiful!

What will others do?

So that brings me to the smaller players in the storage industry. What are they going to do? They don’t have the breadth of products that EMC has. They don’t have the money EMC has. It’s going to be interesting to see how this all plays out. The good thing about the company I work for is that I am able to pick and choose from all the vendors we partner with to create the perfect solution. Storage companies just don’t have that ability so they are going to have to partner with a company like mine to even think about competing with EMC. For the storage only players, I’m thinking they’re probably just waiting to be bought. 3Par anyone?

I posted a tweet yesterday about a migration and upgrade I was doing and received a couple replies asking to let them know how everything goes. I decided to create a blog post for everyone to read rather than replying in 140 character tweets. =)

Background: This customer has a very simple SMB setup as far as VMware goes. 3 servers in a single cluster, only a few vSwitches per host, and a HDS AMS disk subsystem. They wanted to upgrade to vSphere and are capable of taking some downtime to do it. (Not that downtime is required.) They bought 3 new Dell 710 servers to run vSphere on.

There are at least 20 different ways to go about this and I wanted to keep this upgrade as simple as possible Since the customer can take some downtime for their VMs I decided to do a Cold Migration of the VMs. This is by far the simplest and runs the least amount of risk for having problems with the VMs. The customer, while understanding VMware and the administration of it, could not troubleshoot issues once I was no longer onsite.

Here is the process that we walked through for a successful upgrade and migration:

1. Upgrade vCenter to version 4 – You need to read through the upgrade guide before you attempt this upgrade. There are some specific permissions changes to networks and datastores that could bite you after an upgrade if you don’t understand the way that vCenter changes read-only attributes. We’ll leave it at that. RTFM!

2. Install new ESXi servers on new hardware and add them to the existing cluster. Ensure all updates and patches have been applied. – Let’s discuss a few key reasons that I pushed this user to go with ESXi. All new implementations I have done over the past year have all been done with ESXi.

A) They were not running any fancy monitoring tools in the Service Console.
B) ESXi is a much smaller attack surface for hackers since RedHat is no longer running underneath.
C) Because there is no RedHat, VMware has full control over all patches which will allow for much quicker turn around on bug fixes, etc.
D) The user didn’t have any scripts or jumpstart servers that required the Service Console to run. Even if they did, we could have re- written them to run against the vMA (Virtual Management Appliance).
E) No Service Console! Personally I see more people get on the Service Console and make a mistake that causes a lot of problems. “Let’s see what this command does.”
F) THERE IS NO TECHNICAL REASON NOT TO GO TO ESXi!!!!!!

3. Map out all networks and datastores attached to 3.5 cluster – Self explanatory. You need to know what you need to know.

4. Create identical networks on new vSphere cluster – Again SE. The VMs will have to connect to the network after the migration. If the networks aren’t there, migration validation will fail.

5. Test cold migration to local storage and test networks with a test VM – Just test things out prior to the big outage.

6. Remediate any changes that need to be made. – SE. Fix any problems that pop up.

7. Present all datastores that reside on the old ESX servers to new ESXi servers. Verify connectivity, specifically LUN IDs. – Since the new ESXi boxes are part of the same cluster we can present the same LUNs/Datastores to them without any potential problems. Scan them and bring in the VMFS Datastores and verify everything looks OK.

8. Shut down all VMs – SE.

9. Cold migrate all VMs to new ESXi servers. – SE.

10. Remove all ESX servers from cluster. – This will ensure that during the next step DRS won’t try and spread the workload to the old servers. Just make things cleaner and less checking on hosts for DRS as a whole.

11. Upgrade the VMware Tools and Virtual Machine Hardware on all VMs. – Using Update Manager go ahead and upgrade things prior to putting them back online for your users. A reboot is required so the users would see interruption anyway. I’m not going to walk you through this. RTFM!

12. Install the vMA for command line management of the ESXi hosts. – SE.

On the ESX vs. ESXi front, I truly believe that once you try it you won’t even notice a difference and you’ll probably like ESXi better. If you’re an old ESX shop that loves the Service Console, then I would challenge you to use the vMA for a month and I would bet that you won’t go back.