Recovering from halted Isilon upgrade
The other day I was upgrading my Isilon cluster to 7.0.1.8. Since I wanted to minimize customer impact I elected to do a rolling restart – something I’ve done several other times without problems. Isilon has a feature in SmartConnect Advanced which allows the cluster to dynamically move IP addresses between nodes of the cluster. What that means to my upgrade is when a node reboots, the IP address moves to a different node and my clients don’t notice the impact. The entire upgrade process usually takes about 10 minutes per node. About three nodes into the rolling reboot, the node I was running the upgrade from, lost network connectivity in an unrelated problem. This stopped my upgrade process, leaving me running two different versions of OneFS. The fix was actually pretty simple, I had to restart my upgrade process. In order to do that I had to stop the upgrade service which was already running onRead More →