Below is a step-by-step guide on how to add a third node to an existing 2-node Storage Spaces Direct (S2D) cluster with the production workload still running.
1.) Ensure that the cluster is configured with a witness.
2.) Pause/Drain a node in the cluster.
3.) Place the paused node's drives into a storage maintenance mode.
4.) Physically add the 3rd node into the 2-node cluster configuration by cabling the three servers like the image below:
DataON2UPlatforms:
DataON1UPlatform:
5.) Make sure the node that is being added to the cluster has the same firmware and drivers installed as the two servers in the existing 2-node cluster. Also install the necessary Windows features and configure the network correctly. Before adding the third node to the cluster, run a cluster validation.
8.) By adding a third node to the S2D cluster, 3-way-mirror resiliency is unlocked; Run the following command to configure this setting on the existing storage pool.
The value of business data grows to the extent that companies simply cannot ignore the importance of a safe data storage. Modern technologies offer a variety of methods how to warrant a highly-available and fault-tolerant storage. One of the techniques includes synchronous mirroring – an approach when a source storage has an exact replica (one or more). At the same time, the data is considered written only when the primary storage receives a signal that all secondary copies have been created. This document suggests taking a look at two-way and three-way synchronous mirroring to better understand the benefits each of them has behind.
Cost-efficiency
2-way synchronous mirroring
This configuration requires storage redundancy on the nodes. The use of RAID10* is recommended. 2-node HA ensures the synchronous mirroring of data between two storage nodes. Taking into account that each storage node has only 50% of usable capacity with RAID10, synchronous mirroring makes the further dividing of those 50% by half resulting in the underutilization of storage capacity – only 25% is used.
* RAID10 use is recommended for a HDD setup. In case of RAID5 use in a HDD setup, there arises the risk of disk failure while rebuilding RAID5. RAID6 in a HDD setup gives low write performance. At the same time, RAID5 and RAID6 configurations can be used for SSD setups due to a high tolerance to physical failures and faster performance of the latter.
With 2-way synchronous mirroring, usable capacity is 25% – ¼ of storage space
3-way synchronous mirroring
Non-redundant RAID0 configuration provides the highest level of performance, therefore, RAID0 can be used for performance. Synchronous mirroring between 3 storage nodes with RAID0 configured on each, results in 33% usable capacity and thus provides a higher level of storage utilization compared to 2-way synchronous mirroring. While it looks like this configuration does not require storage redundancy on nodes (since 3-way synchronous mirroring already ensures the required level of data protection), you should evaluate the potential risks of disks failing and potential data loss probability (if disks failed on all 3 nodes). Assuming the above, please make sure that the data is protected by backup applications according to the 3-2-1 backup rule.
With 3-way synchronous mirroring, usable capacity is 33% – 1/3 of storage space
As a result, 3-way synchronous mirroring increases the storage utilization rate. The cost-efficiency of this configuration differs depending on the medium type – spindle or flash. The difference is shown on the charts below.
Increased reliability
2-way synchronous mirroring
2-way synchronous mirroring provides 99.99% uptime. The outage of one storage node results in a single point of failure and immediately brings the system to a degraded performance mode. Cache is flushed and turned from write-back to write-through mode on the running node. A number of MPIO paths is reduced twofold because one node is down. Consequently, the storage performance falls.
With 2-way synchronous mirroring, there is a risk of downtime
3-way synchronous mirroring
3-way synchronous mirroring provides 99.9999% uptime. No single point of failure occurs when one node of a 3-node storage cluster goes down. In such a situation, storage performance falls by up to 33% because the system loses 1/3 of the MPIO paths. Performance-critical applications usually can continue running in an ordinary way. The 3-node HA configuration tolerates a double fault and retains the availability of service.
With 3-way synchronous mirroring, constant system uptime is ensured
Higher performance
Higher storage performance is influenced by a number of factors, which include I/O policy, RAID level and cache policy. The effect of these factors is given below:
MPIO paths
2-way synchronous mirroring
With Round Robin/List Queue Depth policy used, I/Os are processed up to two times faster comparing to a single-node configuration.
3-way synchronous mirroring
Owing to the Round Robin/List Queue Depth policy, the I/Os throughput rises by a factor of 3 compared to single-node storage.
As a result, performance is increased by up to 50% compared to a 2-node configuration.
RAID10 vs RAID0
2-way synchronous mirroring
This configuration strongly requires extra redundancy for data protection on the storage nodes themselves. This redundancy can be provided through the use of RAID. RAID10* is recommended for a HDD setup as it ensures mirroring between the disk stripes and makes fast reads and writes. However, storage utilization is considerably low because the same data is mirrored and stored on two stripes of RAID.
* Use of RAID5, RAID6 for a HDD setup is possible but not recommended because of the high probability of a disk failure while rebuilding RAID5, and the low write performance of RAID6. At the same time, RAID5 and RAID6 configurations can be used for SSD setups due to a high tolerance to physical failures and faster performance of the latter.
3-way synchronous mirroring
RAID0 can be used for performance. Both reads and writes are faster here as the system reads the data from all disks simultaneously. With RAID0 you should evaluate the potential risks of disks failing and potential data loss probability (if disks failed on all 3 nodes).
Cache policy
2-way synchronous mirroring
If one node fails, the cache is flushed and turned from write-back to write-through mode and the system immediately switches to a degraded performance mode on reads.
With 2-way synchronous mirroring,cacheis flushed and turned to write-through mode causing critical performance degradation
3-way synchronous mirroring
If one node goes down, the system downgrades from a 3 to 2-node cluster and continuous operation with minimal performance degradation (about 33%) due to the absence of one node and because a number of MPIO paths is reduced.
With 3-way synchronous mirroring, cache policy remains write-back that results in minor performance degradation
Important note:Even the highest possible level of redundancy and reliability does not ensure 100% protection against data loss, e.g. due to malicious actions or disaster. So, nothing compares to a good old backup that substantially increases the chances your data is in place.
Conclusion
With synchronous mirroring admins can decide on either two-way mirroring – to ensure basic data protection and high availability, or three-way mirroring – to increase overall system reliability and to also “play games with” cost-efficiency and performance.
