SU544: [Impact: Critical] HUH7210* and HUH7212* HDD firmware to prevent potential data loss or disruption / unavailability

Summary

[Impact: Critical = Data loss and/or cluster data outage.]

NetApp^® has previously identified that the drive models listed in the table below have the potential to fail at a higher rate than other drives shipped by NetApp. Due to the nature of this issue, NetApp strongly recommends performing this upgrade as soon as possible to avoid excessive idle time wear of the HDD that can lead to increasingly disruptive scenarios.

This bulletin is a follow up to SU478 to emphasize the criticality of specific drives that have not been upgraded. While the core guidance has not changed, this message is being upgraded to Critical due to the increasing age of systems with long running drives.

Issue Description

Drives on firmware versions less than those listed in the table above are at risk for a higher-than-expected rate of failure due to excessive background media scan, which can lead to data loss, disruption, or unavailability, if multiple drives fail simultaneously:

• In a multiple drive failure scenario, RAID limits may be exceeded, in which case a Volume Group would go Offline (or fail), and data in cache could be lost.
• In a single drive failure scenario, a drive will likely be failed for degraded performance. This would result in a degraded volume group.
• Note: In any event where more drives are impacted than RAID tolerance, immediate engagement with technical support is strongly recommended.

Symptom

Multiple drives all reporting timeouts would be a potential indicator of the problem. Repeated events like the major event log below can be used to try and detect this, but identification of drive failures as this specific issue requires engagement with NetApp Technical Support.

Date/Time: 4/1/23, 7:18:11 AM
Sequence number: 5012
Event type: 100D
Event category: Error
Priority: Informational
Event needs attention: false
Event send alert: false
Event visibility: true
Description: Timeout on drive side of controller
Event specific codes: 0/0/0
Component type: Drive
Component location: Shelf 1, Bay 8
Logged by: Controller in bay B

Additional Information

See Bug 1372240, 1407507

In accordance with the Support Services terms, always update NetApp products with the latest version of firmware and software to provide the best reliability, availability, and serviceability:

• Download drive firmware from the E-Series Disk Firmware page.
• Upgrade instructions: Upgrading drive firmware.
• Upgrade instructions for StorageGRID appliances using maintenance mode: Upgrading SANtricity OS Software on the storage controllers using maintenance mode.
• For more information: How to obtain the latest drive firmware for E/EF-Series.

Hot spare drives: To best maintain the continuous presence of hot spare drives available in the system, adhere to Hot Spares Best Practices and follow the standard drive replacement process if a drive fails.

Active IQ System Risk Detection:
For customers who have enabled AutoSupport on their storage systems the Active IQ Portal provides detailed System Risk reports at the customer and site and system levels. The reports show systems that have specific risks as well as severity levels and mitigation action plans. Drives that are not running the latest firmware is an example of such a risk. Not upgrading to the most current drive firmware could leave the storage appliance vulnerable to undesirable behavior.

Important: The purpose of this communication is for NetApp to notify its installed base end users about urgent and important product information that may affect product performance or reliability. The information contained herein and the distribution lists are NetApp confidential materials that are subject to restrictions on redistribution and that cannot be shared outside of this e-mail distribution list.

Services Partners Additional Notes

Media wear cannot be definitively identified by drive failure rates alone. NetApp Engineering needs drive logs from offline drives and/or drives returned via RCA to specifically confirm media wear failures.