Consumer hard disks: a large-scale reliability study

Backblaze, an online backup service, have published statistics about the reliability of consumer hard disk drives (HDDs) in their enterprise environment: http://blog.backblaze.com/2014/01/21/what-hard-drive-should-i-buy/

Throughout the last couple of years, they have applied different models of Seagate, Western Digital, and Hitachi drives in large scale (thousands of individual disks) in their storage pools, enabling them to collect conclusive statistical data, such as the “annual failure rate” (cumulative drive years of service / number of failures). An impressive visualization is the 3 year survival rate:

Backblaze blog: disk survival rate by vendor

Backblaze data: disk survival rate by vendor

Note: for a normal consumer type, a Seagate drive should (on average) last longer than indicated by the graph above, since Backblaze is putting their drives under heavier load than normal users do (Backblaze for sure performs significantly more write operations on the disks, also the normal user does not spin the drive 24/7). While there is no doubt that Seagate drives were identified to be significantly less reliable in this study (compared to WD and Hitachi), Backblaze still likes Seagate drives, because they usually are quite cheap.

You might be wondering why exactly Backblaze is using consumer hard disk drives for running their service instead of enterprise class disks. It’s all about money — especially the value-for-money ratio. Backblaze realized that the relative enhancement in reliability and warranty of enterprise class disks compared to consumer HDDs is not as high as the relative prize increment. On a small scale, it might be worth having enterprise disks, because one has less trouble with failing hardware. When operating on the large scale, however, Backblaze concluded that it is cheaper to operate with consumer HDDs. Obviously, Backblaze has to make sure (and they do, I guess), that data is stored with sufficient redundancy, so that the overall likelihood of data loss is on the same level (almost zero) as with enterprise disks. Also, Backblaze has to properly adjust their infrastructure and internal processes/guidelines according to the “high” rate of disk failures. Considering 30,000 disks being operated simultaneously, and an annual failure rate of 5 % (which is realistic, according to their data), about 4 disks have to be exchanged per day (30000*0.05/365).

For my NAS, I have recently bought Western Digital’s 3 TB disks from the Red line, which are energy-saving consumer disks ‘certified’ for 24/7 operation — a very good choice, according to technical reviews and Backblaze’s data. You might want to look into these Red line models yourself, they come at a very good price. So far, I was assuming that the differences in HDD quality on the consumer market are small. Backblaze’s data disproves this assumption and although Seagate models surely are fine for many types of applications, I won’t blindly buy Seagate anymore. It is impressive to see certain models being much more reliable than others (on average) under heavy load conditions.

I am looking forward to seeing more conclusive data provided by Backblaze in the next couple of years. This kind of data is something that usually is kept secret within the manufacturer companies, and normal tech reviews simply cannot provide these data as of missing statistics.