In the last part of this article we covered the basics around what makes audio file data delivery different, and what to look out for. In this part we cover RAIDs and what to consider when connecting them to your computer.
RAID, BlackFlagMagic and Other Data Pest Control Tools
As of 2017, a single 7200 RPM hard drive running USB/eSATA can be expected to read/write around 160 MB/s. This is an amazing jump in throughput in the last 7 years. I have an array of 4 drives from 2010 which can only do
160 MB/s in total. But even a modern single drive, because of the nature of spinning platters and heads which need to move across those platters, are not going to be sufficient to access lots of files at once or even single large files for any length of time. There needs to be an “array” of drives present which can share the data load from demanding applications. We call these “RAIDs” or Redundant Array of Independent Disks. You can take four drives which can do 160 MB/s and format them in such a way that data which comes to them gets distributed across them bit-for-bit, drive-by-drive. They make enclosures specifically for this where you drop 4 internal SATA drives in, and plug a usb 3.x cable into the enclosure, and voila! Those four drives show up on your computer desktop as a single super fast drive. You can test the speed of any drive or RAID with BlackMagic’s Speed Disk Test program. It’s free, and does a good job of telling you what footage you can play/capture based on the stats of your drive. There are some things to keep in mind.
Hard drives are dumb instruments. They don’t know they’re in a RAID or an enclosure, they only know to read and write data. So you need to have a RAID controller in order to have these drives operate in concert. Enclosures generally have hardware RAID controllers built in which allow you to format the drives in various RAID configurations, or just have them JBOD (Just a Bunch Of Disks). Hardware controllers are inexpensive but robust devices whose sole aim is the maintenance of data integrity on and off the drives, and maintaining the union of the separate drives. If a disk goes bad, they’re smart enough to rebuild a replacement drive (if you’ve used a RAID system which allows it), and they do a good job of basic data management.
“I Know My Data Is Safe Because I Have a Kick-Ass RAID Controller.”
The trouble with hardware controllers is they usually have very few controls, and you generally have to use the
same size drives. Software RAID systems, while are only as reliable as the computer operating system you’re running, transcend this. You can use all manner of different sizes, types, brands and interfaces to create a RAID, and they run flawlessly. Moreover you get all manner of controls including various ways to rebuild, manage integrity and keep data secure from the most regularly destructive force to critical data: human error.
“I’m Going To Double My Speed By Doubling My Drives!”
Now you’d think, “well, if I’ve got 4 160 MB/s drives I should be able to get 640 MB/s!!” Not so. Because drives are always spinning, the mechanism doesn’t have instant access to all the data it contains. It has to wait for the data to spin around to the heads and the heads to move to the data. Sometimes, depending on the speed of the drive, it can take up to 30 ms to get there. In data throughput, that’s an eternity. While having a RAID helps with this, now and again it hinders, because you might have the drives spinning “just so” that requires all drives to have to do a 300º rotation before the next piece data can come from any of the drives. Everything has to stop until data can be found. The more drives you have in your array, the more chance of this happening.
Of course, this takes place very quickly and doesn’t happen that often, but lesser versions of this scenario happen every rotation of one of the drives in your array. This disallows the drives to work at their maximum capacity. How your have formatted your RAID is also a major factor in performance which we will talk about later.
My experience is Tweet: you can expect a 4 drive RAID 0 to be able to crank out 530 MB/s without trouble. This reflects a 18% reduction in efficiency of the drives, but a 331% increase in speed from one drive. But just before you say,
“Yes!! I’ll make an array of 20 drives!!,”
Adding more drives creates diminishing returns. A 4 drive array is not twice as fast as a 2 drive array. An 8 drive array isn’t twice as fast as a 4 drive, and so on. This also doesn’t take into account data integrity if one of your drives goes down, because in the case of a 20 drive RAID 0, if one drive goes down, you lose everything.
“I’ve Got The Fastest RAID Ever Plugged Into My eSATA Port!”
Also, you need to check you connection bandwidth and determine if the desired drive speed throughput exceeds it. For example: I used to have a two drive array attached to Firewire 800. I thought I was so cool. Except, if you know that firewire 800 means it’s top throughput is 800Mb/s (lower case “b”) or 800 mega-baud, (baud is another word for bit, and there are 8 bits in a byte), then you know that even one 160 MBYTE/s drive won’t go above 100 MBYTE/s on Firewire 800. I was terribly choking my array by 67%. USB 2.0 is even worse at 480 Mb/s. Connections to consider include:
USB 3.0 = 5 Gb/s,
USB 3.1 = 10 Gb/s,
eSATA = 6 Gb/s,
Thunderbolt 1 = 10 Gb/s,
Thunderbolt 2 = 20 Gb/s.
So just some simple math: my 5 drive RAID can do 600 MB/s easy and one of my 4 drive RAIDs can do 430. They’re both on the same USB 3 bus. 600 MB * 8 bits per byte = 3 Gb/s. 430 MB * 8 = 3.44 Gb/s. 3.44 Gb + 3 Gb = 6.44 Gb. As you can guess, this exceeds the USB 3.0 connection’s bandwidth by 1.44 Gb/s or 180 MB/s. That sucks. The good news is that one of the RAIDs is for Video Rendering/Uncompressed playback and the other is for Audio Mixing. They’re never driving at the same time. As a knowledgeable user, I’m fine with this, but as a storage delivery system, it’s not the best.
These things must be kept in mind whether you’re doing audio or video. If you have a Mac, you should also not be fooled by having multiple ports on your machine. They are almost always locked to a single controller and it means you only get one bus for all of them. A notable exception is the dual t-bolt connection on the Mac Pros 2013 and up. Those are two distinct ports on two distinct controllers.
In the next part of this article, we’ll cover the types of RAIDs I recommend for audio and how best to implement them.
Let us know your thoughts on this article or Tweet about it!