Use a bunch of USB Flash drives in a RAID array.

Use a bunch of USB Flash drives in a RAID array.


Hello! Welcome to another episode. I got this crazy idea the other day. I bought this 10-port USB hub from eBay for like 8 bucks and I bought this in order to transition my server away from Firewire since Firewire’s been abandoned by the industry and so I’ll be plugging in all of my external hard drives via USB, but I had this crazy idea to connect a whole bunch of flash drives to this thing to see if I could set ’em up as RAID. My first experiment was just to connect two 8GB drives to the USB ports on my monitor. I tried copying a 900MB file to the first drive and it took about 4 minutes 17 seconds to write and then I performed a read test and it took 48 seconds So, the next thing I did was to go into the Disk Utility and configure both drives as a RAID. This was a success and I now had a single 16GB drive as far as my computer was concerned. I tried the same test again and found that the speed was essentially doubled. I now had a write time of 2 minutes and 6 seconds in a read time of 25 seconds. Now, I had wondered how the system would handle me disconnecting these and reattaching them. Would it matter what order I put them back? As it turns out, no: as soon as I connected the second drive, it would immediately remount to my desktop. I even tried it on a totally different Mac and it remounted right up as soon as I connected both drives. I even tried it on an old G4 system running Leopard and they mounted right up. Alright, for the large-scale test I decided I wanted to use this laptop and the reason is, this laptop already has an SSD in it and so when I hook up all the flash drives to here the main limiting factor in speed is going to be the USB 2.0 bus that I’m using to connect it with. So, I thought it was time to scale this up a bit. I got out the big USB hub and connected eight identical 8GB drives. These eight drives together only add up to an underwhelming 64GB but I could just as easily do a setup like this with only four of these 256GB drives and I’d have a whopping terabyte, which is a substantial amount of storage. OK, so what I did is I first copied the file to the first drive by itself and measured the time and then I copied the file to two drives in a RAID setup and measured the time again and I did this for three drives, four drives, all the way up to eight. So, looking at the write speed you can clearly see a huge advantage going from one to two drives and then a smaller advantage for each additional drive. Read speed, on the other hand, doesn’t seem to be affected as drastically which is weird because on the first machine when I tested the two drives on the side of my monitor, it was quite a bit more difference. So you might be wondering: why does making it RAID make it faster? A lot of people would assume in a RAID setup that one drive would simply fill up and then the next and so on but that isn’t how RAID works. The data split up evenly and written to all drives simultaneously. By doing this, the speed is multiplied greatly. So you might ask, “Is a setup like this practical?” or, “Is it even cost-effective?” At first I thought it would be since I was scouring over eBay and kept finding great deals on large-capacity USB flash drives, like 256GB drives for essentially 20 bucks; however, after reading this warning from eBay itself about fake USB drives, I soon realized that those drives were almost certainly fakes. Okay, so these fake drives I’ve been reading about – they’re typically 16GB drives and they’ve had their firmware hacked so they report back to the computer that they’re either 128 or 256 or more gigabytes in size. In reality, they are still only 16GB and they’ll actually work just fine until you put more than 16GB of data on them and then they’ll begin to corrupt data and so I guess the idea is that the user will buy them, they’ll try ’em out, they’ll appear to work, and it might be weeks or months later till they find out they’re fakes; by that point, it’s too late to send ’em back. So, apparently the real deal was gonna end up costing a lot more money than I originally thought. Also keep in mind, this is January of 2015 so if you’re watching this, you know, years in the future, keep in mind prices are probably different. I’ve decided to use 1TB as a benchmark. So, a traditional hard drive will cost you around 89 bucks; an SSD will cost you around 399 bucks; and using eight flash drives will cost you around 320 bucks. Interestingly enough, going to four drives would still cost about the same; however, there is a cost advantage that might not be obvious. Now, in my tests I’ve just been striping these drives as RAID 0 but what if I wanted a little bit of redundancy? Well, if I was using a regular 1TB hard drive I would need to add an additional 1TB hard drive for redundancy, which essentially doubles the cost; however, if I were using eight flash drives I’d only need to add one more flash drive for a parity drive. That does give at least some cost advantage to the flash drives. So, what are some of the advantages of using a setup like this versus a traditional hard drive? Most of the advantages will be similar to using an SSD: there’s no noise; the low energy consumption, which also translates into no heat-buildup; these will be instant access – no waiting for the drives to spin up after they’ve gone to sleep; they’ll be more reliable. So let’s face it: mechanical hard drives have a very limited operational lifespan. USB drives are far more durable and then there’s the raid redundancy I was talking about earlier. OK, so what are the disadvantages? Primarily, the disadvantages you’re gonna get from using flash drives is gonna be, well, cost – that’s the one thing we already talked about – and the other disadvantage is gonna be write cycles. Now, in a situation like mine with long-term storage I’m typically not writing very often to my server; I write to it, you know, a few times – maybe once or twice a week – but most the time I’m reading from it, so I don’t think the write cycles would be an issue that would affect me but everybody’s gonna be different, of course. So, I don’t expect everyone to go out and just start buying a bunch of flash drives, I just thought this was kind of a neat experiment – kind of one of those “What if…”, make-you-think type things. So, anyway, I hope you enjoyed it, and I’ll see you again next time.

100 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *