A question of Flash

I've come to have a certain, distaste, for solid state storage. Don't get me wrong, solid state it is extremely useful and convenient. If you need to store a bunch of photographs, or conveniently transfer documents, few things are better now than a USB stick or some sort of flash drive. However, there has been a trend pushing solid state storage units as replacements for hard-drives. There has also been a push from a certain group of people who use a certain file system, that their pre-existing file system is uniquely suited to use on Flash-Memory devices.

The case against solid-state storage hasn't exactly been helped by tech giant and all-around bad boy Microsoft. Consider one of Vista's features called ReadyBoost. It uses solid state memory in order to boost power on times by taking advantage of the extremely low access times of solid state memory compared to the magnetics based memory in most hard-drives today. Then there are flash memory vendors like Samsung that are heavily pushing high-priced notebooks with solid state drives citing the improved performance, light weight, and the complete absence of impact problems that could ruin standard magnetic drives.

So, it sounds like Solid State memory is the future of hard-drive computing. That might be. It is, however, a horrible choice today. Thing is, thanks to a couple of conventions I had met with one of the sales reps for one of the vendors that offers solid state drives. I sent the rep some questions asking about Linux compatibility, and received a surprising reply. They hadn't tested the system under Linux not because they don't offer the operating system (They do on other systems), but because the average life-span of their solid-state drive in normal usage... was estimated at 8 months for somebody replacing a desktop. For those who used the laptop in mobile patterns, such as at airports or away from the desktop but not on a continual basis, the drive was expected to last about a year.

Now, what you need to understand is exactly how expensive these drives and systems are. Back in April 2007 when Dell introduced Solid State Drives to it's rugged notebooks, the drives cost $549 for just 32gb. By comparison, just over $200 will net you a 320gb laptop drive. A quick search of Dell for the term SSD netted 1 computer that would feature the drive, the Dell XPS M1730. At a base price of $2399 it seems like a bargin. Until you get to the options screen and find that a single 64gb Solid State drive will run you $1000... and you have to also get it with a 200gb magnetic disk. If you have $1800 you can get two 64gb drives. It's not really that surprising. By itself Dell sells the 64gb Solid State Drive that is designed for laptop use... for $949.

So, if you get an M1730 with a single solid state drive, your looking at just about $3400. Which is why the rep I contacted said they weren't too worried about people buying the drives with such a short life span. Anybody who was shelling out $3400 or greater for a laptop probably wouldn't care that they'd be looking at buying new drives in a years time. According to the rep, most of the people who were in that price range already were buying complete new systems ever 6 to 8 months. So big deal if their drives went bust.

So, why exactly do solid state drives have such a lousy life time? Well, lets run some numbers. Take a basic flash music player, say something like an Iriver T30 1gb media player. If you can find one they run around $40. Now, take a 128kbps music file. That's 128,000 bits ever second. In 60 seconds of playback then, you'll have a bit rate of 7,680,000 bits. In terms of bytes, that 960,000 bytes every minute. Now, lets play that for two hours. 960,000 bytes every minute for 120 minutes. The total is 115,200,000 bytes in 2 hours. That's around 109.86328125 megabytes... and we'll just use 110mb to be polite.

So, in two hours running a constant stream of 128kbps music, a standard drive will transfer around 110mb of data.

A Windows NT5 system with 2gb of ram shows 2,096,354 in the process manager. On boot, the system cache is listed as 400,000 and rapidly climbed to 500,000, and the page file is listed at 455mb. After a few minutes the system cache had settled around 1,700,000.

On a Linux 2.26.22 system, 2gb of ram shows as 2,124,955,648 bytes. Disk cache is listed as 754,057,792 bytes which totals out to about 718mb in Kinfocenter, although the swap file is 100% free.

The point is this : in just a few minutes booting on the Windows NT5 system has completely blown past the total amount of memory usage a flash drive would get in two hours. The Linux system also completely blows past the flash drive's total memory usage. Now, what would happen if these systems didn't have such high memory counts? What happens when I load a game, a heavy duty graphics application, or start actually using the computer? In just a few minutes, the amount of memory transfer on a standard magnetic hard-drive under standard usage under Windows NT or Linux, can easily surpass the total amount of transfers a standard flash media player is expected to get in an entire day.

So, why are flash drives so ill-suited for such high read/write operations? Why does it matter how large the memory transfer is on a computer system? Well, you need to understand the basics of solid-state storage then, and there is no better place to turn to than the dedicated file-system for flash drives... JFFS2: The Journalling Flash File System, version 2. Track their information down and you'll find there is a nice FAQ on Infradead.org. As you read through the FAQ you'll find little tidbits of information, such as this one : Eraseblocks get worn-out (i.e., bad and unusable) after about 10^4-10^5 erase cycles. Magnetic Hard-drives? They don't suffer from wear out on Erase Cycles.

You'll also find this little tidbit:

USB sticks, CompactFlash cards and other removable flash media are not MTD devices. They are block devices. They do contain flash chip inside, but they also contain some translation layer above which emulates block device. This translation layer is implemented in hardware. So for outside world these devices look exactly as hard drives, not like MTD devices.

....

Also note, these devices are "black boxes". The way they implement this flash-to-block device translation layer is not usually published. And in many cases the algorithms used at this layer are far from brilliant. For example, many USB sticks and other cards lose data in case of unclean reboots/power cuts. So, be very careful.

Catch that? A hardware translation layer prevents the user from being able to determine the state of the drive itself. The details are often not published either on how the translation is accomplished.

One other detail needs to be mentioned as well... these little tidbit

Maintains 3 main operations: read from eraseblock, write to eraseblock, and erase eraseblock.

Bad eraseblocks are not hidden and should be dealt with in software

Magnetic hard-drives? Well, they only read/write, and bad blocks are generally hidden from the system... in the hardware.

See where this going? Flash drives have more operations to perform for the same functionality, wear out, and depend on the software to identify bad blocks. Magnetic drives allow the hardware to detect bad sectors, and are less complex. Now, this is coming from the people who are actively trying to write a usable file system that accounts for the differences in solid-state versus magnetic drives.

Why then... would Reiser be pushed as the best file system for solid-state drives? Oh yeah, now we come back around to the opening quote. Seems to be several people who are claiming that if you are going to be running a flash drive or a solid-state drive on Linux, you need to be using Reiser.

Now, I've gone into Reiser partially before. Back when I posted the huge log from Linux.com and the follow-up about the testing I provided the hardware for I couldn't say I was anti Reiser. I probably couldn't care less. Now... I'm beginning to wish that the Reiser people would just go away. It's hard enough getting the point across that solid-state drives are an extremely bad idea for operating system usage. It's not helping that some of the same people pushing the latest ReiserFS are now trying to push it as the best file system for an awful storage system.

The overall point then, if you look at a flash drive and think, "Oh that will be nice to run my Linux off of." Stop. It isn't. You WILL RISK complete and total information loss. You WILL RISK system stability. There will be a serious trade-off on usable life for convenience.

Now, if you have $1000 to drop on a dedicated solid-state drive designed to be used as a hard-drive... go ahead. I'm not going to stop you. I can't. I can tell you that you probably just wasted your money though.

****

There is a counter point to the entire solid-state memory storage issue. It was made by Gigabyte. It's called the Gigabyte i-Ram. It uses stock RAM on a SATA Bus, so it will easily hit transfer speeds of SATA 150 and sustain them. It is even faster than flash drives because it moves at the speed of System Memory... which it uses. It has a battery, so your data is contained for a while after you log out. Here's the catch though. The i-Ram only will take up to 4gigs. So, if you want appreciable space to work with you'd need 4 of these in a quad-stripe for 16gb.

Good luck.