I just finished building the production release of my fileserver hardware and the last of it just went in this past weekend; along with the OS installations.

For the past day and a half, I have been working on consolidating 525 GB of data from across my servers to the new one.

So, what is ZFS? In short, it is a file system that allows for a lot of storage, flexibility, scalability, and performance. (No, this is not just marketing jibberish. I've actually been testing it for purposes of deployment).

Originally an acronym for "Zettabyte file system", it was then reduced back to a pseudo-acronym that "vaguely suggests a way to store files that gets you a lot of points in Scrabble." [1]

A summary of the details of it's capabilities can be found on Wikipedia here.

Additional details can also be found from ZFS' manager/creator (or lead creator), Jeff Bonwick's blog.
ZFS Learning Center
ZFS - The Last Word In Filesystem Presentation (PDF)

So, what does it all mean? What can do it? Why should anybody care?

Well, for one - RAID controllers are expensive. My USED 16-port Adaptec SATA-I RAID HBA was in the sub-$500 range. So, if you have growing file storage needs; how do manage all of it without having to use expensive controllers? Expensive controllers, means that it has special system requirements such as PCI-X or PCIe x8 slots; which limits your ability to use it because of hardware constraints.

Second, once you get the controller; then what? Hardware RAID? Software RAID? What if the array failes, the drives die, or the operating system get's corrupted? Then what happens to my data?

Well, fortunately for ZFS, administration is a piece of cake.

For example, from within Solaris; to create a stripped array (say 4 SATA drives: c0t0d0 c0t1d0 c0t2d0 c0t3d0), you would type:

# zpool create test c0t0d0 c0t1d0 c0t2d0 c0t3d0

And that's done. What happens if you want to create a mirror of stripes (i.e. "RAID10"):

# zpool create test c0t0d0 c0t1d0 mirror c0t2d0 c0t3d0

And that's done. But the coolest thing about it is this thing called "RAID-Z". There are particular advantages of RAID-Z over traditional RAID5 that I won't detail here. And to create a RAID-Z array:

# zpool create test raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0

(I won't detail any more of the commands). But unlike historically with software driven RAID, you have to partition the drives, set them up, bring them online; etc, etc etc. Painful, slow, and lots of work.

They've automated a lot of those processes.

So, now that we've gotten all of that out of the way - what can it do? How does it perform?

Because most of my work is going to be write speed limited (since writes are always slower than reads), I focused primarily on benching write speeds (but knowing that read speeds are faster than that).

With my dual Socket F system; here are the results:

1 OS drive, plugged onto the onboard SATA-II controller; plus 3 drives managed by ZFS:

My minimum write speed was 78 MB/s (RAID-Z) and 120 MB/s (stripped). So I figured, if it were possible to plug more drives in, that means that performance should go up from there.

With 1 OS drive plugged onto the onboard SATA-II controller; plus 16 drives on the Adaptec 21610SA 16-port SATA-I RAID HBA:

16 drive write: 48.9-75.7 MB/s (RAIDZ-RAID0)
8 drive write: 40.5-60.5 MB/s
4 drive write: 38.6-57 MB/s
2 drive write: 29.3-52.4 MB/s

Comparing that to the onboard 1 drive Sandra 2005 (file system) results: a single drive should be able to do 67 MB/s.

So what does that mean? It means that I am limited by the controller.

On 16 drive read (RAID0), I can get about 112 MB/s (or 74.7% of the theorectical limit for 150 MB/s for SATA-I).

That also means that so long as I have multiple systems feeding it data, that I can use my gigabit network to 76% of capacity before I start hitting bottlenecks.

What about CPU usage? When I was copying the data over (migrating the servers); I took a look at my CPU usages.

Going from a Duron 1.8 GHz to the new system; the Duron was at 14% CPU utilization, and the dual Opteron 2210 (Socket F) was at 0.8, peaking at around 1.3%. So, for all practical intents and purposes, it was idling AND doing the parity calculations at the same time.

That also means that if I move to two 8-port SATA-II controllers (non-RAID); I should be able to see the numbers go up much further than that.

Another test that I also did for "fun" was to enable compression to see the maximum read and write rates, and testing it with a 10 GB file of zeros:

16 drives: write - 500.1 MB/s, read - 1,005.4 MB/s
8 drives: write - 450 MB/s, read - 1,005.4 MB/s
4 drives: write - 498.7 MB/s, read - 986 MB/s
2 drivse: write - 516.2 MB/s, read - 1,003.5 MB/s

It looks like that my max host-to-interface speed about 0.5 GB/s for writes, and about 1 GB/s for reads; and it didn't seem to matter whether I was using the onboard or the controller.

In theory, if you have a system that can run Solaris, you could use four SiI3114 or SiI3124 SATA HBAs for a very inexpensive, but high performance file server.

It even survived me literally pulling the plug on it; and plugging it back in! (unlike my Linux system; which died).

Wow, only dorks use Solaris. That went out back with parachute pants.
Maybe you should try Lindows.

awww... how cute.... look at the fashion accessory

Quote: Originally Posted by THRiLL KiLL awww... how cute.... look at the fashion accessory

Up yours!!!

Quote: Originally Posted by Fluffy Up yours!!!

dead puppies dead dead dead puppies arent much funn

Oh this one again....... can I please have some fun before he goes bye bye?
I promise to let Fu3lman in on the fun too for good measure. (if he wants)
Huh, can I......?

Wow Alpha, lots of interesting info there.

So, since most consumer level technology trickles down from the server grade, can we expect to see the ZFS file system implemented for desktops, or is it even necessary considering the inability of a single user to request that much archived data at one time (excepting you, of course)?

Oh, and Fluffy. My cat could take you.

I'm curious about the constant referring to the equipment as "my" and "mine". Alpha even lists his work equipment as his own in his sig. If I were to do that I imagine I have around 55 computers right now. If I were to refer to my companies equipment as my own, I would probably have enough computing power to rule the world. Bet my company could take his company.

I'm not even sure that he even owns a personal computer.

That really interesting alpha.

To the people that bash Solaris, it's probably the best OS for programming


One thing though, you still need a raid controller to hook up more HDD, so how does this make it cost effective?

well i find this info ironic, as i talk about buying a itanium board for my nice dual chips (thanks again twizted) and thinking of how to run them as a massive file server (over 5 TB) for my house as well as my grandparents nextdoor.

Thats a lotta geek talk i must say, but very interesting

EDIT: to coment on the captains houghts, he was saying torun ZFS on a normal say your PC with a few cheap sata 1 controllers, it would be cost effective easy file serving

Quote: Originally Posted by Fluffy Up yours!!!

im sure its your life dream to be a gerbil replacement...

Quote: Originally Posted by BigAkita I'm curious about the constant referring to the equipment as "my" and "mine". Alpha even lists his work equipment as his own in his sig. If I were to do that I imagine I have around 55 computers right now. If I were to refer to my companies equipment as my own, I would probably have enough computing power to rule the world. Bet my company could take his company. I'm not even sure that he even owns a personal computer.

See, I am not the only pne that thinks this guy is a tool.

Nice to see you back smack-tard. Are you going to avoid my questions this time around like a little beeyatch?
Last time we went 'round you all but avoided my picking apart of your "logic'.
This stuff may be cut and dry to you but there are some that find it interesting so quit p|ssing in their pot or you will get the smackdown you deserve. I am just keeping you around for a feeding toy at the moment........

WTF? I don't even know you. We went round?

Maybe this will jog your memory......

http://forums.pcapex.com/anything_go...e_hell_up.html

Quote: Originally Posted by Fluffy WTF? I don't even know you. We went round?

My Primate could take ya out to dinner and my Min Pin whould eat the rest.

I think I'm missing something here Alpha. You say that you got a minimum of 120mb write speed with one OS drive + 3dries on the raid controller. Then you said that by adding more drives you should be able to get the speed up yet with 16 drives you were getting max 76mb write.

Sorry if that doesn't make much sense but I'm really really tired right now.


As for you mr.fluffy...I don't know you(nor do i feel a need to)...but you sound sound like an asshole so....I shall call you an asshole.

Cheers

Quote: Originally Posted by THRiLL KiLL im sure its your life dream to be a gerbil replacement...

So does that mean you are gonna take the gerbil out first?

Oh damn, here I was all set for a witty reply when I accidentally hit the BAN key on my keyboard.
What a shame................

Edit: Haha, the tard is trying to reply, muhahahahahahaha!!!1111WTFBBQRIBZ

Now back to your regularly scheduled thread.

Quote: Originally Posted by godfoot Oh damn, here I was all set for a witty reply when I accidentally hit the BAN key on my keyboard. What a shame................ Edit: Haha, the tard is trying to reply, muhahahahahahaha!!!1111WTFBBQRIBZ

lol I have no Idea what just happend but it sure was funny to watch.

EDIT

Putting that 'godfoot' down haha