Drive Pooling - Multiple Drives in 1 logical drive - 1 BIG C DRIVE!

in #computer7 years ago (edited)


Mediasonic - 8 Bay E-sata/ USB3.0 Hard Drive Enclosure   -  $269.00 on AMZ



Stablebit Drivepool
https://stablebit.com/DrivePool

Below, we see a screen shot of the User Interface (UI) for the DrivePool app (with only 4 drives pooled):


I have been experimenting with Drive Pooling software applications. 

It makes so much sense, yet, most people have never heard of drive pooling!

Consider an average computer user, over 10 years time, probably collects 10 hard drives in this time; if they purchase 1 new hard drive every year.. a backup drive, a new computer with a hard drive, etc.  In my experience, any serious computer user can easily assemble a large collection of drives over time. 

It is not obvious to most computer users, how to organize and arrange such a "stable" of drives collected haphazardly over a long period of time, such that it is easy to access years of data collected over time.. and keep important files where they can be searched for, and reliably backed up as well.

Yet, how to manage 10 hard drives in a 10 year period? 

In my own case, I would end up with 16 hard drives 10 years time. What to do?

After researching this situation; it is most easy to configure these drives as a storage "Drive POOL".  

In other words, 1 large 'C drive' disk array!

My experience: 

I have 16 drives, connected to 1 computer; which I use as a data and entertainment server; they form a "pool" of drives configured to be seen as 1 logical drive, under 1 folder. A 20 terabyte storage array on my desktop computer, that I share on my LAN.

Stablebit Drivepool indexes all the data on every one of these drives, to form a single common file manager. This "Pooled folder" then contains all contents of all 16 drives, and can easily be backed up, and searched, as a single folder.

If you think about it, it makes keeping your data over decades so much easier than it would be otherwise: 

- Taxes go in the tax folder

- Movies in the movies folder

- Music in the music folder. 

Same drive pool folder, across 16 drives, indexed as 1 folder. No matter which drive the movie file lives on, it shows up when I open the movies folder of the drive pool.

When picking out a storage solution, I had several requirements I felt were absolutely important, and greatly reduced my available alternatives:

1) All files currently on the drives I wish to include in the POOL, will not have to be relocated to other drives. They can stay where they are, and be included.

2) All drives do not need to be formatted or prepared to be part of the POOL in any way. They can simply be NTFS drives for Windows, and stay as they are.

3) All files can still be read from the drive, even if it is taken out of the POOL! Files are in no special format, and there is no special striping required, etc.

4) Some kind of backup solution would need to be included, such that it would manage my backups for the pool automatically once configured as I desired.

5) Must scale to accept 16 hard drives, minimum, as that is how many hard drives I had enclosure cabinets for.

6) Must allow for easy search across all drives in the POOL, to find files quickly, with a single search for a single file name.

Keep in mind, drive pooling software is working at the application level. This frees the software to do many things that would be difficult to do if it were built into the operating system itself, or an extension of the operating system. It also allows the OS to simply see the drives as mounted file systems, and the pooling software handles all the combined indexing at the application level, but then mounts that indexed folder, as a shared folder on the operating system. (Confused yet? ;-)

In other words, if you want to turn off the drive pool for any reason and access folders directly in the operating system, you can simply quite the pooling service running in Services, and the pool will be turned off.

I also had features I was willing to give up if necessary:

a) No special performance was required. If i could read files faster by keeping 2 copies (backups) then so be it, but not a requirement.

b) No RAID specification or criteria need be applied to my storage pool solution.

c) Storage could exist on my existing Windows Home Theater PC, and need not be stand-alone, or run on Linux, or run as a stand alone NAS. A simple PC running Windows, sharing files would be a minimum requirement I could deal with.

The software package I eventually found is called Stablebit Drivepool. 

Unfortunately, it runs on MS Windows only.. but for me, that has been a viable trade-off. I am able to read these shared folders on a Mac for example, since the MacOS includes the ability to read NTFS shared folders.

My server setup: 

- Simple desktop computer with slots for 8 drives.

- Extension E-sata cabinet - 8 Bays

- 1 SSD to boot the entire system from

- 1 - 2 Bay USB 3.0 drive bay (for emergency disk recovery)

- 1 - E-Sata card to plug the E-sata cabinet into

- 1 - Sata card with 4 extra slots to extend my 4 Sata slots already on the motherboard

- 1 - 4K capable HDMI out graphics card - since my motherboard does not support 4K video


Sort:  

Apologies; this post was accidentally posted by a bad keystroke. I had no idea it had gone live yet; and by the time I was ready to post it (thought I had lost the original) I started to see people commenting on it already - OOPS!

Please read it again if you have time; it's been updated extensively..

I never beg for this ever, but in this case, please Resteem!

Resteemed!

Thanks for sharing this. I'm going to check Drive pool out. Does the app stay running all of the time? if so, does it use a lot of resources?

Thank you again!

I does not use a lot of resources, for a desktop machine in 2017. But keep in mind, it's really for a desktop, so if you have 8gb of memory, that should be plenty. And it won't keep your machine busy, except at night when it's backing up files. You can set it up to back up at 4am, for example.

Oh nice, I have 32 GB of ram som I should be okay. I have a few video cards mining 24 hours a day in my PC but if 8 GB can handle it, I'll be golden. Thanks for the info, i'm headed to check it out now.

Thats a nice development you have been creating :)

wow 20 terabyte, thats some huge drive.

I never realized this was possible. Thanks for the post! I've been stacking HD's for the last 6 years and didn't think I'd be able to use them in this capacity. I JUST purchased a 4tb expansion drive to combine them all onto one device, and that used up almost half the EDD.

I'm following your lead on this one...AND it looks nice and sleek.

Thanks for the compliment. I don't think many people know this tech exists!

And I don't think many people saw this article because it got launched early..
Please resteem!

Upvoted AND Resteemed...fer sure!

You should try cloud storage

Do u know how much 20TB of cloud storage would cost?
40TB?

I agree the cloud has a smaller foot print but I just sling the 2 boxes under my TV..

Why u need 40 tb?

I don't need 40TB now, but at one time, before I knew more about streaming, I did.
I'm a librarian, I collect books, pictures, movies, porn, and apps... not necessarily in that order.

Lately, I've been using Kodi, and don't need so much storage as in years previous.. ;-)

I think has gone now for keeping aps anf porn in disks now, time is now for streaming

I agree most people don't need 20TB of storage.
What most people need, is a way to organize 5-10 drives they collected over time, and be able to search them for important files.

It looks like a fridge for a non-technical person! Thanks for sharing

@surfyogi
Sir do you upvote others post or comment also?
Because I need it ! Hehe 🙈

This is a example of technology at its peak ....nice man

Upvoted and resteemed man

You should post often. Your comments are so professional and your posts 1000 PRO level :D

upvote e and resteem my post plzz sir

i like all your follow itry to upvote ur post but me vote is power i like to follow me againe thnx