Storage Solutions RAID advice required

[blueren]

Hi All,

I have a ton of photographs which I wish to move to a more fail safe medium. I've thought of moving them to the cloud (Google photos) but am also wanting to invest in decent raid 1 setup and move them there. Kindly advice on what I essentially require in order to achieve this.

What I currently have:

1. Gigabyte GA-Z97-D3H
2. Samsung SSD (OS)
3. An old 640GB WD HDD (SATA)
4. An even older 120GB Seagate IDE drive (Which I plan to decommission)

What I plan to buy:

1. Two 1 TB storage drives (Please suggest which model would be best)

Things to note -

This will be purely for storage and backup purposes only. No program installations, no nothing. Just data backup.
How do I go about with the whole process?

Thanks in advance.

 

[booo]

if you just have two hdds, then raid1(mirroring) is the best option. it ensures safety of the data in case of one hdd failure. just buy wd greens and setup raid1 on the mobo if it supports. its better offloading the raid to mobo rather than setting up a software raid. if you want to get anymore adventurous than two hdds and raid1, I suggest lsi megaraid card which supports almost all the types that you will ever require including hotspares.

 

[blueren]

if you just have two hdds, then raid1(mirroring) is the best option. it ensures safety of the data in case of one hdd failure. just buy wd greens and setup raid1 on the mobo if it supports. its better offloading the raid to mobo rather than setting up a software raid. if you want to get anymore adventurous than two hdds and raid1, I suggest lsi megaraid card which supports almost all the types that you will ever require including hotspares.

Raid 1 is what I need. Is there any specific model I can buy off amazon / fk? Also, Its mentioned in the other thread that green has a higher failure rate? Is this sporadic? How about the purple? Will that do the job as well given that this will be only for backup?

 

[booo]

frankly I dont have much idea about hdds actually, I said green because it consumes less power and I've been using it for a while. @Crazy_Eddy and others should know about the hdds that are trending now.

but if I were you and I want to securely store my photographs, I would setup a nas box with few hdds and hotspares. basically if you dont know a hotspare, the raid system keeps a hdd on spare and if finds that a hdd is failing, it would rebuild the data on that spare and flags the bad hdd.

hotspare because once you loose your photos you will never get them back.

 

[Lincon_WD]

Raid 1 is what I need. Is there any specific model I can buy off amazon / fk? Also, Its mentioned in the other thread that green has a higher failure rate? Is this sporadic? How about the purple? Will that do the job as well given that this will be only for backup?

Hi @blueren,

I'd like to assist here, both WD Green and WD Purple are not recommended for RAID setup.

The WD Green drive had a high failure rate due to RAID setup. The platform or technology of the WD Green is not recommended to use the drive in a RAID environment. It's only recommended to use for storage drive, and not as a performance drive.

For your information, any WD internal drive can be used as a storage and boot drive. In WD, there are different type of drives represented by different colors. The WD Purple drive highly recommended to use for Video Surveillance purpose and WD Red drive is designed for NAS environments and recommended for RAIDs, but if you use it as a normal drive, it will function like a normal hard drive too.

Hope it helps.

 

[vivek.krishnan]

What you need is the WD My Book Duo or something with two HDDs. It will allow you to configure the internal HDDs in RAID 1.

However, if you are going to keep adding more data, then it is best to setup a NAS yourself with any entry level Pentium/i3 desktop. This will allow you to keep expanding the storage. For the OS, you can use Windows Server 2012/Windows 8 with Storage Space or Linux/FreeBSD based OS with software raid.

 

[blueren]

Hi @blueren,

I'd like to assist here, both WD Green and WD Purple are not recommended for RAID setup.

The WD Green drive had a high failure rate due to RAID setup. The platform or technology of the WD Green is not recommended to use the drive in a RAID environment. It's only recommended to use for storage drive, and not as a performance drive.

For your information, any WD internal drive can be used as a storage and boot drive. In WD, there are different type of drives represented by different colors. The WD Purple drive highly recommended to use for Video Surveillance purpose and WD Red drive is designed for NAS environments and recommended for RAIDs, but if you use it as a normal drive, it will function like a normal hard drive too.

Hope it helps.

Thanks @Lincon_WD I'd never given any thought about WD red. In fact, I didn't even know a red variant existed! I'll see how best this will work out and try to find myself a good deal online if possible. I see that it's about 5.3k on flipkart at the moment.
My usage will be mostly only to dump data there. No other purpose.

What you need is the WD My Book Duo or something with two HDDs. It will allow you to configure the internal HDDs in RAID 1.

However, if you are going to keep adding more data, then it is best to setup a NAS yourself with any entry level Pentium/i3 desktop. This will allow you to keep expanding the storage. For the OS, you can use Windows Server 2012/Windows 8 with Storage Space or Linux/FreeBSD based OS with software raid.

I'm wondering if what you've suggested is an overkill for me. I'm looking for a humble backup. I don't mind doing manual work from time to time. Right now, I have a backup on one of my desktop drives, and another copy on my trusty oooold ext storage drive which I use ONLY for this purpose. I use syncToy to maintain a sync btwn the drives. I've haven't really done research on finding better means to keep the folders and files synced, any inputs on this is most welcome!

At the end of the day, I can have raid 1 (1tb each) And a fail-safe drive like you'd suggested earlier. However, how do I actually confirm if my Motherboard can support this kind of a setup? Also, along the same though, would it be possible to turn off these drives when I really don't need them? I suppose that's where the NAS setup comes into play, right?

 

[vivek.krishnan]

I'm wondering if what you've suggested is an overkill for me. I'm looking for a humble backup. I don't mind doing manual work from time to time. Right now, I have a backup on one of my desktop drives, and another copy on my trusty oooold ext storage drive which I use ONLY for this purpose. I use syncToy to maintain a sync btwn the drives. I've haven't really done research on finding better means to keep the folders and files synced, any inputs on this is most welcome!

At the end of the day, I can have raid 1 (1tb each) And a fail-safe drive like you'd suggested earlier. However, how do I actually confirm if my Motherboard can support this kind of a setup? Also, along the same though, would it be possible to turn off these drives when I really don't need them? I suppose that's where the NAS setup comes into play, right?

It could be an overkill from your end. I was assuming you are a photgrapher, hence suggested.

If your mobo has an option of RAID, apart from IDE/AHCI, you have RAID support. In the newer Intel chips, its there only on the B85 and above. Dunno about the older ones.

I would suggest a NAS anyday. It may be more expensive now, but it can also do downloads and other stuff like DLNA, which make it a VFM device.

 

[blueren]

It could be an overkill from your end. I was assuming you are a photgrapher, hence suggested.

If your mobo has an option of RAID, apart from IDE/AHCI, you have RAID support. In the newer Intel chips, its there only on the B85 and above. Dunno about the older ones.

I would suggest a NAS anyday. It may be more expensive now, but it can also do downloads and other stuff like DLNA, which make it a VFM device.

I do photography as a hobby, humbly. I'd much rather invest in good lenses at this point in time than a nas setup, sure enough I'll try to find the time and resources for a good setup in the near future. I'd just like to address my current issue over having spares Also, RAID 5 = RAID 1 + hot spare?

Looks like my mobo supports RAID 0, RAID 1, RAID 5 and RAID 10
http://www.directron.com/gaz97d3h.html

 

[booo]

Raid5 is not raid1+hotspare. Raid5 is stripe with parity.
Raid0 is striping
Raid1 is mirroring
Raid 5 is stripe with parity
Raid 10 is raid 1+0
There are few more complex types like 50 and 60 and more. Phew!!!

 

[vivek.krishnan]

If you plan to use the onboard RAID feature on your mobo, please stick to RAID 1. RAID 5 is usually not that great. RAID 10 would be better, but you need 4 identical drives.

You have two types of RAID - standard and nested (combining two or more standard RAID). Its better to read up on Wikipedia.

I am not getting into software RAID such as ZFS.

 

[blueren]

Along the same lines, what kind of a budget would I require (And the components) if I were to have a NAS set up? Also, many folks offline have suggested that HGST disks for the best reliability out there. What are your thoughts?

 

[vivek.krishnan]

For an Intel based solution, You need a B85 chipset for motherboard RAID support. Any CPU will do. Some mobos have the support even in H61 based boards, but can't confirm. Cost for components [mobo+CPU] should be under 8-10K.

If you want software OS RAID, Windows 8 has Storage Spaces which should do a similar job, with additional CPU resources utilized. Dunno about cost.

Another option is a PCI RAID card - cheap.

 

[Mr.J]

Correct me if I'm wrong but isn't RAID considered risky for backups? Especially if you buy HDDs from same batch?

 

[booo]

^Depends on what kind of SLAs and uptime you want and the budget ofcourse. for more info google RPO and RTO.

 

[vivek.krishnan]

If you buy HDDs from the same batch, then its risky, but then just buy one Seagate and WD

If you consider it that way, all my servers are vulnerable since we buy and configure the server [memory, HDDs, CPU, software] in one shot

 

[Arv90]

I doubt whether different types of HDD will be really useful.
As far as I know, HDDs have to be identical in terms of RPM, size,etc. to enhance performance.
Matched or unmatched drives for RAID arrays?

Points to remember for a RAID 1:
The SLOWER of the drives... will dictate the overall speed.
The SMALLER of the drives... will dictate the overall size.

I can see that your motherboard can support RAID 0, RAID 1, RAID 5, and RAID 10
http://www.gigabyte.com/products/product-page.aspx?pid=4961#sp
Chek for the CD that you got with the Mobo to configure.

If you are looking for low budget, I would suggest RAID 5.
We have tried removing 2 out of 5 HDD's from a live server to check the HW RAID we configured and the server still working fine.
Maybe it was due to H/W raid on an IBM P series server

And if more budget, there is nothing like RAID 1.

EDIT: Forgot to mention, RAID 5 requires a min 3 nos of disk.You can go for 3 cheaper 250/500 gbs
3*500 GBs will give you 1 TB space on RAID 5 whereas 2*1 TB will give you 1 TB on RAID 1

[DOUBLEPOST=1436768044][/DOUBLEPOST]After going through your Mobo maual:
http://www.megekko.nl/productpdf/00162304-manual.pdf

 

[blueren]

Correct me if I'm wrong but isn't RAID considered risky for backups? Especially if you buy HDDs from same batch?

RAID is more so for resiliency than backup. Resiliency and back are often confused, although they come close in certain aspects to meaning the same thing.

I doubt whether different types of HDD will be really useful.
As far as I know, HDDs have to be identical in terms of RPM, size,etc. to enhance performance.
Matched or unmatched drives for RAID arrays?

I can see that your motherboard can support RAID 0, RAID 1, RAID 5, and RAID 10
http://www.gigabyte.com/products/product-page.aspx?pid=4961#sp
Chek for the CD that you got with the Mobo to configure.

If you are looking for low budget, I would suggest RAID 5.
We have tried removing 2 out of 5 HDD's from a live server to check the HW RAID we configured and the server still working fine.
Maybe it was due to H/W raid on an IBM P series server

And if more budget, there is nothing like RAID 1.

EDIT: Forgot to mention, RAID 5 requires a min 3 nos of disk.You can go for 3 cheaper 250/500 gbs
3*500 GBs will give you 1 TB space on RAID 5 whereas 2*1 TB will give you 1 TB on RAID 1

[DOUBLEPOST=1436768044][/DOUBLEPOST]After going through your Mobo maual:
http://www.megekko.nl/productpdf/00162304-manual.pdf
View attachment 58436
View attachment 58437

This is AWESOME mate! Thanks for taking some time off into digging this for me! I really appreciate it!



Is there a possibility to have some sort of raid mirror setup where in the mirroring happens asynchronously and not in real time? That way, 1 disk can be primary, the other backs up the primary in say, once a day / week / whenever. That provides more insulation against accidental deletion and also file system related issues. I'm just throwing around some ideas now. Any comments about this?

Thanks!

 

[Arv90]

Is there a possibility to have some sort of raid mirror setup where in the mirroring happens asynchronously and not in real time?

Nope buddy!!
I guess you got the concept wrong for RAID
Just have a look on the wiki article on RAID here :RAID

is a data storage virtualization technology that combines multiple disk drive components into a single logical unit for the purposes of data redundancy or performance improvement. Data is distributed across the drives in one of several ways, referred to as RAID levels, depending on the specific level of redundancy and performance required.

What I mean to say is for every bit that is changed on your HDD, the RAID makes a parity bit or a mirror,depending on the RAID type configured.
RAID is not something like an Extension drive that can be plugged in or out as and when desired.

RAID is virtualization concept.
Your OS will see only 1 TB of HDD space on a 3*500GB RAID 5 setup.
The OS will not even detect 3 nos. of HDD connected.
It will see a logical 1 TB hard disk connected

If am right, what you have in your mind is the HOT SPARE concept : Hot spare

A hot spare disk is a disk or group of disks used to automatically or manually, depending upon the hot spare policy, replace a failing or failed disk in a RAID configuration. The hot spare disk reduces the mean time to recovery (MTTR) for the RAID redundancy group, thus reducing the probability of a second disk failure and the resultant data loss that would occur in any singly redundant RAID (e.g., RAID-1, RAID-5, RAID-10).

So whenever a Hard disk fails on your RAID, you can readily plug-in a replacement drive.
RAID will automatically reconstruct the data lost from the faulty hard disk.
But that will depend on the type of RAID configured and the level of redundancy of the RAID.
As i mentioned earlier, we went further to remove a 3rd HDD from the RAID 5 and the system crashed.
So here the redundancy level on the RAID was 2 HDD out of 5

I think you can make a hot spare as there is an option of Hot-plugging as per your Mobo Manual

I would say try installing an OS after configuring a RAID and try plugging out an HDD from it and check whether the system works
This is purely on experimental basis.Please do not blame me later

 

[blueren]

Nope buddy!!
I guess you got the concept wrong for RAID
Just have a look on the wiki article on RAID here :RAID

What I mean to say is for every bit that is changed on your HDD, the RAID makes a parity bit or a mirror,depending on the RAID type configured.
RAID is not something like an Extension drive that can be plugged in or out as and when desired.

RAID is virtualization concept.
Your OS will see only 1 TB of HDD space on a 3*500GB RAID 5 setup.
The OS will not even detect 3 nos. of HDD connected.
It will see a logical 1 TB hard disk connected

If am right, what you have in your mind is the HOT SPARE concept : Hot spare

So whenever a Hard disk fails on your RAID, you can readily plug-in a replacement drive.
RAID will automatically reconstruct the data lost from the faulty hard disk.
But that will depend on the type of RAID configured and the level of redundancy of the RAID.
As i mentioned earlier, we went further to remove a 3rd HDD from the RAID 5 and the system crashed.
So here the redundancy level on the RAID was 2 HDD out of 5

I think you can make a hot spare as there is an option of Hot-plugging as per your Mobo Manual
View attachment 58490
I would say try installing an OS after configuring a RAID and try plugging out an HDD from it and check whether the system works
This is purely on experimental basis.Please do not blame me later

The more I read about it the more scarier it gets when it comes to ways in which data loss can happnen Thanks for the deep insight. I'm scourging around for HGST drives at the moment