Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Serious Backup data-life strategy considerations?
12-26-2010, 10:26 AM
Post: #1
Question Serious Backup data-life strategy considerations?
Hi guys, i find myself in the following situation, and not too sure what is a good

way forward....


(HISTORY)
over the years, ive ended up with quite a bit of "data", on various storage media.


Data is quite diverse:
ranging from family photos, document scans, large multi-versioned multimedia

projects, family video conversions from cinefilm, games development content, anims,

backups, sound and graphic icons/librarys etc. (small files/larger files)
(even huge collections of doom2 wad files and game mods - which even now there are

massive websites doing well for that niche) Smile


Storage mediums:
a) on old ida/pata drives (still working but not sure for how long) (15yrs)
b) on older cd's (a good few hundred or so main ones) (10yrs)
c) on data dvds around 100 (last 5yrs) (mainly compressed data)
d) on cd/rw (since approx 10yrs)
e) on iomega zip disks (10-15yrs approx)

Total data probably about 1tb compressed to (estimate 1:10 ratio) say 10tb uncompressed).

(heh, have about 100 of these - back in the day, friends often would offer me cheap

Mac formatted ziptools disks (macs werent nowear as popular as pcs back then) for a

few quid.. this was excellent for me as pc formatted ones were ranging from about

£4 in massive bulk deals, to around £11-15 for a 1 off) - ended up getting them to

go through all the departments buying up old mac tools disks, and i simply re-formatted them and they were perfect LOL - got super deals

lord sugar would be proud of Big Grin


Problem:
while i probably have more data then i "actually" really need if i was to sift

through it all - i dont have super amounts of time to do that and just want to

ensure i can still access that data in the future.

ive heard (from Suite b i think) that the drives only last for about 20 years, and

on various websites that different cd dye materials can only last so long, before

they simply dont work anymore) -

buzzwords include short life strategy, metal azo, something cythaninene, long life

but dont use much, or use as much as you want but only for 5 years etc...


Solutions?:
on the zip disks, ive been using TIP from Steve Gibson (grc)
that seems to work quite well, at only about 1min per zip, taking around 2 hours

every 3 months or so.

on the drives, ive started (dumping) folder contents onto the drobo (for data

rather than installed program files etc)

on the cds/dvd (optical media), am not sure if i should just dump it all onto a

drobo, or if its quicker, (or more cost effective) to buy or rent a basic cd

duplicator, and to just buy a set of cheap blanks and basically clone all the old

ones onto new ones?

unless ive made a mistake, even for standard level brands, the cost per GigaByte

seems to still be cheaper than hard drives. eg 33p/gb hd, vs 10p/GB on a good dvd

say.
but they needing another whole shelf to put them on lol Sad


or is it better to pay a hosting company each month and get some sort of

protected/backed up area with encypted tunnel to be able to access it when needed

(not sure how much that wouldcost though)


Peter B Clements (of quickpar fame) had good info and tools (before his site got done by tukish hackers) with people using some kind of universal base version of an atapi cd reader (which would apparently always create the same byte-level cd extracted image data, upon every runtime), which was used to create a master parity quickpar set, and in the event of a scratched disk it would be used to read each and every possible bit of info, and then when used with a set of par2 data you could effectively re-create an iso image to then make a fixed new cd. (that was the theory but i cant seem to find more about it recently)

i dont know how many of you are in the same boat - but i guess as time goes on we're all collecting or generating more data each year that theres probably a lot of you in a similar boat? Smile

always good to hear your thoughts
thanks,

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
12-26-2010, 10:29 AM
Post: #2
RE: Serious Backup data-life strategy considerations?
"seems to still be cheaper than hard drives. eg 33p/gb hd, vs 10p/GB on a good dvd "

check your maths:

2,000GB WD drive from PC world 80 quid

thats 4p/GB

hard drives and now the cheapest and fastest form of storage (large scale) storage around

My Avatar is my Panther Chameleon named 'Leon.

I have a owned a Drobo v1, v2, DroboPro, Drobo 5D, DroboFS, Drobo 5N and DroboPro FS

I currently have a Windows 2016 Server with 8 x 8TB Seagate Archive Drives, 14 x 8TB WD Red and 3 x 8TB WD Gold
Find all posts by this user
Quote this message in a reply
12-26-2010, 06:05 PM
Post: #3
RE: Serious Backup data-life strategy considerations?
oops i sit corrected Smile

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
12-26-2010, 09:24 PM
Post: #4
RE: Serious Backup data-life strategy considerations?
btw i found a good tool which might compliment things..
its called "dvdisaster"

i can put my cds in, and check if all the sectores are readable, and if any show yellow or red, i'll dump them to drobo.
that should help provide a halfway solution for the time being

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
12-27-2010, 03:36 AM
Post: #5
RE: Serious Backup data-life strategy considerations?
Ahh, the good 'ole Zip disks and Click of Death... At one time I had all the Zip100 models - SCSI, Parallel, ZipPlus and ATAPI...

I can tell you from first-hand experience that things do have a lifetime.
CDs and DVDs - they'll wear by either light or humidity.

I went back to my parents' house to collect my "old" data on Zip, floppies (5.25" and 3.5"), CD-Rs and CD-RWs...
Most of the data was at least 10 years old.
A lot of the CDs were unreadable.
Some burned CDs had succumbed to what I guess is dye degradation
Others had visible oxidation "holes" because the edge seals had degraded (in the laserdisc world, this is called "laser rot")
The Zip disks were mostly OK, except for a few.
The floppies, the oldest of the types of data, were hit-or-miss. Granted, they were 10-15 years old at this point but some of them literally broke in the floppy drive - the floppy head wore a hole through a few.
Others just had trouble reading.

Guess I'll never get to relive my old ANSI animations in TheDraw... :\

So, end of semi-sad story...

For your optical media especially, I would look to imaging the discs - you can automate utilities like ImgBurn

dvdisaster looks interesting, but if I'm understanding it correctly, it creates discs with error-correction data, which is good, but to protect what you already have you'd have to read it and create new copies with dvdisaster anyway, which IMO would take longer than simply pulling the content to your Drobo?

Then again, this is from someone who has a CD/DVD autoloader...
But a Primera Composer isn't too much money nowadays on eBay, or you could spend a lot more and get a Nimbie type unit. I retrofitted a SCSI Composer with a SATA-USB bridge board and a new drive for under $200 (not including price of the Composer unit).

Where are you located? If you're nearby we could figure something out - at least for your optical discs.


For floppies I use WinImage (a lot of the self-extracting boot disks use WinImage) but I don't know of a floppy autoloader. Maybe you can find a crazy person with a USB Floppy RAID...
Seriously though, $20-$30 for a USB floppy drive isn't much of an investment if it's something you really want to keep.

This one's more of a diligence thing - stick it in, let it image, change discs, just make it a habit while you're on the computer or something. Same with Zip disks.

You can always mount and look through the images later. The point would be just to get your old data in a less-perishable state, not to sift through it all - unless you have more data than you can currently store.

Data-longevity's candle burns at both ends - one for the media readability, and the other for the availability of the hardware to read it. Finding a well-aligned 1.2MB 5.25-inch drive is... not fun.
Sorcerian did not read well. Still hoping for a Sierra On-Line "Everything" collection one day. I lost, then found, then re-lost my The Wizard & The Princess floppy long ago, in a galaxy far, far away. I can see it in my mind's eye still... nice black floppy with brown and beige label... and the snake early in the game. Darn snake! *sniff*

--Brandon | WHS2011+Drive Bender/2x Drobo v2/Drobo S G1/ Drobo S G2/Transporter
Drobo provides fault-tolerance, it's NOT a substitute for regular backups.
Drobo Best Practices - Official and Community-sourced.
Find all posts by this user
Quote this message in a reply
12-27-2010, 07:55 AM
Post: #6
RE: Serious Backup data-life strategy considerations?
thanks for the info Brandon, am checking out imgbrn.

dvdisaster appears to have 2 modes.. to create error correction ecc data and burn to the cd at the time of burning.
or, (if the cd is still readable) to create an ecc file somewhere else (like a drobo), which can then be used in conjunction with the cd in future. (eg say after 3 years, 100 sectors are playing up, you can use all the other secotors plus the ecc file to create another iso of your cd (which can then be mounted or reburned etc)


that memory stick and floppy raid looks cool. i wouldnt go that far myself - but i did used to have at least 3 Amiga 500 floppy drives all daisy chained, so that i wouldnt have to keep inserting disk x all the time Smile

that cd autoloader looks interesting but also serious money, i think what you said about imaging them could work.
(maybe better than copying and pasting all, as no files to virus scan or take more space) and i could probably enable WinXP ntfs compression on the iso folder, and simply mount them when needed etc.

i got the maths wrong earlier though as docchris pointed out (i blame the "Caliber beer" i tell you) Big Grin
it actually looks a lot more attractive now using data on drives, rather then shelfs of cd boxes etc

ive got 7x1.5tb WD15EADS's on the way - found a bargain £44
so i think i'll have to start going through the boxes at random to see if i can see which types of discs or brands seem to still be fully readable with all sectors etc, and then to start dumping things before laser rot kicks in Smile

(interestingly enough.. the older media seems to be more branded stuff like Sony and TDK/Verbatim/Memorex - its the newer media which is the cheaper unbranded ones - so maybe we do get what we pay for) Smile

Sorcerian looks nice btw: looks a bit like Super wonderboy in monsterland, and final fantasy 2/4 (heres a link for you)
http://www.hardcoregaming101.net/sorceri...http://www.hardcoregaming101.net/sorcerian/sor Smile

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
12-27-2010, 01:45 PM
Post: #7
RE: Serious Backup data-life strategy considerations?
Hmm... I like the "add ECC data to the disc" concept, but it still feels a little moot. It's kind of adding duct tape to the splintered pole rather than replacing it.

As I often tell customers - "You pay either way - in time or in money." Adding the ECC data would buy you more time, assuming the discs can still have sessions added. Just spreads the work out over a longer period of time, I guess. Better than nothing I guess, but I'd still go with the imaging+verify route if you can.

Seeing as your currency is in £, there's an ocean between us, so I won't be of much help on imaging your discs. I wouldn't be comfortable asking someone to ship their only copy of data overseas.

CD-Rs were funny in the beginning. I had more green TDK's that read fine compared to gold, silver and blue. YMMV, definitely. Had a couple of boxes of blanks that totally oxidized - they looked cloudy like they got wet, but the boxes were fine, so I know they didn't get flooded.

Thanks for the link - makes for a great trip down memory lane on a slow workday. Smile

--Brandon | WHS2011+Drive Bender/2x Drobo v2/Drobo S G1/ Drobo S G2/Transporter
Drobo provides fault-tolerance, it's NOT a substitute for regular backups.
Drobo Best Practices - Official and Community-sourced.
Find all posts by this user
Quote this message in a reply
12-27-2010, 06:07 PM (This post was last modified: 12-27-2010 06:09 PM by Paul.)
Post: #8
RE: Serious Backup data-life strategy considerations?
oh, thanks for the offer though Smile
memory lane, i guess thats what a lot of data is all about Smile

(btw that dvddisaster tool eec isnt to add a 2nd session to an existing cd, but rather to save off eec data to somewhere else.
eg an iso around 500meg generated about 60meg of eec data)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
12-28-2010, 12:42 PM (This post was last modified: 12-28-2010 12:58 PM by bhiga.)
Post: #9
RE: Serious Backup data-life strategy considerations?
Oooh... Well hmm... If it's saving ECC elsewhere, I guess it's just a matter of maintaining that database.

Very RAID 3 ish. Smile

Whatever way you choose, definitely keep us updated on your status - this is a problem others will run into soon as well.

...and I contribute a link back: Dungeon Master Encyclopaedia

--Brandon | WHS2011+Drive Bender/2x Drobo v2/Drobo S G1/ Drobo S G2/Transporter
Drobo provides fault-tolerance, it's NOT a substitute for regular backups.
Drobo Best Practices - Official and Community-sourced.
Find all posts by this user
Quote this message in a reply
12-29-2010, 06:24 PM
Post: #10
RE: Serious Backup data-life strategy considerations?
thanks for the link Smile

sure will let you know how things go on with time.
(so far i found a bunch of Verbatim Metal Azo's, which say 100 years life so will put those to the test) Smile

dvddisaster i think only works on data discs which are single session, but will save the eec files to drobo - and that should be safe enough.

(i have another thread about 2nd volumes which will be updating soon as i insert the 4th drive into my v1 drobo)

(btw i have XP home SP2, a Drobo v1 with 2x 1TB/2x 1.5TB WD greens, & a bkp Drobo v2 with the same + a DroboShare: unused)
& a DroboS v2 with 3xWD15EADS &2x1TB in DDR mode on win7, & a drobo5D (all usb)
  • btw i did a sustained (write) operation for about 6 hours, and got 13.2MB / sec ...objection? "sustained" :)
    (16.7MB/s on a v2 & 47-96MB/s drobo-s)
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: