By continuing to use this site, you agree to our use of cookies. Find out more
Forum sponsored by:
Forum sponsored by Forum House Ad Zone

Archiving old data

All Topics | Latest Posts

Search for:  in Thread Title in  
Danny M2Z25/12/2019 15:20:35
avatar
963 forum posts
2 photos

As mentioned in the Win 10 thread, I just built a new (old) Win XP PC, partly for the fun of it but mainly to archive a few hundred CD's and floppy discs to an SSD drive for posterity but also to de-clutter a book rack.

Some of this data is important to me such as my first ever email (1996) - trivial to most people and also my first pcb design from 1979.

An old collection of computer virii (about 174,600) with their disassembly notes and fingerprints might be of value to historians one day. The original 'marijuana/paki' virus still lives on a 5,25 floppy from 1989.

So my problem is: How does one archive such data to last such that future generations might access what we were up to in our time?

Floppy disc's and CD's are already obsolete, SSD's are maybe headed the same way and as for the 'cloud', this just means storing one's data on somebody else's computer (not really comfortable with that)!

Too much info to chisel into stone, still have the old DOS 3 boxes to interrogate and then my 1983 Z-80 which still manages to boot up and talk through it's serial port.

So what's the best way to archive all this data so that it it might be readable to future generations?

* Danny M *

peak425/12/2019 15:44:19
avatar
1678 forum posts
179 photos

I guess two separate hard drives from different manufacturers, stored with care in two different places.
As the new technology comes along to replace HDDs, transfer the data onto that as well.
I can remember a friend of mine storing stuff on video disks, and then a few years later couldn't retrieve it, a bit like Betamax etc.

Bill

Enough!25/12/2019 15:59:56
1719 forum posts
1 photos
Posted by Danny M2Z on 25/12/2019 15:20:35:

So my problem is: How does one archive such data to last such that future generations might access what we were up to in our time?

Floppy disc's and CD's are already obsolete, SSD's are maybe headed the same way




... and none of them are really "archival" anyway. Least of all CDR/DVR because of the dye-layers.

 

There are, I believe, new forms of CD that don't use dye layers and are considered archival. Whether they are available commercially to the general public (and indeed, whether they require special writers) I don't know.

It's a good question though. One, indirect, way to make something readable to future generations might be as simple as uploading it to the internet. I think there are sites for that.

 

(Edit) Like my Uncle Bill said "Google first, reply second" (well he didn't actually - he way pre-dated the Internet - but it's the sort of thing he would have said). This is interesting.

Edited By Bandersnatch on 25/12/2019 16:05:26

Hacksaw25/12/2019 16:21:38
463 forum posts
199 photos

I can see a hell of a lot of personal photos / images which will one day be historically important / interesting to someone just lost and forgotten...sad

asimpleparson25/12/2019 17:24:08
23 forum posts
5 photos

Perhaps look at what the Government uses to store data. It's usually a format that's going to last a long time. Both Oxford and Cambridge house vast archives of data, it may be worth getting in touch.

Frances IoM25/12/2019 18:46:58
1250 forum posts
28 photos
I regularly handle 450 year old rag paper documents - still as readable as the day they were written - catch is their information density is somewhat low but would 450 old documents describing early computer viruses specific to long dead O/S be of any interest.
HOWARDT25/12/2019 19:52:00
902 forum posts
39 photos

In days long gone gold coloured CDs were used for archiving as they were less susceptible to degradation by light. But whatever you use it should be checked at least yearly to ensure there is no data loss or failure of the media. I started using floppy disks back in 1990 on a daily basis and can say have never lost any data stored for archive. Also have only had two drive failures in PCs in daily use. What ever you use you should keep three backups and not keeping them in the same place.

JA25/12/2019 20:23:14
avatar
1346 forum posts
80 photos

Earlier this year I went to a morning meeting held by the Gloucestershire County Archives for local history groups. It was really a sales talk but subject of digital archiving was included. It appears to be a very, very serious problem with formats changing every few years and unstable storage devices.

If you want to take this further I would suggest you contact your local county archives for advice. They may even welcome the query as a change from looking at parish records.

Today we can read day to day letters written by the Romans on clay tablets. In 2000 years time will anyone be able to read anything on an i-pad?

JA

John Haine25/12/2019 22:17:41
4639 forum posts
273 photos

Technically, the best solution is cloud storage. Any sensibly managed service would regularly back up data to whatever storage technology is best from time to time. That's the magic of digital data that as long as it is stored with some redundancy it can be exactly reconstituted and re-stored. The problem is having a provider that will be around for as long as there could be people who want to see the data. One solution might be a sort of "Long Now" movement that constructs and runs a civilisational repository.

Nick Clarke 325/12/2019 22:36:27
avatar
1397 forum posts
61 photos

+1 for cloud storage

A separate issue for me as a photographer is the lack of a permanent record of digital images. On the trashier channels of cable TV one sees Hitler, Churchill etc in otherwise insignificant photos. With digital these will be deleted.

How will historians of the future manage?

HOWARDT25/12/2019 22:59:49
902 forum posts
39 photos

Remember also it is not only the media which may not be be able to be read it is also the file format. Some software like cad has changed file format during its lifetime which if not updated the file format becomes unsupported.

Andrew Evans25/12/2019 23:28:17
344 forum posts
8 photos

It's an interesting and complex topic. I used to work on the multimillion £ Digital Preservation project at the British Library. As part of that we built a system that automatically copied each digital file to 3 remote storage sites, it automatically checked each file on a regular basis to ensure it hadn't changed and if it had changed recovered it from a remote site. Files on hard drives can spontaneously have bits changed and of course there can be bugs and physical problems like fires. We archived digitised books and newspapers as well as websites and other 'born digital' files such as the huge master files for Ordnance Survey maps. When one disk started to fail it was swapped for a new one and data automatically copied over. I am sure that project has evolved since then.

Files stored on older CDs can be unreadable as the surface degrades, generally they are poor way to store data long term.

90% of the data collected during the Apollo program is now unreadable - the data format was never recorded but the data itself is fine.

For files created with a specific program you need to store the program with the file or instructions on how to read the data - no point in having a well preserved file if you have no idea how to read it.

The BBC had a project in 1981 called the Doomesday Project to record census data and general information about life in Britain - the data was stored on Laser Disks. I understand there is only a single, working Laser Disk player in existence now that is capable of reading this and this is used as a classic example if digital obsolescence. So it can be a hardware problem as well as a software one.

Data loss isn't always accidental, governments can try to change data - look how Trotsky was erased from photos after he fell out of favour in the Soviet Union. So a good preservation strategy has to cope with deliberate data tampering as well.

Brian G26/12/2019 09:38:40
839 forum posts
37 photos
Posted by Danny M2Z on 25/12/2019 15:20:35:

...

Floppy disc's and CD's are already obsolete, ...

* Danny M *

Perhaps your statement is untrue with regard to CDs as there are still large numbers of CD/DVD/BD disks sold, and that, so far at least, all 120mm optical disk readers are backward compatible, and that pressed (rather than burned) optical disks have a long lifespan. It may be there is so much material on these formats that cash-strapped future historians would be more likely to preserve or re-create hardware to read even fragments of material from these than any other current data format.

Whilst conventional writeable optical disks have a very short lifespan, M-DISC compatible DVD writers are available for under £15, and Blu-Ray for around £50. The disks can be read on most DVD/Blu-Ray readers have a claimed readable life of "up to" 1000 years, so it might be reasonable to expect a century or two under less than ideal conditions. Unfortunately the disks themselves are relatively expensive compared to other media (unless you compare the total cost of ownership over the next few hundred years.

I fitted one M-DISK and one LightScribe DVD writer to my PC, so I have perhaps achieved a 50% success rate in choosing drives

Brian G

John Haine26/12/2019 09:41:40
4639 forum posts
273 photos
Posted by Andrew Evans on 25/12/2019 23:28:17:

.....

The BBC had a project in 1981 called the Doomesday Project to record census data and general information about life in Britain - the data was stored on Laser Disks. I understand there is only a single, working Laser Disk player in existence now that is capable of reading this and this is used as a classic example if digital obsolescence. So it can be a hardware problem as well as a software one.

.....

A great example of a failure to think the problem through. When laser disks became obsolescent there should have been a plan in place to transfer the data to the next mass storage technology - preferably in the cloud. As always, this isn't a technology problem, it's a process and system failure. I've recently been reading up on Bletchley Park again, and what comes through from a couple of the better books is that, whilst the technology was key, it would not have worked without a carefully constructed and maintained process to industrialise codebreaking.

JA26/12/2019 10:05:42
avatar
1346 forum posts
80 photos

During the mid 1960s NASA spent an awful lot of money photographing the other side of the moon. In about 1995 someone at NASA actually wanted to have a look at the pictures. They found the tapes easily enough but the viewing equipment had gone. Five machines were finally found as scrap at Edwards Air Force Base.

JA

Nick Clarke 326/12/2019 11:34:55
avatar
1397 forum posts
61 photos
Posted by Andrew Evans on 25/12/2019 23:28:17:

The BBC had a project in 1981 called the Domesday Project to record census data and general information about life in Britain - the data was stored on Laser Disks. I understand there is only a single, working Laser Disk player in existence now that is capable of reading this and this is used as a classic example if digital obsolescence. So it can be a hardware problem as well as a software one.

The Wikipedia article on the BBC Doomesday Project makes interesting reading, not just to show the issues surrounding the preservation and access to this information, but also the copyright nightmare associated with trying to do so.

BBC Domesday Project on Wikipedia

Ed Duffner26/12/2019 11:47:11
834 forum posts
94 photos

I still use floppy disks on my Amiga computers. yes

For the virii I would get rid of them, just my honest opinion. I'm sure there are copies of them archived at the companies who write software to tackle those. ...and I doubt you'd be unable to store them in the cloud (a remote disk somewhere on the internet) because of the risk and possible breach of usage agreements.

At HP our DAT tape backups were only guaranteed for 7 years by our IT Dept..

You could use an optical disk, but I've had a heck of a job getting my Blu-ray DVD writers to read disks on windows 10, but are ok on Windows 7. I have to go into device manager, disable the drive, then re-enable it so the OS can then read from it.

USB Thumb drives could also be an alternative but how long will the interface connector be available into the future?

Ed.

Perko726/12/2019 11:58:10
422 forum posts
33 photos

Not a big fan of 'cloud' storage. My previous employer engaged an outside organisation to archive all their records. For some years now they have been progressively converting all their paper filing to digital archives in-house but the task was getting too big. As a consulting engineering firm they had to make sure design information on each project was available for the life of the project in case of any future liability claims. The statue of limitations in most Australian States can be extended where civil suits pertain to engineering designs. Seems like their highly reputable international data storage contractor lost a big chunk of stuff pertaining to some quite notable projects from the last 10-15 years. They are not happy, and neither are their insurers as they now have no evidence to build a defence against any potential claims.

Enough!26/12/2019 18:23:06
1719 forum posts
1 photos
Posted by Nick Clarke 3 on 26/12/2019 11:34:55:

The Wikipedia article on the BBC Doomesday Project makes interesting reading ....

.... although correctly called the "Domesday Project" devil

Nick Clarke 326/12/2019 19:00:56
avatar
1397 forum posts
61 photos

A fair cop.

Spelt it wrong three times but only corrected it twice.

But as Meatloaf sang 'Two out of three ain't bad'

All Topics | Latest Posts

Please login to post a reply.

Magazine Locator

Want the latest issue of Model Engineer or Model Engineers' Workshop? Use our magazine locator links to find your nearest stockist!

Find Model Engineer & Model Engineers' Workshop

Latest Forum Posts
Support Our Partners
Eccentric July 5 2018
cowells
Dreweatts
Rapid RC
Eccentric Engineering
Subscription Offer

Latest "For Sale" Ads
Latest "Wanted" Ads
Get In Touch!

Do you want to contact the Model Engineer and Model Engineers' Workshop team?

You can contact us by phone, mail or email about the magazines including becoming a contributor, submitting reader's letters or making queries about articles. You can also get in touch about this website, advertising or other general issues.

Click THIS LINK for full contact details.

For subscription issues please see THIS LINK.

Digital Back Issues

Social Media online

'Like' us on Facebook
Follow us on Facebook

Follow us on Twitter
 Twitter Logo

Pin us on Pinterest