Alexandria II, aka Foundation

[jor-el]

There are two events that set back western civilization centuries of progress.

The Destruction of the Library of Alexandria, Egypt. Now, this happened over a period of centuries, but the Library was attacked several times. The facility also would have been vulnerable to accidental fires. The Library's contents were after all composed of paper and pencil manuscripts, read by candle or torchlight. Now the material written in these texts could have been anything from navigational charts to pharmaceutical preparations to pornographic literature to quiche recipes. Who knows for sure? Anyway it's all gone now.

The Fall of The Roman Empire. The Romans were pretty far along, civil engineering wise. Roads, aqueducts, central heating and plumbing among other things. Once Constantinople fell to the Muslims it would be centuries before people had things like this again.

Civilization once again seems close to a fall. I think this time more people are aware of this. Some seem to want this to happen, hoping to press an agenda for civilization more to their liking. That, or they're just TROLLS that want to be fed.

RAther than simply allowing that to happen, some method of collecting and storing the sum total of our knowledge is required. Moreover, a means to disseminate the information is needed.

Libraries? Current libraries are as much internet cafes as book repositories. In a SHTF, they are as vulnerable to attack and depredation as Alexandria was. Books are made of paper, and they can still burn.

Schools? They do have libraries in them, usually. They also have computers as well.
Most importantly, they have teachers, not librarians. A teacher provides a human face to the data or processes and can explain the terms ideally to anyone, not dictate the data as some would.

So, what would it take to create the equivalent of a Foundation?
For those who don't read: http://en.wikipedia.org/wiki/Foundation_series

[jor-el]

I would appreciate serious discussion on this topic. Please do not troll.

If you genuinely want civilization to fail go somewhere like DU.

[F.I.B.]

What are you asking? What is the best way to store our knowledge and after something happens, what is the best way to redistribute it again?

This is a difficult question and probably depends on a lot of factors: How society will fall, Time frame, Who becomes the dominant power, How many survive etc etc

Are you asking me if i want society to fail?

If so no i don't, nor do i believe this is imminent.

[spacetyrant]

RAther than simply allowing that to happen, some method of collecting and storing the sum total of our knowledge is required. Moreover, a means to disseminate the information is needed.

Peer-to-peer networking would seem to be the obvious solution. The best way to preserve knowledge would be to decentralize it -- the more copies of something in circulation, the less likely the loss of a central repository would result in knowledge loss. As a delivery/distribution mechanism, bittorrent is hard to beat, and data stored in the network cloud is pretty robust given the numerous copies available. Long term storage would be a concern. Magnetic and optical media aren't really long term solutions, as the former is subject to mechanical failure and the latter tends to degrade over time, but the emergence of solid-state drives might be an answer (they're pretty tough), and as it becomes more and more common for new PCs and laptops to feature SSD for storage, data stored on this medium will be more likely to survive the PAW.

The question remains on how to access all of this data in the PAW; access to electronic media will obviously be much more challenging. My thought is that short of an all-out nuclear exchange (in which case EMP would have destroyed most of the electronically stored data anyway), our technical deevolution won't happen overnight. A procedure would need to be established within this "grace period" while we still have access to electronically stored data, to get as much of it transcribed as possible to book/scroll form. Any surviving printers and photocopiers would of course be pressed into service for this purpose.

My 0.02.

SpaceTyrant

[tilt]

I'm going to interpret this as asking what books (already in publication) should be included in an encyclopedic collection to encompass the most knowledge possible. I'll make a list below and add to it as ideas come to me...

-Gray's Anatomy

That's all I got for now... I'll add more this afternoon following work...

[jor-el]

Here's a good example. Here at the range there is a storage room full of training videos produced by my and other departments. Stuff from basic search procedures, CPL, techniques for basic marksmanship, knife/stick defense, tactics on patrol. Twenty years of collected material.

The problem is the storage conditions are far from ideal. The media is VHS. Many of the vids have no storage box and have collected dust. The room has flooded more than once over the years and most of the vids stored on or near the floor are now unviewable.

I have proposed transcribing the surviving vids to hard drives using video capture tech. What was I told?
1) The vids are old.
Funny, but guns have been around in roughly their current form for over a century. Semi-auto pistols have been around since 1896. Double action revolvers since the 1870s.
A vid showing how to properly reload and fire a revolver with just the left hand is still relevant as there are still a couple of thousand cops on my job who only have revolvers.
2) It would cost too much money.
A video capture card is like a hundred bucks. 500 GB hard drives are maybe 120 or so. They intend to purchase new DVDs.
3) No manpower. I offered to do the work myself, but apparently I cannot be spared for the six months it would take to do it.

Time, or an accident could wipe it all out. Hell, a delivery truck could strike the corner of the building and crush them all.

[jor-el]

RAther than simply allowing that to happen, some method of collecting and storing the sum total of our knowledge is required. Moreover, a means to disseminate the information is needed.

Peer-to-peer networking would seem to be the obvious solution. The best way to preserve knowledge would be to decentralize it -- the more copies of something in circulation, the less likely the loss of a central repository would result in knowledge loss. As a delivery/distribution mechanism, bittorrent is hard to beat, and data stored in the network cloud is pretty robust given the numerous copies available. Long term storage would be a concern. Magnetic and optical media aren't really long term solutions, as the former is subject to mechanical failure and the latter tends to degrade over time, but the emergence of solid-state drives might be an answer (they're pretty tough), and as it becomes more and more common for new PCs and laptops to feature SSD for storage, data stored on this medium will be more likely to survive the PAW.

The question remains on how to access all of this data in the PAW; access to electronic media will obviously be much more challenging. My thought is that short of an all-out nuclear exchange (in which case EMP would have destroyed most of the electronically stored data anyway), our technical deevolution won't happen overnight. A procedure would need to be established within this "grace period" while we still have access to electronically stored data, to get as much of it transcribed as possible to book/scroll form. Any surviving printers and photocopiers would of course be pressed into service for this purpose.

My 0.02.

SpaceTyrant

BTW, you know Demonoid is back up?

Books can be a part of the solution, but books can burn. For backup purposes they should also be scanned and composed into PDFs.

[thelight]

Couple of thoughts...

Multiple copies - Like the P2P idea. Except that in the PAW there will be no, or minimal, bandwidth to trade information. This limits copies to physical media (DVD, Tape, books, etc...). When hardware is scarce, like lack of computers or printing presses, this information will be worth more than its weight in gold.

Multiple formats - PDF is fairly ubiquitous in the digital world and since it is largely search-able it would be a good standard. However digital media has issues as well, like needing a computer to access it (and power). On that note I would keep paper copies (acid free paper, stored in secure location,etc) of both the information as well as the indices. What good is the information if you can't lay your mind on what you need in a timely manner.

Archival Grade Media - Touched on above, but really important. What good is accumulating the information if you put it on a standard DVDr? they tend to degrade after a few years in storage. I like MAM-A media as it's projected lifespan is around 300 years. Tape usually still wins for cost/performance/longevity comparisons. As for paper... I'll have to ask my librarian friends to comment. Any brands of acid free paper and toner for our survival printing needs? How about waterproof papers?

Disseminating Information - Since copying the information is an issue, disseminating it would be even more difficult. Having a library that people could come to is the only practical way of doing this. There is an alternative in Amateur Radio (a library type BBS with search capabilities, text only) but with great range comes real low bandwidth.

A Foundation - In the series, a visionary created a haven for knowledge so that an empire could be reborn after its dark age. Doing so in our world would take quite a bit of effort and presents many challenges. I see survival groups as part of the solution but more emphasis needs to be placed on knowledge, schooling, and retention. If we could create a network of information centers that are loosely linked (interlibrary loan style, perhaps?) this project would seem to be feasible.

On a side note:
Wikipedia Database Download - for all of your local mirroring needs.

Random thought:
"With great power comes great responsibility" && "Knowledge is power" = "With great knowledge comes great responsibility"?

[spacetyrant]

RAther than simply allowing that to happen, some method of collecting and storing the sum total of our knowledge is required. Moreover, a means to disseminate the information is needed.

Peer-to-peer networking would seem to be the obvious solution. The best way to preserve knowledge would be to decentralize it -- the more copies of something in circulation, the less likely the loss of a central repository would result in knowledge loss. As a delivery/distribution mechanism, bittorrent is hard to beat, and data stored in the network cloud is pretty robust given the numerous copies available. Long term storage would be a concern. Magnetic and optical media aren't really long term solutions, as the former is subject to mechanical failure and the latter tends to degrade over time, but the emergence of solid-state drives might be an answer (they're pretty tough), and as it becomes more and more common for new PCs and laptops to feature SSD for storage, data stored on this medium will be more likely to survive the PAW.

The question remains on how to access all of this data in the PAW; access to electronic media will obviously be much more challenging. My thought is that short of an all-out nuclear exchange (in which case EMP would have destroyed most of the electronically stored data anyway), our technical deevolution won't happen overnight. A procedure would need to be established within this "grace period" while we still have access to electronically stored data, to get as much of it transcribed as possible to book/scroll form. Any surviving printers and photocopiers would of course be pressed into service for this purpose.

My 0.02.

SpaceTyrant

BTW, you know Demonoid is back up?

Books can be a part of the solution, but books can burn. For backup purposes they should also be scanned and composed into PDFs.

My assumption has been that P2P networks would distribute archived data or books as PDF for all the beneficial reasons already associated with the format. If nothing else, there will always be a master copy of what has been saved in digital form, simply because that's the way in which it was initially acquired. In case it wasn't obvious, P2P networking is only a distribution method pre-PAW (in my opinion, the best means of getting as much data into as many hands as possible while we're still able to do so). The problem comes down to distribution after we enter the PAW. It obviously will depend on how much infrastructure actually remains, but I'm going off a worst case scenario in which reliable power is hard to come by and the internet is a thing of the past. Having a device that can read PDFs will be another challenge, one I'm thinking that can only be overcome by going old-school via written material. The only wide-area network operating at this point would be good old-fashioned sneakernet. While some surviving communities may be able to cobble together a WIFI LAN, not everyone would have access to it, and it would be much harder to share the few remaining devices that can read electronic media than it would be to pass a book around.

SpaceTyrant

[Hatch]

This issue (long term archival of data) has actually been vexing librarians, archivists, and futurists for quite some time now, even before we stepped into the digital age. I don't think even the experts have come up with a real solution to the problem. I've read a couple of articles recently about public records being lost due to degradation of storage media. Like stuff that was written to magnetic tape less than two decades ago.

Magnetic media degausses over time, optical media chemically degrades. I don't think we've come up with a data storage medium yet that would survive thousands of years of archival. You need something that can store many bits of stateful information, in a manner that will not allow state to change over time. Maybe solid gold punch cards?

Here's another issue though:

I did some work for a small town government a number of years back, helping them implement an upgrade of their antiquated computer systems. They had 100MB or so (I know, it doesn't sound like a lot these days) of documents written in WordStar or an old DOS version of WordPerfect, that they couldn't carry forward. The old software wouldn't run on the new OS, and the documents couldn't be read by MS Office. A lot of man hours were spent converting the old documents to ASCII and then importing the documents into MS Office files and re-applying minimal formatting where necessary. That technology gap was around 10 years. Imagine trying to read those electronic documents 100 years from now. Will ASCII exist as a standard in 100 years? Who knows?

So you've got the old format issue, Betamax vs. VHS vs. DVD. You have to have a means of structuring the data that will be readable by later generations, and the only approach that seems to even partially work is to structure it in the most basic, general format. For example, ASCII's been around since the 60's, and the last major revision was in the mid-80's or so. It's been a pretty stable format given the time in use. But then the problem is, going with such a basic format really limits the options of how data can be presented.

There's a way to get around this formatting issue, but you sacrifice portability of the data. You could create a library of digital information, readable only by a dedicated terminal. The terminal is designed to decode and present the stored information in a human-readable form. Then the only issue becomes creating a user interface to the terminal that could be understandable to later generations. But then you have to go to the terminal to access the information stored in the library.

On the longer term, you've also got the language issue. Remember that had it not been for the Rosetta Stone, a couple of Egyptian languages might have been lost to antiquity in the span of only a couple thousand years. The designers of long-term nuclear waste storage facilities face similar challenges. How do you communicate "don't dig here or it will kill you" to a person 10,000 years in future? English as a language may be lost to antiquity. The best solution they have found is pictographs, and even then there is much debate over exactly what to draw.

In the case of the nuclear waste storage facility, there's also the issue of how to mark the location to withstand 10,000 years of topography changes. The same challenge might present itself with a grand repository of knowledge. You could assemble all the combined knowledge of the human race throughout history and store it in a Grand Library. But if it gets buried under the changes of time and lies lost in the earth until the Sun explodes, it doesn't serve any real purpose.

From a library perspective, you've also got the problem of cataloging. What use is all the knowledge of man, if you can't find the specific knowledge you are looking for? Were it not for the Dewey decimal system, a person could wander around even their local branch library for days, maybe even weeks, looking for the specific information they are seeking.

So, this kind of question truly presents a major challenge. Gathering the data is actually the easiest part, I would think. The real challenge would come in the formatting, the cataloging, the translation, the preservation, and the recoverability of the data you've gathered.

So far, the only data stored by members of the human race to have survived the span of several thousands of years, has been the data painted on or carved into stone or metal, the most durable mediums of the ages.

This question is an interesting thought exercise, but I don't think a solution is imminent. History has shown that the acquisition of knowledge is a cyclical thing, despite our best efforts as a species. Knowledge is gained, then lost, then rediscovered. Maybe that's a good thing, since if knowledge was retained in perpetuity, wouldn't we eventually run out of questions? And in the end, isn't it the questions that drive us as a species, not the answers?

Too deep. I'm going to swim back to the shallow end of the pool now.


--Hatch

[Hatch]

If you genuinely want civilization to fail go somewhere like DU.

University of Denver?

http://www.du.edu

[jor-el]

Each time the technology changes, there's usually some process that would allow conversion of files to a new format. Someone would have to be around to bridge the gap.

Consider February 17 2009. Every standard television set in the CONUS becomes a lores monitor once all major broadcasters stop broadcasting NTSC analog signals. There are ATSC boxes that will extend their function for a few years, but these CRT sets will soon be bricks.

[Hatch]

Each time the technology changes, there's usually some process that would allow conversion of files to a new format. Someone would have to be around to bridge the gap.

The problem is though, that the more data you have, the longer it takes to convert. The government office I worked with had a few thousand relatively simple documents to convert, and it took a dedicated team weeks to do the conversion.

Keep in mind too that the folks doing the converting also had access to the original program and computers the documents were written on. Without those things, conversion would have been far more difficult, if not impossible.

Any Great Library concept would need to be designed to withstand a break in continuity of stewardship, in order to serve the purpose of passing down information to future generations.

--Hatch

[jor-el]

A break in stewardship like loss of the crew assembling or maintaining the Library?

I've already mentioned risks. I'm asking for solutions, or at least stopgaps.

Making the Library portable to minimize the risks? A physical library is tons of books which are perishable. I think many agree to the value of at least backing up the books with PDFs or text documents on some sort of improved hard disk makes a whole lot of sense. The backup can be transported to safer locales on alternate routes or by wire if the infrastructure is still intact.

Hatch, you seem dead set against electronic storage of data for any purpose.
Consider this. On the laptop I'm currently posting this message with, I have my department's entire lesson plan for training police officers. Police procedures, Vehicle and Traffic Law, Criminal Procedure Law, and NYS Penal Law. The entire folder is backed up on a second internal drive. And backed up to a spare external drive with additional drives at home.
In theory, if NYC were devastated by some cataclysm and the bulk of the officers were lost, if I survived with my gear intact, I could train nearly every officer niche position, such as Highway, ESU, Harbor and Aviation (that last one would be a lot easier if my old partner with a Commercial Pilot's license survived as well).

If the material were in paper and videotape format, I would need a long bed p/u truck to haul it around.

For a physical library to work it would also need some librarians to maintain it. I have just me.

[Hatch]

I am not dead set against anything. I'm just pointing out the real challenges to electronic data storage for long term archival. You have to define a problem before you can develop the solution.

In my previous post I suggested several possible means of long term storage of electronic data.

The problem I see here is that you aren't seeing the scope of what you are proposing. You're talking about storing data in PDF on a hard drive and calling it Alexandria II. In 100 years, will PDF still be a readable format? Will the hard drive be able to be plugged into a computer to be read? Look how much computer hardware interfaces have changed in 20 years, and extrapolate the problem to 100 or 1000 years.

Say I discovered one of the first hard drives ever manufactured in the basement of a university, along with notes or documents to suggest that it contained valuable scientific research. First, no modern computer would have an interface to plug it in, so I'd have to build an interface adapter (provided I could find specs for the old interface) or I would have to cobble together scavenged antique computer hardware to read the disk.

Provided I can connect the disk and it spins up, I now have to read the data from it. If the documents were written in an antique proprietary format, that might be a big problem. And we're only talking about the span of decades here.

So I'm not naysaying just to aggravate, I'm trying to point out the real challenges to what you propose, so that real solutions might be discussed.

--Hatch

[jor-el]

I am not dead set against anything. I'm just pointing out the real challenges to electronic data storage for long term archival. You have to define a problem before you can develop the solution.

In my previous post I suggested several possible means of long term storage of electronic data.

The problem I see here is that you aren't seeing the scope of what you are proposing. You're talking about storing data in PDF on a hard drive and calling it Alexandria II. In 100 years, will PDF still be a readable format? Will the hard drive be able to be plugged into a computer to be read? Look how much computer hardware interfaces have changed in 20 years, and extrapolate the problem to 100 or 1000 years.

Say I discovered one of the first hard drives ever manufactured in the basement of a university, along with notes or documents to suggest that it contained valuable scientific research. First, no modern computer would have an interface to plug it in, so I'd have to build an interface adapter (provided I could find specs for the old interface) or I would have to cobble together scavenged antique computer hardware to read the disk.

Provided I can connect the disk and it spins up, I now have to read the data from it. If the documents were written in an antique proprietary format, that might be a big problem. And we're only talking about the span of decades here.

So I'm not naysaying just to aggravate, I'm trying to point out the real challenges to what you propose, so that real solutions might be discussed.

--Hatch

You've done nothing BUT point out challenges. I put it that because of dynamic changes to technology, I personally don't have to make it last 100 years. It only has to last maybe 20 years, at most 50. If changes occur, I can make upgrades.
Despite my reputation, I am not immortal. Don't expect me to plan for trends 200 years from now. I won't be here.
Real solutions discussed now could be irrelevant if tomorrow the asteroid hits.
No doubt the feds have one, but good luck accessing it.

[spacetyrant]

I agree with Hatch that we definitely need to better define the scope of the discussion. For starters, some of the issues just discussed such as:

- Archive accessibility due to format changes
- Format conversions

...are clearly pre-PAW concerns (ie. I wouldn't be overly concerned about Adobe coming out with a replacement for PDF in the PAW, forcing you to upgrade to the latest version of Acrobat). It would by necessity, be the responsibility of the archive maintainer to insure they have the ability to access the information they're storing. It would also be a major PITA, but unless there is a way to stem the tide of technical development, revisiting and perhaps converting the archive to an updated format every decade is simply something that needs to be done.

In the PAW, your concerns would focus more on distribution of the archived knowledge to start the rebuilding process. Since access to any specialized tool (ex. PC, CD-ROM/DVD Drive, flash-reader, etc.) needed to read a digital archive would be questionable, a back up built around the need for such a tool may well defeat the purpose of the archive. Books are the obvious solution to this, but hauling around a physical library would be just as impractical. Perhaps what's needed is a solution that mates the two. One idea that comes to mind is to maintain equipment needed for a portable print shop along with a digital archive. Basically have the ability to print your own books as needed, whether that be by inkjet printer hooked up to your digital archive (hooked up to a battery pack recharged via solar, etc.), or an old fashioned printing press with manually placed slugs. I'm not an expert in this field by any means, though I do understand that the consumables involved (paper and ink) would be the achilles heel of such a plan, but the way I see it, it would be a lot easier in the PAW to produce paper and ink than anything electronic.

SpaceTyrant

[phil_in_cs]

Just a couple comments:

As a software professional, a big step at the beginning is pointing out the problems with what's been proposed. The sooner those problems are ID'd, the easier they are to fix. That doesn't make the person pointing them out very popular though.

There needs to be a scope defined for when the information will be used. If it is used during the PAW, we could assume English, and hard copy only. If you are assuming use after a new civilization is restored after TEOTWAWKI, electronic media could be used, but would it even be recognized as such? Any one read Pohl's Gateway series? The aliens left electronic media everywhere, but no one recongized them as such.
Someone used this example: if you gave a modern engineering calculus textbook to one of the Greek masters, with a couple generations they would read, write, and speak something close to English, and science would leap thousands of years forward. If you gave the same people every book ever written on DVD, they'd hold it up to the sun and say "oh! pretty colors!"

I do support this idea of a Foundation, for the reasons Jor-el stated

[Hatch]

First, I have offered a couple of solutions to what I thought was the question posed. I suggested using punch cards as a digital storage medium, like the early days of computer engineering, only using a durable medium for the cards such as gold. I also suggested a solution to the format problem by taking an appliance approach to such a library, thus the library is designed to decode and present it's own data irrespective of future technological change.

Again, I think your frustration at the lack of a simple and direct answer is due to a failure to see the true scope of what you're asking.

But, now I'm confused because in your first post you were proposing a Library of Alexandria or a Foundation, to preserve and pass on the knowledge of the human race to future generations. But now you say you don't need it to last 100 years because you won't live that long. So at this point, I have no idea what you're looking to accomplish?

Why don't you layout the exact problem again so we can try to get on the same page here?

--Hatch

[tilt]

I think there are a few items that we are overlooking here.

First, libraries must have librarians. If Jor-El is saying (in his particular case) that he intends to steward this library, then this means that he will be updating and maintaining the library for as long as the means exist to do so, all the while adding updated information and converting formats to coincide with the technology of the day.

That said, if NYC (in this case) were to befall catastrophe, the information for restoring the police force would be in an easily read format, because it has been maintained up until the point of disaster.

If a disaster happens on such a scale that no area is left untouched and information for whatever reason has been destroyed, the digital media will likely not have survived either. If digital media stored on a few drives can survive, I'm sure there is a ruggedized toughbook somewhere that made it as well. One would have the means to recall and process data as well as an acceptable format to process in.

Reading the line of thinking thus far, it almost seems like most folks are thinking that the human race may be thrown back to the stoneages fr so long that we rebuild society on top of the old one and never find the media in question until the technology of the new society has long since passed the technology epoch of the old society. All things being equal, it is much easier to utilize more advanced data to read legacy formats than it is to use inferior technology to read and process advanced formats.

Take home lesson: storing information in digitl format will work like a champ for 99.9% of contingencies. On the off shot that the Earth is hit my a meteor and life is wiped out, what do we care if explorers thousands of years from our doomsday find the place and can't read our data?

Jor-El isn't trying to design the perfect time-capsule, he's trying to plan for business continuity for low to medium spectrum contingencies of any intensity.

Given the confines of what I believe the spirit of the exercise to be, I would say that information should be saved redundantly in several different locations (in NYC's case at a station in each of the five burroughs (spelling?)) as well as on the internet in a secured FTP server locate in Iowa (or wherever... not in NYC or any other of the top 10 largest cities). Said information could be updated as needed on a quarterly, semi-annually or annually basis and updated at remote sites.

[kir]

I'm with Hatch here as well. He is making some very good points.

Knowledge loss and attrition is something that I suspect is a force that can not be stopped entirely, though steps can be taken to make knowledge preservation easier.

When I consider the ways that information has survived from previous time periods the only solution I can think of is to take the information you want to preserve, make LOTS of copies in multiple interlinear languages (ala the aforementioned Rosetta stone) and store it on many different kinds of media (including paper). Then one must then bury, hide or seclude it in wide range of locations in containers that may minimize damage from sunlight, water, earthquakes, fire and so forth.

One can then only hope that something from this shotgun approach will surviving the geological and linguistic changes that inevitably will come.

I prefer long term data storage methods that do not require interpretation by a piece of technology. Complications provide greater points of failure and I'd prefer not to put all my eggs into one basket. It's the same principle behind a network and the fundamental fault of a centralized library. If you put everything in one format in one location the entire storehouse is going to be vulnerable to complete erasure/destruction to a specific set of instances. From the standpoint of genetics, genome diversity provides for the greater survivability of a species and it is my suspicion the same principle can be applied to information.

As tilt pointed out above, the fundamental principle here is diversify and decentralize.

That said, I do have a bootable SLAX CD with a stackload of pdfs with basic information that I've termed my "survival distro". The hope is that even in a case of total collapse I would be able to cobble together enough working parts and some kind of power source to transcribe what information was needed. Plenty of scenarios where this would not work and a million things that go wrong... but carrying a CD isn't going to add that much weight to my BOB or take up enough space that I'd be swapping it out for something I actually need for immediate survival. (and there's always the chance I could use the CD as a reflection signaling device)

Outside of my BOB I have a spare Toughbook with it's HD removed and some alternative power choices, in my case solar. I'm not going to lug it around in an immediate survival situation, but I'm maintaining it as one more option in the thought of knowledge preservation.

[Hatch]

I had a thought today of trying to mathematically define the probability of retention of data following a disaster. I'm not sure if this makes sense or not, but here goes:

If the value of R is an arbitrary relative probability that data will be retained following a disaster, E is the number of experts on such data prior to the disaster, and T is the time until normalization following the disaster, the value of R might be expressed as:

R = E/T

So, for example: if you have 100 experts on a subject at the time of a disaster, and the time until normalization of society is 10 years following the disaster, the probability of data retention would be a value of 10.

10 = 100/10

However, if there are fewer experts, say 10, the probability of data retention decreases accordingly:

1 = 10/10

Likewise, if the period until normalization is longer, the probability is reduced:

0.1 = 100/1000

This algorithm makes two assumptions: that humans have a tendency to pass on information to each other in a natural fashion, and that the quality of data being passed on has a tendency to degrade over time.

Keep in mind that the value of R is not a percentage, just an arbitrary, relative value. But I think the relationships of the variables are more or less valid.

For example, if we have 10,000 auto mechanics for every 5 molecular biologists, following a catastrophe lasting 100 years, there would be a 2000 times higher probability that we would still have people who know how to work on cars (R=100), then who know how to design gene therapy protocols (R=0.05).

EDIT: I realized the T value is actually meaningless in this scenario, as the relation between the two R values is the same whether T=1 or T=100. It's the relative value of R to itself as T increases that is of importance, and the relative differences between the R values of two knowledge sets.

In the case of the NYC police training thing, if we were to say that there were 1000 experienced police officers capable of retraining a police force at the time of a catastrophe, and the period of disaster was 10 years, the probability of retention of the ability to train a police force would be a value of 100. Should the disaster last 50 years, the probability is reduced to only 20. At 100 years, the probability is reduced further to 10.

This mathematical relation (if correct) would suggest that for areas of knowledge where there are many experts, the need of data archival for short term catastrophes is little, as there will be a high probability of the knowledge being passed on organically, as it were. The need becomes more pressing in the short term for areas of knowledge where there are few experts.

In the long term (T values in excess of 100) it seems the value of data archival would be high, as a high value for time would offset even a high value for the number of experts.

Anyway, there it is in a nutshell. I welcome feedback on the idea.

--Hatch

[jor-el]

Allow me to qualify my requirements for this archive.

To my understanding, there are maybe 5 people who have even bothered to assemble a unified training archive as complete as mine. There is no departmental initiative to do so, and no financial support.

On my own dime, I'm assembling as much diverse media as I can find and storing them to pocket drives.

I've already had one drive stolen. Eighty gigs I'll have to find again.

In the series by Asimov, Foundation was a community of people NOT part of the Galactic government preserving technical and cultural knowledge. They had to do it without resources or aid from the Galactic Empire.
So gold leaf punch cards or handmade personal computers built to last a thousand years are out. The resources are limited to what you/we have now, what we can afford.

Maybe stone knives and bear skins...

[jeremya]

So the goal is in the near term (ie your lifetime) have an archive of this material mainly for you disseminate in the PAW? (Just clarifying.)

If it were me I would research a medium that has 10-20 years longevity under optimum conditions. DVD's might fit the bill perfectly, but I have no idea what the archival specs of a DVD are.

I would then try to find a good storage container for then media. Possibly a archival quality sealed container. Something that is going to keep the media safe and under the best possible conditions even if it is abandoned. I would also try to figure out a label the container so it is recognizable.

I would then make several copies, put them in their own containers, and then distribute them to friends, family, or cache them on bug out land.

Again if this is mainly for your retrieval after the PAW... you can store the media in any format or like you said you can update the formats from time to time. For Text PDF is good... but plain text files are pretty generic and the format is less likely to disappear since it's used a lot even now. (programmers and comma separated value files) plus I believe a text can be opened on a mac, pc, or in linux.

BTW I think it's an interesting project and it definately comes with its own set of problems that have to be overcome.

-- Jeremy