A. Hollier E-archiving WG/36

January 2000

It is widely acknowledged that individuals and organisations will increasingly rely upon electronic information systems in the conduct of their business, and also that authentic records will not automatically be created or preserved by these systems. It is necessary to be sure that current records are created and maintained in such a way that they have genuine evidentiary status, and that they will remain accessible in the long term without losing any of the essential characteristics which give this evidentiary value. This issue is being approached in various ways around the world, but no preferred solution or standard has yet been found. Some of these initiatives are described below, divided into research projects and implemented systems as follows:

1. Research projects

1.1 Research Libraries Group (RLG)

1.2 The Pittsburgh Project

1.3 The UBC Project (including InterPARES)

1.4 The DLM Forum

1.5 The Arts and Humanities Data Service

1.6 The Consultative Committee for Space Data Systems (CCSDS)

2. Implemented systems

2.1 Victorian Electronic Records Strategy (VERS)

2.2 Defense Information Systems Agency - Standard DOD

2.3 The EROS Project

2.4 The Internet Archive

2.5 Pfizer - Central Electronic Archive

RLG is a not-for-profit group of institutions devoted to improving access to information that supports research and learning. In December 1994, the Commission on Preservation and Access and the Research Libraries Group created the Task Force on Digital Archiving, composed of individuals drawn from industry, museums, archives and libraries, publishers, scholarly societies and government. The Task Force’s remit was to frame the key problems; define the critical issues that inhibit resolution of each identified problem; recommend actions to remove the issue from the list; consider alternatives to technology refreshing; and make other generic recommendations as appropriate.

Amongst other things the Task Force found technology refreshing to be insufficient, and recommended data migration instead. It also commented that materials in digital archives may need to contain a digital digest or signature that users can independently verify to assure themselves that the object is unchanged from its archival state.

The report puts forward a number of recommendations for next steps, which mainly have to do with its observation that long-term preservation of digital information on the necessary scale requires a deep infrastructure capable of supporting a distributed system of digital archives. It recommends its sponsors to: foster co-operation between existing and potential digital archives, and other relevant bodies; secure funding to sponsor an open competition for proposals to advance digital archives; foster practical experiments or demonstration projects in the archival application of technologies; and be actively engaged in national policy design.

Homepage < http://www.rlg.org/toc.html>

‘Preserving Digital Information’ a Report of the Task Force on Archiving of Digital Information, commissioned by The Commission on Preservation and Access and The Research Libraries Group, Inc. May 1, 1996

The University of Pittsburgh School of Information Sciences (supported by the National Historical Publications and Records Commission) has conducted a research project to examine the variables that affect the integration of recordkeeping requirements in electronic information systems. The major objectives of this research project were to develop a set of well-defined recordkeeping functional requirements, satisfying all the various legal, administrative, and other needs of a particular organization, which can be used in the design and implementation of electronic information systems.

In September 1996 the project published a Framework for Business Acceptable Communications, comprising:

• Functional Requirements for Evidence in Recordkeeping
• Production Rules Version of the Functional Requirements
• Metadata Specifications Derived from the Functional Requirements
• Literary Warrant Supporting the Functional Requirements

This set of metadata specifications has been adopted as the basis for a number of other electronic recordkeeping implementations, for example the Victorian Electronic Records Strategy (see below).

Various publications are listed on the project’s homepage < http://www.lis.pitt.edu/~nhprc/ >

This research project by the University of British Columbia's School of Library, Archival and Information Studies aimed to identify and define the requirements for creating, handling and preserving reliable and authentic electronic records. Findings fell into two categories:

• Specific methods for ensuring the reliability and authenticity of electronic records - including the need to embed procedural rules in the overall records system and integrate business and documentary procedures; and the need to manage electronic records in their documentary context.
• Management issues concerning the maintenance and preservation of reliable and authentic records - including the statement that the integrity of electronic records is best preserved by entrusting the creating body with responsibility for their reliability and the preserving body with responsibility for their authenticity.

The UBC Project researchers worked in close collaboration with the U.S. Department of Defense Records Management Task Force to identify requirements for Records Management Applications (RMA). The resulting standard (DOD 5015.2) is now being used by the U.S. Defense Information Systems Agency to certify RMA vendors (see below).

The second phase of the UBC Project was intended to address the long-term preservation of inactive

electronic records (ie. records which are no longer needed for day-to-day business but which must be

preserved for operational, legal, or historical reasons). This required an interdisciplinary, international approach - the InterPARES Project, which was officially launched on January 1, 1999.

"The Preservation of the Integrity of Electronic Records" by Luciana Duranti and Terry Eastwood

< http://www.slais.ubc.ca/users/duranti/index.htm>

< http://www.interpares.org/index.htm>

On the 18th and 19th of October 1999, the European Commission organised a multidisciplinary

Forum on the problems of the management, storage, conservation and retrieval of machine-readable data. Public administrations and national archive services, as well as representatives of industry and research, took part in the Forum.

Their October 1999 Conference has published draft conclusions, covering development of a reference model for managing electronic documents and records in public administration; a modular European training programme for administrators and archivists on electronic documents and records management to be put in place in 2000; and a reinforced "DLM Action Plan, 1999-2004: access for the European citizen and funding priority activities to facilitate access to information".

The Forum takes the view that while it is the responsibility of information professionals to specify their needs, it is up to industry to provide solutions. Consequently it is formulating a "DLM-message" to the Information and Communication Technology Industry.

The proceedings of DLM-Forum '96 on electronic records, which took place in Brussels between the 18 and 20 December 1996, include a number of useful papers and case studies.

Homepage < http://www.dlmforum.eu.org/ >

The Arts and Humanities Data Service (UK) has produced a Strategic Policy Document presenting thirteen recommendations in the areas of long-term digital preservation, standards, the policy framework, and future research.

The study emphasises that digital preservation is an essentially distributed process including a range of different (and often differently interested) stakeholders. The different groups need to become more aware of the effects of their involvement on the whole life-cycle of the digital resource. Certain best practices appropriate for digital preservation can be automated for data creators through the application software they use. Accordingly, the development of appropriate software and tools may play a key role in digital preservation.

Various standards exist or are emerging, but more information is needed about how they can be applied

effectively to a digital resource at various stages of its life cycle in order to achieve very specific and

clearly articulated aims. Further research is also required into the data policies and practices already implemented.

Arts and Humanities Data Service (UK) - A Strategic Policy Framework for Creating and Preserving Digital Collections. By Neil Beagrie and Daniel Greenstein. Version 4.0, 14/7/98 can be seen at

<http://ahds.ac.uk/manage/framework.htm>

The Organization for International Standardization (ISO) has encouraged the development of standards in support of the long-term preservation of digital information obtained from observations of the terrestrial and space environments. It has requested that the Consultative Committee for Space Data Systems Panel 2 coordinate the development of those standards. The initial effort has been the development of a Reference Model for an Open Archival Information System (OAIS). That is now being reviewed as an ISO Draft International Standard (DIS).

The Public Record Office Victoria (PROV), Australia, has successfully set up a prototype electronic document processing and record capture system. Their Final Report describes the project's findings and includes a general description of the project demonstrator system, functional descriptions of electronic archiving, full details of the VERS long term electronic record format, the metadata schema used by the project and cost estimates of possible VERS compliant system implementations.

The report Keeping Electronic Records Forever, commissioned by Public Record Office Victoria (PROV) in 1995-96, advocated that instead of taking a system oriented approach to electronic records, a data driven approach was more appropriate as the records would outlast any system developed to manage them. In order to do this the VERS project team first examined how records were created within Government and how they were used and archived, and made this explicit via process maps. They also looked at developments elsewhere in the world.

They concluded that an electronic record had to be a fully self-documenting object, and chose to

describe these objects in eXtensible Markup Language (XML). They also determined that an electronic record was made up of one or more documents, contextual information relating this record with other records, and evidential integrity checks. They then built an archival system demonstrator and a retrieval system demonstrator, and created several sample working environments. They adopted a standard record format - the ‘layered’ (onion) model which encapsulates the documents, the context, and authentication in a single object, and standard metadata sets.

The report makes a number of recommendations, but their main findings are that:

• capture of electronic records into long term format, with much of the contextual information captured automatically, is possible and achievable now
• archiving electronic records is possible and achievable now.

The Victorian Electronic Records Strategy Final Report was launched on 31March 1999

< http://home.vicnet.net.au/~provic/vers/final.htm >

The Design Criteria Standard for Electronic Records Management Software Applications, DOD 5015.2-STD (Department of Defense, USA) was signed on November 24 1997. It is endorsed by NARA (National Archives and Records Administration) and is the first example of a federal agency developing formal criteria for electronic records management. In addition to the standard, a software test procedure has been developed along with a register of records management software applications products that have passed the test. All record management software products purchased by the Department of Defense will be selected from certified products listed on that register.

Homepage < http://jitc-emh.army.mil/recmgt/ >

The overall goal of the UK Public Record Office project on Electronic Records in Office Systems (EROS) is to ensure that electronic records of long term value, created across government, are available for future access. The programme is working in three areas:

• Records management guidelines for departments: the PRO is providing standards for use across government and is publishing the results of an annual survey of departments that shows the issues and trends across government. Case studies are also being published to make available some of the approaches and lessons learned in departments.
• The records processes: the appraisal, accessioning and preservation of electronic records - a working group from the PRO and departments is developing appraisal guidelines.
• Long term transfer and access strategy: the EROS Programme aims to develop standards to provide software suppliers and departments with a way of passing records to the PRO electronically - and without loss of functionality.

Homepage <http://www.pro.gov.uk/recordsmanagement/eros/>

The founder, Brewster Kahle, described his project in Scientific American, March 1997. He aims to use web crawlers to collect the public materials on the Internet to construct a digital library. This collection will include all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews bulletin board system, and downloadable software.

An electronic archive was developed within Pfizer Central Research in Sandwich, England, during the mid-1990s, overseen by a project team that included computing, IT, records management and quality assurance representatives. Users select records to be transferred to the archive and supply metadata via the user interface on their PC. The metadata describing record series type is linked to a retention schedule maintained by the Records Management Unit. Once a record has been transferred to the Central Electronic Archive it is the responsibility of the Computing Services Department to ensure that the records are preserved through system migration. Users can retrieve copies of archived files, but the original is kept inviolate.

Described in the Proceedings of the DLM-Forum 1996 <http://www.dlmforum.eu.org/dlm96/index.html>