Implementation Guidelines for the Open Archives Initiative Protocol for Metadata Harvesting
	Specification for an OAI Static Repository and an OAI Static Repository Gateway
	Protocol Version 2.0 of 2002-06-14 Document Version 2004/03/31T19:05:00Z http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm

Editors

The OAI Executive:
Herbert Van de Sompel <herbertv@lanl.gov> -- Los Alamos National Laboratory - Research Library
Carl Lagoze <lagoze@cs.cornell.edu> -- Cornell University - Computing and Information Science

From the OAI Technical Committee:
Michael Nelson <mln@cs.odu.edu> -- Old Dominion University - Dept of Computer Science
Simeon Warner <simeon@cs.cornell.edu> -- Cornell University - Computing and Information Science

Contributors:
Patrick Hochstenbach <hochsten@lanl.gov> -- Los Alamos National Laboratory - Research Library
Henry Jerez <hjerez@lanl.gov> -- Los Alamos National Laboratory - Research Library

This document is one part of the Implementation Guidelines that accompany the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH).

Table of Contents

1. Introduction
2. Concepts and Definitions
3. Static Repository Description
   3.1 Static Repository Overview
   3.2 Static Repository Conformance Rules
   3.3 Initiating Intermediation with a Static Repository Gateway
   3.4 Terminating Intermediation by a Static Repository Gateway
   3.5 Ongoing Access to a Static Repository
   3.6 Static Repository Example
4. Static Repository Gateway Description
   4.1 Initiating Intermediation for Static Repositories
   4.2 Terminating Intermediation for Static Repositories
      4.2.1 Terminating Intermediation by Request
      4.2.2 Terminating Intermediation Unilaterally
   4.3 Providing Intermediation for Static Repositories
   4.4 Additional OAI-PMH Issues for Static Repository Gateways
      4.4.1 Contents of the OAI-PMH Identify Response Provided by a Static Repository Gateway
      4.4.2 Static Repository Gateway Flow Control
   4.5 Security Considerations
A1. XML Schema
   A1.1 Static Repository XML Schema
   A1.2 Restricted OAI-PMH XML Schema
Acknowledgements
Document History

1. Introduction

Knowledge of the OAI-PMH is a prerequisite for understanding this document.

A Static Repository, introduced here, provides a simple approach for exposing relatively static and small collections of metadata records through the OAI-PMH. The Static Repository approach is targeted at organizations that:

• Have metadata collections ranging in size between 1 and 5000 records;
• Can make static content available through a network-accessible Web server;
• Need a technically simpler implementation strategy compared to acting as an OAI-PMH Repository, which requires processing OAI-PMH requests.

A Static Repository is an XML file that is made accessible at a persistent HTTP URL. The XML file contains metadata records and repository information.

A Static Repository becomes accessible via OAI-PMH through the intermediation of one Static Repository Gateway. The restriction that only one Static Repository Gateway acts as an intermediary for each Static Repository reduces potential problems with large-scale duplication of metadata records among OAI-PMH repositories. A Static Repository Gateway uses the metadata records and repository information, provided via XML in the Static Repository, to respond to the six OAI-PMH requests for access to that information. Because a Static Repository Gateway maps a unique Static Repository base URL to each such Static Repository, harvesters can access a Static Repository in exactly the same manner as they access any other OAI-PMH Repository.

The relationship between Static Repositories, a Static Repository Gateway, and an OAI-PMH harvester is illustrated in the figure below. The Static Repository and the Static Repository Gateway are described in the remainder of this document. Implementers whose sole interest is the creation of a Static Repository may skip Section 4 that describes the Static Repository Gateway.

Static Repositories, a Static Repository Gateway and an OAI-PMH Harvester

2. Concepts and Definitions

Some concepts that are essential for an understanding of this specification are given below.

Static Repository: A Static Repository is an XML file, available at a persistent HTTP URL that conforms to the rules in Section 3.2. A Static Repository contains metadata records and supporting information required for the purpose of harvesting via the OAI-PMH through the intermediation of a Static Repository Gateway. A Static Repository is not a OAI-PMH Repository, because it is a file, not a server that can respond to the six OAI-PMH requests.

Static Repository URL: The HTTP URL where the Static Repository is accessible.

Static Repository Gateway: A Static Repository Gateway provides intermediation for one or more Static Repositories. It assigns each such Static Repository a unique Static Repository base URL, all with a common Static Repository Gateway URL prefix, and thereby exposes each individual Static Repository as an individual OAI-PMH Repository.

Static Repository Gateway URL: The HTTP URL of the Static Repository Gateway that provides the common prefix for all base URLs of Static Repositories for which the Static Repository Gateway provides intermediation.

Intermediation: The relationship between a Static Repository Gateway and one or more Static Repositories, whereby the Static Repository Gateway responds to OAI-PMH requests to provide access to those Static Repositories. This intermediation is opaque to harvesters, who send OAI-PMH requests to the Static Repository base URL associated with the target Static Repository, even though those requests are actually processed by the Static Repository Gateway.

Static Repository base URL: The base URL that provides access via OAI-PMH to the contents of the Static Repository through intermediation by a Static Repository Gateway. This Static Repository base URL is a concatenation of:

• The Static Repository Gateway URL: All base URLs of Static Repositories accessible through a given Static Repository Gateway share a common part of an HTTP address, which is the URL of the Static Repository Gateway itself. This URL is of the form http://gateway:port/gatewaypath .
• A single forward slash [/] character. If the Static Repository Gateway URL ends with a forward slash, then an additional one must not be added.
• The persistent Static Repository URL , which must be of the form http://host:port/path/file, stripped of its prefix URI scheme part (http://). Hence, the resulting sub-string is of the form host:port/path/file .

For example, OAI-PMH requests to a Static Repository at http://an.oai.org/ma/mini.xml using intermediation from a Static Repository Gateway at http://gateway.institution.org/oai must be issued against the Static Repository base URL:

    http://gateway.institution.org/oai/an.oai.org/ma/mini.xml

3. Static Repository Description

3.1 Static Repository Overview

Static Repositories are useful for small and relatively static metadata collections. All metadata, identifiers, and datestamps are managed in a single XML file that must conform to the rules in Section 3.2. This file may be created manually with an XML editing tool, or a text processing application. Alternatively, a Static Repository might be generated periodically by a script that extracts information from an existing database.

A Static Repository must only be accessible at a single Static Repository URL. This must be an HTTP URL of the form:

    http://host:port/path/file

The HTTP URL must not contain a fragment or a query string. The contents of the Static Repository provide metadata records and repository information necessary for intermediation by a Static Repository Gateway. There are a number of restrictions on a Static Repository relative to a standard OAI-PMH Repository. Static Repositories do not support sets, deleted records, response compression, harvesting granularity other than YYYY-MM-DD, or resumptionTokens.

3.2 Static Repository Conformance Rules

A Static Repository must comply with the following conformance rules:

• The MIME type of a Static Repository must be text/xml.
• Encoding of the XML must use the UTF-8 representation of Unicode.
• The Static Repository must validate against the XML Schema shown below.
• The value of the baseURL element of the Static Repository must match the Static Repository base URL at which the Static Repository is available through intermediation from a Static Repository Gateway. Section 3.3 describes the construction of this base URL and the process of initiating intermediation with a Static Repository Gateway.
• Contents of metadata records within the ListRecords elements of the Static Repository must conform to the OAI-PMH record format.
• The Static Repository must not use sets. There must not be any setSpec elements in the headers of records.
• The Static Repository must not include headers for deleted records. There must not be any status attributes on the header elements of records.
• The Static Repository must not claim to support response compression. There must not be any compression elements in the Identify part of the Static Repository.
• The Static Repository must express all datestamps using YYYY-MM-DD granularity. This must be specified as the harvesting granularity in the granularity element.
• The Static Repository must be a single XML file, it must not include any resumptionToken elements.

It is strongly recommended that the identifiers of metadata records in the Static Repository conform to the guidelines for OAI identifiers or other forms of Uniform Resource Names (URNs) in the sense of RFC1737. This practice will facilitate the ability to recognize duplicates among harvested metadata records.

3.3 Initiating Intermediation with a Static Repository Gateway

OAI-PMH access to a Static Repository is only possible through the intermediation of a Static Repository Gateway. To initiate this intermediation, the administrator of a Static Repository must select one Static Repository Gateway that will act as the intermediary and must issue an HTTP GET request of the form:

    <Static Repository Gateway URL>?initiate=<Static Repository URL>

For example, an administrator of a Static Repository with a Static Repository URL of http://an.oai.org/ma/mini.xml requesting intermediation from a Static Repository Gateway with a Static Repository Gateway URL http://gateway.institution.org/oai must issue the HTTP GET request:

    http://gateway.institution.org/oai?initiate=http://an.oai.org/ma/mini.xml

The policies and implementation of the Static Repository Gateway will determine both whether any additional steps are necessary, and the length of any delay between issuance of this request and the actual initiation of intermediation. For example, a particular Static Repository Gateway may have a policy that requires gateway administrator intervention and email exchange before initiating intermediation. The Static Repository Gateway administrator should follow the instructions concerning ongoing access to a Static Repository to check that intermediation is successful.

3.4 Terminating Intermediation by a Static Repository Gateway

To terminate intermediation by a Static Repository Gateway, the administrator of a Static Repository must first either remove the Static Repository from its Static Repository URL, or change the baseURL element so that it no longer matches the Static Repository base URL. The administrator must then issue an HTTP GET request of the form:

    <Static Repository Gateway URL>?terminate=<Static Repository URL>

For example, an administrator of a Static Repository with a Static Repository URL of http://an.oai.org/ma/mini.xml requesting the termination of intermediation from a Static Repository Gateway with a Static Repository Gateway URL http://gateway.institution.org/oai must issue the HTTP GET request:

    http://gateway.institution.org/oai?terminate=http://an.oai.org/ma/mini.xml

The Static Repository Gateway will respond to a termination request according to the following rules.

• If the baseURL element of the Static Repository no longer matches the Static Repository base URL then termination of intermediation should be immediate.
• If the Static Repository has been removed from its Static Repository URL then interval between issuance of the termination request and the actual termination of intermediation by the Static Repository Gateway may vary according to the policies and implementation of the Static Repository Gateway.
• Termination requests must be ignored if the baseURL element of the Static Repository still matches the Static Repository base URL.

The policies and implementation of the Static Repository Gateway will determine both whether any additional steps are necessary, and the length of any delay between issuance of this request and the actual termination of intermediation. The Static Repository Gateway administrator should follow the instructions concerning ongoing access to a Static Repository to check whether the attempt to terminate intermediation is successful.

3.5 Ongoing Access to a Static Repository

An administrator of a Static Repository should be aware that access to the Static Repository is dependent on the intermediation of a single Static Repository Gateway. The Static Repository conformance rules prohibit intermediation by more than one Static Repository Gateway by demanding a match between the baseURL element in the Static Repository and the Static Repository base URL at which it is available through the intermediation of the Static Repository Gateway.

It is recommended that the administrator check that intermediation is successful by issuing an OAI-PMH Identify request to the appropriate Static Repository base URL:

• After initiating intermediation to check that the initiation request was successful.
• Each time the Static Repository is changed to check that the change has not negatively affected intermediation, for example if the Static Repository is no longer conformant.
• On a periodic basis to determine whether the Static Repository Gateway has unilaterally terminated access. Note that it is recommended that a Static Repository Gateway send an email message to the Static Repository administrator in the case of this termination, but this may not always happen.
• After terminating intermediation to check that the termination request was successful.

The response to this Identify request will be one of the following:

• An HTTP response with a Status-Code of "200 (OK)" and an Identify response in which the baseURL element matches the baseURL element in the Static Repository. This indicates that intermediation is successful.
• An HTTP response with a Status-Code of "502 (Bad Gateway)". This indicates rejection or termination of the intermediation due to the failure of the Static Repository to follow the Static Repository conformance rules or other reasons, such as a policy decision by the administrator of the Static Repository Gateway. The response may also include a Reason-Phrase that is a human readable explanation of the rejection or termination.
• An HTTP response with a Status-Code of "503 (Service unavailable)", with a Retry-After period specified. This indicates that the Static Repository Gateway is delaying intermediation. The Gateway may be busy or may be delaying due to processing an earlier request from the Static Repository to initiate intermediation. The administrator of the Static Repository should wait for the Retry-After period before reissuing the OAI-PMH Identify request to determine whether the Static Repository Gateway is intermediating.

3.6 Static Repository Example

Below, an example is shown of a Static Repository. As can be seen, it contains:

• A single Identify element. The content of the baseURL child element is the Static Repository base URL at which the selected Static Repository Gateway makes this Static Repository accessible via the OAI-PMH.
• A single ListMetadataFormats element. In this example, the content of this element reveals support for two metadata formats, with respective metadataPrefixes oai_dc and oai_rfc1807.
• Two ListRecords elements. As can be seen from the metadataPrefix attributes provided with each of the ListRecords elements, the first ListRecords element contains all oai_dc metadata records, while the second ListRecords element contains all oai_rfc1807 metadata records.

<?xml version="1.0" encoding="UTF-8"?>
<Repository xmlns="http://www.openarchives.org/OAI/2.0/static-repository" 
            xmlns:oai="http://www.openarchives.org/OAI/2.0/" 
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/static-repository 
                                http://www.openarchives.org/OAI/2.0/static-repository.xsd">
  <Identify>
    <oai:repositoryName>Demo repository</oai:repositoryName>
    <oai:baseURL>http://gateway.institution.org/oai/an.oai.org/ma/mini.xml</oai:baseURL>
    <oai:protocolVersion>2.0</oai:protocolVersion>
    <oai:adminEmail>jondoe@oai.org</oai:adminEmail>
    <oai:earliestDatestamp>2002-09-19</oai:earliestDatestamp>
    <oai:deletedRecord>no</oai:deletedRecord>
    <oai:granularity>YYYY-MM-DD</oai:granularity>
  </Identify>
  <ListMetadataFormats>
    <oai:metadataFormat>
      <oai:metadataPrefix>oai_dc</oai:metadataPrefix>
      <oai:schema>http://www.openarchives.org/OAI/2.0/oai_dc.xsd</oai:schema>
      <oai:metadataNamespace>http://www.openarchives.org/OAI/2.0/oai_dc/</oai:metadataNamespace>
    </oai:metadataFormat>
    <oai:metadataFormat>
      <oai:metadataPrefix>oai_rfc1807</oai:metadataPrefix>
      <oai:schema>http://www.openarchives.org/OAI/1.1/rfc1807.xsd</oai:schema>
      <oai:metadataNamespace>http://info.internet.isi.edu:80/in-notes/rfc/files/rfc1807.txt</oai:metadataNamespace>
    </oai:metadataFormat>
  </ListMetadataFormats>
  <ListRecords metadataPrefix="oai_dc">
    <oai:record> 
      <oai:header>
        <oai:identifier>oai:arXiv:cs/0112017</oai:identifier> 
        <oai:datestamp>2001-12-14</oai:datestamp>
      </oai:header>
      <oai:metadata>
        <oai_dc:dc 
            xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
            xmlns:dc="http://purl.org/dc/elements/1.1/" 
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ 
                                http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
          <dc:title>Using Structural Metadata to Localize Experience of 
                    Digital Content</dc:title> 
          <dc:creator>Dushay, Naomi</dc:creator>
          <dc:subject>Digital Libraries</dc:subject> 
          <dc:description>With the increasing technical sophistication of 
            both information consumers and providers, there is 
            increasing demand for more meaningful experiences of digital 
            information. We present a framework that separates digital 
            object experience, or rendering, from digital object storage 
            and manipulation, so the rendering can be tailored to 
            particular communities of users.
          </dc:description> 
          <dc:description>Comment: 23 pages including 2 appendices, 
            8 figures</dc:description> 
          <dc:date>2001-12-14</dc:date>
        </oai_dc:dc>
      </oai:metadata>
    </oai:record>  
    <oai:record>
      <oai:header>
        <oai:identifier>oai:perseus:Perseus:text:1999.02.0084</oai:identifier>
        <oai:datestamp>2002-05-01</oai:datestamp>
      </oai:header>
      <oai:metadata>
        <oai_dc:dc 
            xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
            xmlns:dc="http://purl.org/dc/elements/1.1/" 
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ 
                                http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
          <dc:title>Germany and its Tribes</dc:title>
          <dc:creator>Tacitus</dc:creator>
          <dc:type>text</dc:type>
          <dc:source>Complete Works of Tacitus. Tacitus. Alfred John Church. 
            William Jackson Brodribb. Lisa Cerrato. edited for Perseus. 
            New York: Random House, Inc. Random House, Inc. reprinted 1942.
          </dc:source>
          <dc:identifier>http://www.perseus.tufts.edu/cgi-bin/ptext?
            doc=Perseus:text:1999.02.0083</dc:identifier>
        </oai_dc:dc>
      </oai:metadata>
    </oai:record>
  </ListRecords>
  <ListRecords metadataPrefix="oai_rfc1807">
    <oai:record>
      <oai:header>
        <oai:identifier>oai:arXiv:cs/0112017</oai:identifier>
        <oai:datestamp>2001-12-14</oai:datestamp>
      </oai:header>
      <oai:metadata>
        <rfc1807 
            xmlns="http://info.internet.isi.edu:80/in-notes/rfc/files/rfc1807.txt" 
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xsi:schemaLocation="http://info.internet.isi.edu:80/in-notes/rfc/files/rfc1807.txt
                                http://www.openarchives.org/OAI/1.1/rfc1807.xsd">
          <bib-version>v2</bib-version>
          <id>cs/0112017</id>
          <entry>December 23, 2001</entry>
          <title>Using Structural Metadata to Localize Experience of 
             Digital Content</title>
          <author>Naomi Dushay</author>
          <date>December 14, 2001</date>
        </rfc1807>
      </oai:metadata>
      <oai:about>
        <oai_dc:dc 
            xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
            xmlns:dc="http://purl.org/dc/elements/1.1/" 
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
            xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ 
                                http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
          <dc:publisher>Los Alamos arXiv</dc:publisher>
          <dc:rights>Metadata may be used without restrictions as long as 
            the oai identifier remains attached to it.</dc:rights>
        </oai_dc:dc>
      </oai:about>
    </oai:record>
  </ListRecords>
</Repository>

4. Static Repository Gateway Description

NOTE: Implementers whose sole interest is the creation of a Static Repository may skip this section.

A Static Repository Gateway provides intermediation for one or more Static Repositories. The OAI-PMH base URL for each Static Repository is unique, but all Static Repositories intermediated by one Static Repository Gateway share a common prefix, the Static Repository Gateway URL. To discourage excessive duplication of metadata records, a Static Repository Gateway must not process OAI-PMH requests for a Static Repository if the Static Repository base URL of the request does not match the value of the baseURL element of the Static Repository. A Static Repository Gateway must not change the metadata expressed in the Static Repository. Downstream services may alter, augment or cross-walk the metadata but the sole purpose of a Static Repository Gateway is to intermediate access to the Static Repository.

The implementation of a Static Repository Gateway has three aspects:

1. Initiating intermediation for Static Repositories.
2. Providing intermediation for Static Repositories
3. Terminating intermediation for Static Repositories

4.1 Initiating Intermediation for Static Repositories

Section 3.3 of this document describes how an administrator of a Static Repository initiates intermediation with a single Static Repository Gateway. This section provides information about this relationship from the perspective of the Static Repository Gateway.

A Static Repository Gateway must process HTTP GET requests of the form:

    <Static Repository Gateway URL>?initiate=<Static Repository URL>

A Static Repository Gateway may implement any local policies or procedures subsequent to receiving this request and before actually acting as an intermediary. These may include additional authorization or certification procedures outside the scope of the OAI-PMH.

4.2 Terminating Intermediation for Static Repositories

Intermediation for a Static Repository may be terminated either following a request from the administrator of the Static Repository, or unilaterally by the Static Repository Gateway.

4.2.1 Terminating Intermediation by Request

Section 3.4 of this document describes how an administrator of a Static Repository terminates intermediation with a single Static Repository Gateway. This section provides information about this relationship from the perspective of the Static Repository Gateway.

A Static Repository Gateway must process HTTP GET requests of the form:

    <Static Repository Gateway URL>?terminate=<Static Repository URL>

On receipt of such a request, the Static Repository Gateway must attempt to fetch the Static Repository and follow these rules:

• Intermediation must be terminated if the baseURL element of the Static Repository no longer matches the Static Repository base URL. Termination should be immediate.
• If the Static Repository is inaccessible then the Static Repository Gateway may implement local policies or procedures subsequent to receiving the termination request before terminating intermediation. These may include additional authorization or certification procedures outside the scope of the OAI-PMH.
• The request must be ignored if the baseURL element of the Static Repository still matches the Static Repository base URL.

It is recommended that an email message to the Static Repository administrator (address available in the adminEmail element of the Static Repository) be sent regarding the termination.

4.2.2 Terminating Intermediation Unilaterally

A Static Repository Gateway may unilaterally terminate an intermediation relationship at any time. Some of the possible reasons for this termination are:

• Continued unavailability of the Static Repository at its Static Repository URL.
• Continued failure to comply with the Static Repository conformance rules. A recommended reason for immediate termination is when the value of the baseURL element of the Static Repository does not match Static Repository base URL to which the request has been issued. This indicates that the Static Repository has initiated intermediation with another Static Repository Gateway.
• A policy decision of the Static Repository Gateway administrator.

It is recommended that an email message to the Static Repository administrator (address available in the adminEmail element of the Static Repository) be sent regarding the termination.

4.3 Providing Intermediation for Static Repositories

Once a Static Repository Gateway has accepted intermediation for a Static Repository, it must respond to all six OAI-PMH requests issued against the Static Repository base URL, for the length of the intermediation relationship.

In order to guarantee the accuracy of the exposed metadata, a Static Repository Gateway must use the most recent version of a Static Repository. A Static Repository Gateway may accomplish this by fetching the Static Repository from its persistent Static Repository URL using an HTTP GET request for every OAI-PMH request received. However, a Static Repository Gateway may optimize its performance by caching Static Repositories. In that case a Static Repository Gateway must perform a freshness-test on the cached Static Repository by comparing it with the version at the Static Repository URL before responding to every OAI-PMH request. It should do so by using a HTTP GET with an If-Modified-Since header that contains the date of the cached version of a Static Repository.

The response of a Static Repository Gateway to an OAI-PMH request issued against the Static Repository base URL must take one of the following forms:

• If the Static Repository Gateway is rejecting or terminating intermediation, it should issue an HTTP response with a Status-Code of "502 (Bad Gateway)". Rejection or termination may be due to the failure of the Static Repository to follow the Static Repository conformance rules or other reasons, such as a policy decision by the Static Repository Gateway. The response may also include a Reason-Phrase that is a human readable explanation of the rejection or termination.
• If the Static Repository Gateway is currently unable to intermediate due to delays in initiating intermediation or server overload it should issue an HTTP response with a Status-Code of "503 (Service unavailable)", with a Retry-After period specified. The Static Repository Gateway must be prepared to respond to another attempt to initiate intermediation after the Retry-After period .
• If the Static Repository Gateway performs the freshness-test as defined above and the Static Repository is not accessible at its Static Repository URL, the Static Repository Gateway must respond to the harvesting request with a HTTP Status-Code "504 (Gateway Timeout)".
• If the Static Repository Gateway performs the freshness-test as defined above and the Static Repository is accessible at its Static Repository URL, and the result of the freshness-test indicates that the cached version is the same as the version at the Static Repository URL, then the Static Repository Gateway may respond to the harvesting request by using the cached version of the Static Repository.
• If the Static Repository Gateway performs the freshness-test as defined above and the Static Repository is accessible at its Static Repository URL, and the result of the freshness-test indicates that the cached version is not the same as the version at the Static Repository URL, then the Static Repository Gateway must fetch the Static Repository from its Static Repository URL.
  • Rather than delaying the response until this fetch from the Static Repository is complete and it is processed, the Static Repository Gateway may respond to the harvesting request with a HTTP Status-Code "503 (Service Unavailable)". This specifies a Retry-After period covering the estimated time of fetching the Static Repository from its Static Repository URL, and validating it.
  • If the fetched version of the Static Repository does not comply with the Static Repository conformance rules, then the Static Repository Gateway must respond to the harvesting request with a HTTP Status-Code "502 (Bad Gateway)". The response should also include a Reason-Phrase that is a human readable explanation of the failure. It must not respond to the harvesting request using the cached version of the Static Repository.
  • If the fetched version of the Static Repository does comply with the Static Repository conformance rules, then the Static Repository Gateway must respond to the harvesting request using the fetched version. The Static Repository Gateway must then replace its cached version with the fetched version.

4.4 Additional OAI-PMH Issues for Static Repository Gateways

OAI-PMH processing by a Static Repository Gateway is mandated by the OAI-PMH specification. This section describes a number of special issues relevant to this specification.

4.4.1 Contents of the OAI-PMH Identify Response Provided by a Static Repository Gateway

A Static Repository Gateway must add a gateway description to the Identify response for each Static Repository for which it provides intermediation. The gateway description reveals that the OAI-PMH response is provided through the intermediation of a Static Repository Gateway. The gateway description must include the following elements:

• source - the Static Repository URL
• gatewayDescription - value must be http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm .
• gatewayAdmin - the e-mail address of the administrator of the Static Repository Gateway.
• gatewayURL - the Static Repository Gateway URL; it is the part of all base URLs assigned by the Static Repository Gateway that is common to all Static Repositories that it makes accessible.

The gateway description may also include:

• gatewayNotes - a URL for additional notes about the particular instantiation of the Static Repository Gateway. This element could be used to described usage policies, software versions, and related information.

To support dynamic discovery of Static Repositories, the Static Repository Gateway may include a friends description in the Identify response that it provides for each of the Static Repositories for which it provides intermediation. If included, that friends description should list the base URLs of all Static Repositories that are harvestable through the Static Repository Gateway. A Static Repository Gateway may develop a policy to periodically refresh the friends description, for instance based on tracking the (un)availability of listed Static Repositories over time.

An example of a response to the Identify request from a Static Repository Gateway with Static Repository Gateway URL http://gateway.institution.org/oai issued against the example Static Repository (section 3.6), is shown below:

• The response contains a friends description that lists all Static Repositories that are accessible through the Static Repository Gateway;
• The content of each baseURL element shown in the response is the concatenation of the Static Repository Gateway URL (http://gateway.institution.org/oai) and the part of the Static Repository URL obtained after removal of http:// prefix, respectively an.oai.org/ma/mini.xml, site1.org/mini/file1, loca.org/data:8080, and univ.edu/lib/pubs.xml . Note that the colon in loca.org:8080/data must be URL-encoded to create a valid baseURL. Note also that a single forward slash [/] character has been added between the concatenated strings.
• The response contains a gateway description that lists the Static Repository URL, as well as some specifics about the Static Repository Gateway.

<?xml version="1.0" encoding="UTF-8"?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/
                             http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2002-02-08T12:00:01Z</responseDate>
  <request verb="Identify">http://gateway.institution.org/oai/an.oai.org/ma/mini.xml</request>
  <Identify>
    <repositoryName>Demo repository</repositoryName>
    <baseURL>http://gateway.institution.org/oai/an.oai.org/ma/mini.xml</baseURL>
    <protocolVersion>2.0</protocolVersion>
    <adminEmail>jondoe@oai.org</adminEmail>
    <earliestDatestamp>2002-09-19</earliestDatestamp>
    <deletedRecord>no</deletedRecord>
    <granularity>YYYY-MM-DD</granularity>
    <description>
      <friends 
          xmlns="http://www.openarchives.org/OAI/2.0/friends/" 
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/friends/
                              http://www.openarchives.org/OAI/2.0/friends.xsd">
        <baseURL>http://gateway.institution.org/oai/site1.org/mini/file1</baseURL>
        <baseURL>http://gateway.institution.org/oai/loca.org%3A8080/data</baseURL>
        <baseURL>http://gateway.institution.org/oai/univ.edu/lib/pubs.xml</baseURL>
      </friends>
    </description>
    <description>
      <gateway 
          xmlns="http://www.openarchives.org/OAI/2.0/gateway/" 
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/gateway/
                              http://www.openarchives.org/OAI/2.0/gateway.xsd">
        <source>http://an.oai.org/ma/mini.xml</source>
        <gatewayDescription>http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm</gatewayDescription>
        <gatewayAdmin>pat@institution.org</gatewayAdmin>
        <gatewayURL>http://gateway.institution.org/oai/</gatewayURL>
      </gateway>
    </description>
  </Identify>
</OAI-PMH>

4.4.2 Static Repository Gateway Flow Control

Like any OAI-PMH Repository, a Static Repository Gateway may use resumptionTokens as a means of partitioning large responses. However, the Static Repository must ensure that the Static Repository has not changed before responding to harvesting requests with a resumptionToken. In other words the entire set of requests in a list request sequence must be based on the same Static Repository contents. To do this, the Static Repository Gateway must perform a freshness-test on each request in the list request sequence. The Static Repository Gateway should do so by using a HTTP GET with an If-Modified-Since header that contains the date of the first request in the list request sequence of a Static Repository. If the result of the freshness-test is that the Static Repository has changed in the course of a list request sequence, the Static Repository Gateway must issue an OAI-PMH error badResumptionToken.

4.5 Security Considerations

The following security issues require attention when operating a Static Repository Gateway:

• A malicious agent may use a Static Repository Gateway as a way to spoof the origin of provocative material (e.g. pornography, slander). The operator of a Static Repository Gateway may wish to restrict the use of its services to exclude certain Static Repositories.
• A malicious agent may use a Static Repository Gateway as a way to launch a denial-of-service attack on a Static Repository. The operator of a Static Repository Gateway may apply mechanisms that prevent excessive access to a Static Repository.
• The content of a Static Repository may be devised to be harmful to a Static Repository Gateway or to a downstream harvester. Strict validation as mandated in section 4.3 should ameliorate this danger. However, additional precautions may be necessary, for example to truncate excessively long responses which may be problematic for an XML parser/validator.
• Static Repository Gateways are subject to security considerations relating to OAI-PMH and HTTP.

A1. XML Schema

Static repositories must validate against the Static Repository XML Schema. This schema imports elements from the OAI-PMH v2.0 namespace via an intermediate Restricted OAI-PMH XML Schema which enforces a number of the Static Repository Conformance Rules by restricting OAI-PMH v2.0 type definitions.

A1.1 Static Repository XML Schema

<?xml version="1.0" encoding="utf-8"?>
<schema targetNamespace="http://www.openarchives.org/OAI/2.0/static-repository"
        xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:sr="http://www.openarchives.org/OAI/2.0/static-repository"
        xmlns:oai="http://www.openarchives.org/OAI/2.0/"
        elementFormDefault="qualified"
        attributeFormDefault="unqualified">

  <import namespace="http://www.openarchives.org/OAI/2.0/"
          schemaLocation="http://www.openarchives.org/OAI/test/OAI-PMH-static-repository.xsd"/>

  <annotation>
    <documentation>
    This XML Schema specifies the structure of an OAI-PMH Static Repository. 
    A Static Repository is an XML file that is valid according to this XML 
    Schema and is described in 
    http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm
    A Static Repository is made accessible as an XML file on a standard 
    web-server. No special software is required at the end of the organization
    that makes the Static Repository available. A Static Repository becomes 
    harvestable via the OAI-PMH through the intermediation of a Static 
    Repository Gateway. 

    This Static Repository XML Schema by Herbert Van de Sompel and Henry N. 
    Jerez (Los Alamos National Laboratory, Research Library, Digital Library 
    Research and Prototyping Team; original 2002-10-26), and Simeon Warner 
    (Cornell University). Inspired by the Vida work by Steven Bird for OAI-PMH
    v1.0 and for the Open Languages Archives Community; see 
    http://www.language-archives.org/docs/implement.html#Vida
    
    beta2 release: 2004-03-30
    $Date: 2004/03/30 16:40:50 $
    </documentation>
  </annotation>

  <element name="Repository" type="sr:RepositoryType"/>

  <complexType name="RepositoryType">
    <annotation>
      <documentation>The Repository element has 2 child elements, Identify 
      and ListMetadataFormats, that are derived from the OAI-PMH v2.0 XML 
      Schema. The third element, ListRecords, is repeatable and is an 
      extension of the ListRecords defined in the OAI-PMH v2.0 XML Schema; 
      it has an additional attribute indicating the metadataPrefix of the 
      included metadata records.
      </documentation>
    </annotation>
    <sequence>
      <element name="Identify" type="oai:IdentifyType"/>
      <element name="ListMetadataFormats" type="oai:ListMetadataFormatsType"/>
      <element name="ListRecords" type="sr:ListRecordsType" maxOccurs="unbounded"/>
    </sequence>
  </complexType>

  <complexType name="ListRecordsType">
    <annotation>
      <documentation>The ListRecords element contains all records with 
      metadata expressed in one of the metadata formats supported by the 
      Static Repository. The mandatory metadataPrefix attribute specifies the 
      metadataPrefix of the included metadata; it must correspond with a value 
      of the metadataPrefix element contained in the ListMetadataFormats 
      element.
      </documentation>
    </annotation>
    <complexContent>
      <extension base="oai:ListRecordsType">
        <attribute name="metadataPrefix" type="oai:metadataPrefixType" use="required"/>
      </extension>
    </complexContent>
  </complexType>

</schema>

A1.2 Restricted OAI-PMH XML Schema

<schema targetNamespace="http://www.openarchives.org/OAI/2.0/"
        xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:oai="http://www.openarchives.org/OAI/2.0/"
        elementFormDefault="qualified"
        attributeFormDefault="unqualified">
  
  <annotation>
    <documentation>This schema restricts the contents a number of elements in
    the OAI-PMH v2.0 schema for use with the static repository schema: 
    http://www.openarchives.org/OAI/2.0/static-repository.xsd
    All elements remain in the namespace http://www.openarchives.org/OAI/2.0/ 
    used by OAI-PMH v2.0 and element instances accepted by these restricted 
    definitions would be valid according to the main schema:
    http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd
    Simeon Warner (Cornell University), 2004-02-12.
    $Date: 2004/03/30 16:43:22 $</documentation>
  </annotation> 

  <redefine schemaLocation="http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
   
  <annotation>
    <documentation>ListRecords block restricted so that it must not include a 
    resumptionToken element.</documentation>
  </annotation>
  <complexType name="ListRecordsType">
    <complexContent>
      <restriction base="oai:ListRecordsType">
        <sequence>
          <element name="record" type="oai:recordType" maxOccurs="unbounded"/>
        </sequence>
      </restriction>
    </complexContent>
  </complexType>


  <annotation>
    <documentation>Identify block restricted so that it must not include a 
    compression element. Some element types also restricted.</documentation>
  </annotation>
  <complexType name="IdentifyType">
    <complexContent>
      <restriction base="oai:IdentifyType">
        <sequence>
          <element name="repositoryName" type="string"/>
          <element name="baseURL" type="anyURI"/>
          <element name="protocolVersion" type="oai:protocolVersionType"/>
          <element name="adminEmail" type="oai:emailType" maxOccurs="unbounded"/>
          <element name="earliestDatestamp" type="oai:UTCdatetimeType"/>
          <element name="deletedRecord" type="oai:deletedRecordType"/>
          <element name="granularity" type="oai:granularityType"/>
          <element name="description" type="oai:descriptionType"
                   minOccurs="0" maxOccurs="unbounded"/>
        </sequence>
      </restriction>
    </complexContent>
  </complexType>


  <annotation>
    <documentation>All records must have header and metadata. As deleted
    records are not supported it is not permissible to have header 
    only.</documentation>
  </annotation>
  <complexType name="recordType">
    <complexContent>
      <restriction base="oai:recordType">
        <sequence>
          <element name="header" type="oai:headerType"/>
          <element name="metadata" type="oai:metadataType"/>
          <element name="about" type="oai:aboutType" 
                   minOccurs="0" maxOccurs="unbounded"/>
        </sequence>
      </restriction>
    </complexContent>
  </complexType>

  <annotation>
    <documentation>The header element must not contain any setSpec elements 
    or a status attribute.</documentation>
  </annotation>
  <complexType name="headerType">
    <complexContent>
      <restriction base="oai:headerType">
        <sequence>
          <element name="identifier" type="oai:identifierType"/>
          <element name="datestamp" type="oai:UTCdatetimeType"/>
        </sequence>
        <attribute name="status" type="oai:statusType" use="prohibited"/>
      </restriction>
    </complexContent>
  </complexType>

  <annotation>
    <documentation>The only granularity permitted is YYYY-MM-DD.</documentation>
  </annotation>
  <simpleType name="granularityType">
    <restriction base="oai:granularityType">
      <enumeration value="YYYY-MM-DD"/>
    </restriction>
  </simpleType>

  <annotation>
    <documentation>As deleted records are not supported, the deletedRecord 
    element may only have the value 'no'.</documentation>
  </annotation>
  <simpleType name="deletedRecordType">
    <restriction base="oai:deletedRecordType">
       <enumeration value="no"/>
    </restriction>
  </simpleType>

  </redefine>
</schema>

Acknowledgements

Support for the development of the OAI-PMH and for other Open Archives Initiative activities comes from the National Science Foundation through Grant No. IIS-9817416. Individuals who have played a significant role in the development of OAI-PMH version 2.0 are acknowledged in the protocol document.

This Static Repository specification is inspired on the ViDa (Virtual Data Provider) work by Steven Bird and Gary Simons for the Open Language Archives Community (OLAC), a leading community of implementers of the OAI-PMH. Special thanks to Steven Bird and Gary Simons. Thanks also to Pete Johnston, Tim Cole and Tom Habing for helpful comments.

Document History

2004-04-23: Release of Specification for an OAI Static Repository and an OAI Static Repository Gateway.
2003-10-10: Beta Release.
2002-11-13: Alpha release.
2002-09-28: Initial pre-release of this document to core OAI team.