[OAI-implementers] Resumption 'from' date.

[Simeon Warner]

This sounds like a best-practice rather than something that should be
mandated. Perhaps the suggestion should be:

"Repositories are encouraged to implement persistent resumptionTokens
that can be re-used in the event that one a request is not successfully
completed. One way for repositories to do this is to use the 
resumptionToken in a way that it encodes the next query in a set of 
queries that will complete the original List request.

On failing to complete a request using a resumptionToken, a harvester may 
re-issue the request using the same resumptionToken. If the repository 
supports this use then it will give the correct response. Otherwise it 
will respond with an badResumptionToken error, in which case the 
harvester must start the complete List request again." 

--
Simeon

On Sun, 10 Feb 2002, Xiaoming Liu wrote:
> On Mon, 11 Feb 2002, Alan Kent wrote:
> 
> > If a goal of OAI is to keep it simple and not change radically (which
> > I can appreciate), then I would revert to my simpler proposed extension
> 
> Based on previous discussions in this list, I think there is one way to
> implement a consistent/stateless view (from the harvester) without
> modifying protocol.
> 
> The suggested way is to encode the resumptionToken with the query
> parameter, at the same time data provider always uses datestamp to sort
> the query result.
> 
> The format should be almost same as I suggested before, but the cursor
> will be the current processed datestamp (borrowed from your solution).
> likely:
> 
> resumptionToken=sets:from:until:metadataformat:processed_datestamp.
> 
> The resumptionToken is transparent to service provider, but when data
> provider sees this resumptionToken, it will re-create the query, and use
> this "processed_datestamp" as a "from" date.
> 
> And whether to implement such a mechanism is totally depended on the data
> provider, a large /frequently changed data provider may want to use such a
> mechanism to support a consistent view to harvester. 
> 
> regards,
> liu
> 
> 
> > which is to allow a server to return an optional addtional date/time in
> > ListRecords/ListIdentifiers responses indicating that "the client can
> > use this as a 'from' date to resume if the response token times out".
> > 
> > For the very simple implementations (or small data volume sites), the
> > server just omits this value.
> > 
> > For more sophisticated implementations with a database engine behind
> > the scenes (for example, so it can easly sort the records), then for
> > each packet it can say "I am guaranteeing to at least have returned
> > everything up to this date". This allows a harvester client hitting a
> > large site for a first time to not have to start again from scratch if
> > something goes wrong (resumption token time out etc). Date resolution
> > is fine here (getting some entries a second time is not the problem -
> > the problem is starting again from the very beginning).
> > 
> > My first attempt at a client harvester for example took about a day to
> > go to several sites and download everything. It hung several times
> > (unknown network issues), meaning I had to restart it on some large
> > sites from scratch. Many other sites it failed for (I can post a list
> > if people are interested - many seemed to only support GET and not POST).
> > 
> > Alan
> > _______________________________________________
> > OAI-implementers mailing list
> > OAI-implementers@oaisrv.nsdl.cornell.edu
> > http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers
> > 
> 
> _______________________________________________
> OAI-implementers mailing list
> OAI-implementers@oaisrv.nsdl.cornell.edu
> http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers
>