[OAI-implementers] Lists/Flow-control/Emptiness

Tim Brody tim@tim.brody.btinternet.co.uk
Mon, 15 Jul 2002 14:58:42 +0100

----- Original Message -----
From: "Michael L. Nelson" <mln@ruby.ils.unc.edu>

> On Thu, 11 Jul 2002, Tim Brody wrote:
> > 1) empty resumption token in complete list
> >
> > Where a result set is returned without flow control (e.g. a small list),
> > an empty resumption token be appended to this list, even if it is
> > "complete"?
> I guess its not explicitly stated in the protocol as "must not"...
> But resumptionTokens are for incomplete lists, and including an empty
> resumptionToken would seem likely to confuse harvesters.  Or maybe it
> wouldn't confuse them, but at a miminum it is superfluous (below).

If the repository implementation is simpler by always including a resumption
token element, and it doesn't cause a problem with the protocol, then it
should be ok to do this?

> > (The logic to cope with determining the difference between the
> > list and a complete list is not trivial)
> >
> check for the existence of a resumptionToken in the CGI vars.  if the
> harvester passed in a resumptionToken, then it is the final incomplete
> list.  if not, it is a complete list all by itself.

I came up with the same solution. But I wouldn't class this as "trivial", as
the logic intersects with other aspects of the protocol (below...).

> > 2) no-records+resumption token
> >
> > An empty list won't validate, e.g. ListRecords without any records.
> >
> > It is possible that flow control may result in an incomplete list that
> > not contain any records, e.g.:
> > Harvester requests (until=2002-03-04)
> > Performs flow control, gets to last page-1
> > Records on last page move out of result set
> > Harvester requests final token, for which no records can be returned
> >
> > An empty token by itself will not validate, and "noRecordsMatch" means
> > no records could be returned for the query, not for the resumption
> >
> > Change schema or return noRecordsMatch?
> >
> this would seem to be covered in section 3.5.1:
> [...] In cases where there are substantial changes to the repository, it
> may be appropriate for a repository to return a badResumptionToken error,
> signaling that the harvester should restart the list request sequence.
> if you have a resumptionToken *and* no records match, then something ugly
> did happen:  either the harvester is trying to "forge" the
> resumptionTokens, or the repository had (potentially significant) changes.
> I think returning both noRecordsMatch and badResumptionToken would be the
> safe way to handle it.

Ok. This is kinda "overloading" errors though. An error hasn't occurred - a
change occurred in the repository that resulted in an empty list.

noRecordsMatch is defined as resulting from the original query, not as a
side-effect of flow-control, and the resumption token in question is valid?

So logic is now:

if( no records ) {
    if( CGI resumption token ) {
        add noRecordsMatch ("no records match resumption token")
        add badResumptionToken ("because it doesn't match any records")
    } else {
        add noRecordsMatch
} else if( no resumption Token ) {
    add empty token

All the best,