FW: [OAI-implementers] Open Archives Initiative Protocol for Meta data Harvesting Version 2 news

[Young,Jeff]

Sorry Walter, I forgot to copy the listserv in my response.

-----Original Message-----
From: Young,Jeff 
Sent: Tuesday, February 05, 2002 12:25 PM
To: 'Walter Underwood'
Subject: RE: [OAI-implementers] Open Archives Initiative Protocol for
Metadata Harvesting Version 2 news


> From: Walter Underwood [mailto:wunder@inktomi.com]
> Sent: Monday, February 04, 2002 5:11 PM
> This is surprisingly close to my suggested list approach. If this was
> changed to have the client to send the cursor and the number 
> of records
> requested, then the resumption token is no longer needed. 
> Think of it as
> moving the database cursor from the server the client. Offloading the
> state. This approach is  proven in high-load applications 
> like LDAP and
> HTTP search engines.

Tim Brody raised a point in a different thread that hasn't been addressed
yet:

	I would be interested to know how resumptionTokens can be avoided,
	as RT is both flow-control (which can be replaced by start-maxrows
	requests), but also state information (i.e. which records are to be 
	returned). If different sections of the same query are requested, 
	without state being maintained, surely there is a risk that some 
	records may be missed in the overlap? (or are you presuming that all

	records are added, and returned, sequentially?)

Of course, resumptionTokens don't guarantee that an arbitrary data provider
will return a complete set of results. They merely provide a mechanism to
make it possible. Without such a guarantee, harvesters are obliged to
periodically reharvest the entire repository if they want to pick up those
missed items.

Jeff