[OAI-implementers] Requesting a part of a record possible wit h OAI-OMH?

Tim Cole Tim Cole" <t-cole3@uiuc.edu
Thu, 22 Jan 2004 14:25:41 -0600

I would second Hussein's response on question of way GetRecordPart, etc.
aren't incorporated into OAI-PMH.

Keep in mind that OAI-PMH originated as a very targeted protocol
designed as a way for eprint archives to share descriptive (typically
bibliographic) metadata. The intent was to keep it simple and keep it
very low-barrier for metadata providers. Many current providers are
using OAI metadata provider implementations that would not be all that
robust when it comes to handling, for instance, the processing of XPath
requests. There are in fact a few OAI metadata providers that don't make
use of an XML parser for any of the dynamic parts of the service (e.g.,
the necessary XML processing is done elsewhere in batch mode).  I think
there's a large segment of the current OAI community that doesn't want
to discourage current and would-be implementers by imposing additional
requirements beyond those already in the protocol which were seen as
(and largely have proven to be) the minimal for its original, and still
primary purpose -- in fact a recommendation was introduced not all that
long ago (the OAI Static Repository) which actually further lowers
barriers for original small-site metadata providers.

Instead the OAI model assumes higher-level service providers will do the
heavy lifting -- which is why I would most favor Jeff Young's approach.
Special purpose third-party services are intrinsic to the OAI model and
I personally would like to see continued further development in this
regard (as opposed to changes in the protocol itself).

Alternatively of course we could see a separate, distinct protocol for
data exchange emerge which is related to or even a superset of OAI-PMH
(think new protocol might be to OAI-PMH as XML is to HTML). Hussein and
others have advocated for and made progress in this direction, and that
would also be a good way to go.

But in brief this is why in my opinion features like GetRecordPart, etc.
weren't and should never be incorporated into OAI-PMH. Someday OAI-PMH
will outlive its usefulness, in the meantime, let's make sure it
continues to meet original purpose and objectives. Tacking on new
features and functionalities will lessen its value to current core

Tim Cole
University of Illinois at UC

----- Original Message ----- 
From: "Hussein Suleman" <hussein@cs.uct.ac.za>
To: "Joachim Wackerow" <wackerow@zuma-mannheim.de>
Cc: <oai-implementers@oaisrv.nsdl.cornell.edu>
Sent: Thursday, January 22, 2004 9:05 AM
Subject: Re: [OAI-implementers] Requesting a part of a record possible
wit h OAI-OMH?

> hi
> just wanted to reply to one specific issue - where you ask if two new
> "Part" verbs should not be considered for a new OAI-PMH version.
> the current OAI-PMH is very popular largely because lots of
> features were thrown out (during development of the protocol) to keep
> things simple - thus making implementation easier, faster and more
> the principle adopted was that PMH transfers data - raw data - and
> it. anything more interesting than raw data transfer should be viewed
> a different "value-added" service provided by a repository or service
> provider. in all likelihood, if there ever is a newer version of PMH,
> will still do only raw data transfer :)
> i should also add that Tim's solution (DC records to describe data,
> pointers to actual data) is something we implemented at VT a few years
> ago for data similar to yours (large files of XML-encoded numerical
> ttfn,
> ----hussein
> Joachim Wackerow wrote:
> > Sorry about the late answer.
> >
> > Thank you very much for all your comments and the summary, your
> > helped me very much.
> >
> > Here is the summary from my perspective:
> >
> > I understand, that the actual version of OAI-PMH can request a whole
> > record (XML document), but not a part of record (a fragment of XML
> > document).
> >
> > The proposal from Naomi (1) is a workaround, which will work if it's
> > only necessary to request only a few defined parts of a record.
> >
> > Jeffs proposal (2a) has the attractive idea to request a part of a
> > record by the means of XPATH. In my eye's it's a drawback to depend
on a
> > external service for the reason, that the "main" protocol (OAI-PMH)
> > lacking this feature (nothing against external services). I would
> > to think more about a general solution (for DDI). But nevertheless,
> > Jeff, thank you very much for your offer.
> >
> > Hussein's proposal (2b) of a new protocol on top of or parallel to
> > OAI-PMH sounds very good. The proposed way to add this new protocol
> > seems to be very flexible also for other new protocols, it would be
> > way of a general extensibility of OAI-PMH. The question seems to me:
> >  "requesting a part of record" not a candidate of a new version of
> > OAI-PMH? "GetRecordPart" and "ListRecordParts" were the tools I'm
> > looking for.
> >
> > The actual version of OAI-PMH has the intention of requesting whole
> > records (primary use of harvesting protocol). The question of
> > a part of a record by the means of XPATH will arise with large XML
> > documents. The service provider could be interested only in a part
> > the information. Large XML documents have usually DTDs/Schemas with
> > deeply nested structure and a lot of elements or have repeatable
> > elements which could be repeated very often.
> >
> > The size of DDI files reach often 500-1000 KB. The description of a
> > social survey consists often in several hundred questions/variables
> > in an extensive general study description.
> >
> > I see one problem with the idea to request fragments of a XML
> > stored in a record: the client would not be able to validate the
> > response, because itís only a fragment of a XML document. A better
> > solution would be to embed the fragment in a frame of a valid XML
> > document (like a "condensed version" of a XML document). That would
> > dependent from the XML DTD/schema (the question would be, which
> > are required).
> >
> > Right now we are just thinking about using OAI-PMH to offer
> > Perhaps there is another way to offer XML files for service
> > Hussein mentioned SOAP.
> >
> > I will try to give the DDI community
> > (http://www.icpsr.umich.edu/DDI/users/listserv.html) a hint on this
> > discussion thread.
> >
> > Achim
> >
> >
> > _______________________________________________
> > OAI-implementers mailing list
> > List information, archives, preferences and to unsubscribe:
> > http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers
> >
> -- 
> =====================================================================
> hussein suleman ~ hussein@cs.uct.ac.za ~ http://www.husseinsspace.com
> =====================================================================
> _______________________________________________
> OAI-implementers mailing list
> List information, archives, preferences and to unsubscribe:
> http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers