[OAI-general] OAI and web crawlers

herbert van de sompel herbertv@CS.Cornell.EDU
Mon, 23 Jul 2001 21:06:00 -0400


"Michael L. Nelson" wrote:
> 
> I've just added:
> 
>         # please use our Open Archives Initiative (OAI) interface instead!
>         # http://naca.larc.nasa.gov/oai/
>         # see http://www.openarchives.org/ for more info
> 
> to my robots.txt file for my two DLs (LTRS & NACATRS).  I doubt these
> messages will be read by humans, but stranger things have happened.
> 

I think this is a great idea.  It would be great PR for the OAI protocol
if important crawlers would start using it.  But as Michael mentions,
the robots.txt approach may not attract the attention of humans behind
crawlers.  

Can we think about more active ways to communicate the existence of an
OAI interface once a repository has been visited by a crawler?  

cheers

herbert van de sompel