[OAI-implementers] Re: OAI-implementers Digest, Vol 30, Issue 4
mdiggory at MIT.EDU
Tue Mar 27 13:12:42 EDT 2007
On Mar 27, 2007, at 1:00 PM, oai-implementers-
request at openarchives.org wrote:
> Torsten Schaßan wrote:
>> Dear all,
>> today in my institution we had a long discussion about the
>> implementation of an OAI interface and the possible dangers of
>> OAI: how
>> to prevent denial-of-service-like numbers of harvester requests?
>> What experiences do you have with such things as OAI doesn't have any
>> precaution to prevent something like this? Did you ever experience
>> problems for your servers or are there any measures we could
>> and which we have overlooked?
If your using Apache 2.x you should make mod_cband your friend. You
can throttle your harvesters with it.
> 5. Flow Control, Load Balancing and Redirection
> It is essential that harvesting software respect flow control
> responses from repositories. Not doing so may turn a harvest
> attempt into a denial-of-service attack on the repository.
> Repositories which issue 503 Service Unavailable HTTP replies as a
> means of flow control should include a Retry-After HTTP header to
> indicate how long a harvester should wait before issuing the
> request again. Harvesters that encounter a 503 reply without a
> Retry-After header should not automatically retry without
> considerable delay (minutes) or, preferably, manual intervention.
> Harvesters must not be written to retry indefinitely.
> Either as part of a load balancing strategy or for other reasons, a
> repository may issue 302 Found HTTP replies to redirect the
> harvester to another URL indicated in a Location HTTP header.
> Harvesters that encounter a 302 reply without a Location header
> should not automatically retry the request.
Mark R. Diggory - DSpace Systems Manager
MIT Libraries, Systems and Technology Services
Massachusetts Institute of Technology
Phone: (617) 253-1096
More information about the OAI-implementers