[OAI-implementers] Handling set membership changes
Sat, 17 Jan 2004 01:34:48 -0500
So, I was looking at what it would take to get Fedora's OAI provider interface to support sets for selective harvesting and I realized that communicating set membership changes to harvesters could be trouble.
Is there a sane way to deal with this? The only thing I can think of is to just treat the set membership change as a record modification, but this leaves certain harvesters in the dark. They will believe the record still exists in its prior set forever and hasn't been modified forever, unless:
They are smart enough to recognize a setSpec change in a header, and
a) They are lucky enough be selectively harvesting
on one of the new sets the record is in, or
b) They only do incremental (not set-selective) harvesting.
They periodically dump everything and do a full harvest.
If I know up front that set membership is apt to change, should I:
1) Not support sets in my provider.
2) Support sets, but have a policy that if someone is harvesting
from me, they agree to either:
i) periodically dump everything and do a full harvest, or
ii) never do set-selective harvesting
3) Something else
Any recommendations or opinions on this?