I have a program running that collects metadata from the internets.
Every so often a summary of this data is written up (every 30 min or so) and the summary is posted to a database.
The summary is also sent out to many web service APIs.
I’m trying to avoid the problem of this program locking up when webservice APIs timeout. So I’m thinking of creating a queue to process these webservice requests.
How should I create this queue? With a database of some sort? It could be a program storing queued items in memory, I suppose. Is there some standard process for creating a queue for webservice calls?