credit_flow-like throttling of queue->writer message flow

Previously we effectively had a credit_spec of {100,1}, i.e. the queue
would send up to 100 messages to a consumer channel/writer, and the
writer would 'ack' them individually. That is horrendeously
inefficient:
- when draining a queue, after the queue had sent 100 messages it
would block the consumer, unblock when the notify_sent 'ack' came in,
send another message to the channel/queue, block again. So a vast
amount of work per message
- in a cluster, the notify_sent 'acks' effectively doubled the
cross-cluster traffic

We now use a scheme much like credit_flow. Except we cannot *actually*
use credit_flow because
- rather than wanting to know whether a sender is lacking credit for
*any* receiver, as indicated by credit_flow:blocked(), we need to know
*which* receiver we are lacking credit for.
- (lack of) credit from receiver should *not* propagate to senders,
i.e. sender and receiver credits are completely decoupled. Instead the
queue should, er, queue messages when receivers cannot keep up.

While we could modify credit_flow to accomodate the above, the changes
would be quite unpleasant and not actually reduce the amount of code
vs implementing a more specialised scheme.

The downside is that the contract for using
rabbit_amqqueue:notify_sent becomes somewhat mysterious. In
particular it sets up a monitor for queues in the caller, and expects
the caller to invoke rabbit_amqqueue:notify_sent_queue_down when a
'DOWN' message is received.

0 files changed, 0 insertions, 0 deletions