Agressively check caches before risking deferring to the msg_store. This often is very beneficial because in practise, the last messages to be pushed to disk under memory pressure are the messages at the head of the queue, not the tail. Thus in the event of the msg_store being overloaded, due to pushing to disk due to memory pressure, checking all the caches first is very effective at keeping the queues moving. The downside is that when we read from the cur_file cache, we don't get the ref count (at this point, the write is maybe only in the mailbox of the msg_store, so it may not even have an index entry, thus we have no chance of knowing an accurate refcount). The result is that the message may end up existing having been read from the cur file cache, then pushed to disk, forgotten, and read in again, only to be added to the dedup cache. However, this is as bad as it can get - there could at most be two binary copies of a message in memory. This actually holds, even though there's a slightly amazing interaction on ack: on ack, the copy read from the cur_file cache will decrement the dedup cache, possibly evicting the entry. At this point there would be no copies in the dedup cache, maybe one copy in the cur_file cache, and maybe one copy in RAM, which is the copy read from the dedup cache. The copy read from the cur_file cache has been ack'd, so has gone. The next read, if there is no copy in the cur_file cache may add to the dedup cache, which will then be emptied on the ack from the first dedup read. I.e. you can achieve a chain in which no more than 2 copies are held in RAM at any one time, but each read really goes to disk. It is left as an exercise to the reader to come up with a test case that exhibits this behaviour.

0 files changed, 0 insertions, 0 deletions