summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* the queue index deletes transient msgs on initialisation. This is rather ↵Matthew Sackman2009-10-141-6/+17
| | | | elegant because it means that the delta gen fun used to seed the msg store does not generate any deltas for transient msgs, which means that the msg_store will take care of deleting transient msgs without any further interaction.
* Wire up the tx_commit callback. Note no coalescing because no sync on ↵Matthew Sackman2009-10-142-3/+10
| | | | queue_index because no fhc. Also added notes about the deletion of queues on startup.
* Be a bit more clever on starting the message store and look up durable ↵Matthew Sackman2009-10-134-37/+76
| | | | queues. However, it's still not quite right because queues can be deleted by other nodes in the cluster between extracting the list of durable queues in msg_store startup and the startup of queues themselves. This means that we can end up seeding the msg_store with msgs from queues that won't actually start up. You might think that we'd be saved by the fact that _process:terminate deletes the queue, but no, because fwics, _process isn't trapping exits, meaning that the terminate won't be called (and most likely rightly so so that it doesn't upset mnesia's state) by the exit(Pid, shutdown) in amqqueue:recover. So there still needs to be some sort of fix somewhere and somehow
* most of the rewiring is done. Need to sort out how to delete non durable ↵Matthew Sackman2009-10-136-1538/+73
| | | | queues on start up, which is a bit cyclical, as I'd like to not start the msg_store until we know which queues are durable and which aren't, but we also can't start the queues until the msg_store is running. Fun.
* added publish_delivered/2 which deals with adding a message, when the queue ↵Matthew Sackman2009-10-131-1/+17
| | | | is empty which we already know has been sent out to a consumer, so it's really just a case of writing to disk the message, and index pub and deliver entries iff the message is persistent
* publish/3 -> publish/2 because the IsDelivered bit is always false when ↵Matthew Sackman2009-10-131-14/+32
| | | | called externally. Also rework requeue, because with the ability to indicate that persistent msgs will already be in msg_store, we don't need to call msg_store:write for persistent msgs, which means that we can also avoid the call to msg_store:remove that would have happened in the call to ack
* it starts up againMatthew Sackman2009-10-133-5/+9
|
* Added ability to seed the msg_store. It's a bit hairy.Matthew Sackman2009-10-131-5/+57
|
* Maybe done. Maybe time to start hacking out mq and dq. Maybe.Matthew Sackman2009-10-132-22/+35
|
* factorisation and application of DeMorganMatthew Sackman2009-10-131-16/+14
|
* in preparation for commit, need to be able to indicate in a publish that ↵Matthew Sackman2009-10-121-22/+62
| | | | persistent msgs have already been sent to disk
* cosmeticMatthew Sackman2009-10-121-5/+6
|
* tx_publish and tx_rollback done. Only tx_commit to go...Matthew Sackman2009-10-121-1/+12
|
* Requeue turns out to be vastly more elegant and predictable than in mq.Matthew Sackman2009-10-121-1/+11
|
* implemented delete. This is slightly less than pretty as, after doing the ↵Matthew Sackman2009-10-121-9/+38
| | | | purge, we have to walk through the index on disk in order to pull up msgs which have been delivered and not acked
* for queue delete I need to be able to discover the lowest and highest ↵Matthew Sackman2009-10-122-37/+38
| | | | non-acked seqids thus splitting out this functionality
* a couple of full list sorts were unnecessary and should only have been max ↵Matthew Sackman2009-10-121-7/+8
| | | | or mins
* Well, it was beautiful, but it was also wrong. Firstly, purge needs to ↵Matthew Sackman2009-10-121-20/+48
| | | | return the count of the msgs purged. Secondly, it needs to remember to purge msgs and indices on disk that are in q1 or q4.
* given the way in which the vq works, the removed tests were unnecessary. For ↵Matthew Sackman2009-10-121-7/+7
| | | | example, in store_alpha, q1 *must* be empty unless one of q2, gamma and q3 is non empty. As such, to determine whether the alpha goes to q4, we only need to test for emptiness of q2, gamma and q3, not q1 aswell. Similar logic holds for store_beta
* merging in from bug 21368Matthew Sackman2009-10-122-80/+59
|\
| * rollback transactions on queue terminationMatthias Radestock2009-10-112-80/+59
| | | | | | | | | | | | | | | | | | | | That way we don't leave garbage - transactionally published, but uncommitted messages - in the message store. Also, we we can get rid of the pending_commits state wart in disk_queue. That is possible because both tx commits and queue deletions are issued by the queue process and tx commits are synchronous, so there is never a chance of there being a pending commit when doing a deletion.
* | Beautiful!Matthew Sackman2009-10-091-1/+40
| |
* | cosmeticsMatthew Sackman2009-10-093-23/+31
| |
* | Added ack to vq. Realised that ack no longer needs the msg itself, so ↵Matthew Sackman2009-10-092-11/+25
| | | | | | | | there've been a few associated changes in queue_process.
* | gee, can you tell none of this has been run yet?Matthew Sackman2009-10-091-1/+2
| |
* | tidying of the beloved msg_storeMatthew Sackman2009-10-092-27/+29
| |
* | Prefetcher has priority over q1.Matthew Sackman2009-10-093-17/+43
| | | | | | | | However, we actually assume that the prefetcher does no work. The only other possibility is to assume that the prefetcher always completes, which can lead to q1 being pointlessly evicted to disk. Also, we stop the prefetcher as soon as have to reduce our memory usage, and at that point everything should work out. So, when starting the prefetcher, we don't adjust the ram_msg_count, but as we drain or stop the prefetcher, we include the prefetched alphas in the ram_msg_count and then evict q1 to disk as necessary. Whilst this means we will have more msgs in RAM then we claim, the fact that we stop the prefetcher as soon as we have to reduce our memory usage saves us from ruin.
* | switching back to lists:min and lists:max as erlang:min/max do not exist in R12Matthew Sackman2009-10-093-4/+4
| |
* | altered the prefetcher to drive the msg_store directly, and wired into vqMatthew Sackman2009-10-093-104/+152
| |
* | having discovered that erlang:min and erlang:max do exist, swap them in in ↵Matthew Sackman2009-10-092-3/+3
| | | | | | | | place of lists:min and lists:max where appropriate
* | adding len and is_empty. Renaming in -> publish and out -> fetch. A bit of ↵Matthew Sackman2009-10-091-65/+86
| | | | | | | | moving around. Generally trying to work towards the mq API so that it can be dropped in
* | on out/1, if the msg or its index are on disk and they don't need to be, ↵Matthew Sackman2009-10-081-12/+32
| | | | | | | | take them off disk
* | And now actually make sure that on start up we load in the first segment ↵Matthew Sackman2009-10-081-39/+45
| | | | | | | | from gamma to q3 if possible. Hence a refactoring here.
* | The calculation of the highest seq id in the index queue was wrong - it was ↵Matthew Sackman2009-10-082-42/+66
| | | | | | | | simply returning the highest unacked seq id, instead of looking for the highest seq id ever encountered. This could have led to reuse of seq ids. We also need to know the total message count in the queue index which is the number of unacked msgs recorded in the index, and we also need the seq id of the segment boundary of the segment containing the first msg in the queue. This is so that we can form the inital gamma correctly.
* | minor further corrections and modificationsMatthew Sackman2009-10-081-18/+21
| |
* | merging in from bug 21368Matthew Sackman2009-10-081-1/+1
|\ \ | |/
| * fixing erroneous use of thunk(_) in spec to fun()Matthew Sackman2009-10-081-1/+1
| |
* | Tidying and refactoring of the variable queue, some documentation, and the ↵Matthew Sackman2009-10-082-174/+273
| | | | | | | | removal of a lot of algorithmic bugs. No real new features, but code in much better state.
* | merging in from bug 21368Matthew Sackman2009-10-085-131/+158
|\ \ | |/
| * cosmeticMatthias Radestock2009-10-081-8/+5
| |
| * make msg_store responsible for sync'ingMatthias Radestock2009-10-082-130/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The API to the msg_store has changed: now instead of asking whether a sync is needed for a set of msg ids, and subsequently requesting a sync, we request a sync for a set of msg ids and supply a callback that is invoked when that sync is done. That way the msg_store can make its own decisions on when to sync, and less logic is required by callers. During queue deletion we must remove *all* queue messages from the store, including those that are part of committed transactions for which the disk_queue has not yet received the sync callback. To do that we keep a record of these messages in a dict in the state. The dict also ensures that we do not act on a sync callback involving a queue which has since been deleted and perhaps recreated.
| * merge default into bug21368Matthias Radestock2009-10-0717-92/+194
| |\
| | * Added tag rabbitmq_v1_7_0 for changeset b1089fcc31b7David Wragg2009-10-070-0/+0
| | |
| | * Bump version to 1.7.0rabbitmq_v1_7_0David Wragg2009-10-072-0/+9
| | |
| | * Merge QAd bug21738 into defaultMichael Bridgen2009-10-051-1/+4
| | |\
| | | * fix race condition in list_connections test when in a clusterMatthias Radestock2009-10-021-1/+4
| | | |
* | | | implemented out. This is getting pretty disgusting, needs some refactoring, ↵Matthew Sackman2009-10-072-29/+164
| | | | | | | | | | | | | | | | marginally more useful variable names, and more API, in particular proper support for the prefetcher. Also, totally untested.
* | | | More work on variable queueMatthew Sackman2009-10-072-43/+174
| | | |
* | | | initial hacking on the variable queueMatthew Sackman2009-10-071-0/+182
| | | |
* | | | Make init include in its result the next sequence id to useMatthew Sackman2009-10-071-15/+33
| | | |
View Lorry Controller Status