summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* minor tidying up: use symbolic record indices instead of numeric onesMatthias Radestock2009-09-081-7/+8
|
* api tweak: pass durable queues in list rather than setMatthias Radestock2009-09-082-5/+4
| | | | | lists are more pleasant than sets in APIs, plus in our use we have a list to start with.
* change guid to a binary, using the md5 of term_to_binaryMatthias Radestock2009-09-085-39/+44
| | | | | | | | | The main motivation is to reduce the memory and on-disk footprint of the guid from ~34 bytes to 16. But it turns out that this actually results in a speed improvement of a few percent as well, even for non-persistent messaging, presumably due to the memory management effects and the fact that 16 byte binaries are easier to copy between processes than the deep(ish) original guid structure.
* refactoring: move dets/ets match into sort_msg_locations_by_offsetMatthias Radestock2009-09-071-18/+9
|
* rename prune_mnesia to pruneMatthias Radestock2009-09-071-10/+10
| | | | since we prune more than just mnesia
* shut up dialyzerMatthias Radestock2009-09-071-3/+4
|
* refactoring of delete_empty_filesMatthias Radestock2009-09-071-23/+26
| | | | | giving it a name that describes what it does, and extracting recursion vestiges into caller.
* generalise persistent flag to message attributes in rabbit_msg_storeMatthias Radestock2009-09-072-21/+19
| | | | | This is just a renaming exercise, but it turns rabbit_msg_store into a general purpose message store.
* generalise persistent flag to message attributes in rabbit_msg_fileMatthias Radestock2009-09-071-50/+43
|
* resolve msg_id type confusionMatthias Radestock2009-09-075-9/+14
| | | | rabbit_mixed_queue and rabbit_disk_queue see guids, not non_neg_integers.
* perform term/binary conversion of msg body in rabbit_msg_fileMatthias Radestock2009-09-072-21/+21
| | | | thus further generalising rabbit_msg_file
* do not return body size from rabbit_msg_file:readMatthias Radestock2009-09-072-12/+15
| | | | | | it's not used anywhere and was cluttering the api Also, make type sigs more meaningful and do not include rabbit.hrl, thus underlining the general nature of this module.
* big refactoring: extract msg_store from disk_queueMatthias Radestock2009-09-062-1157/+1315
| | | | The msg_store knows nothing about queues, or message structure.
* rename #file_summary_entry to #file_summaryMatthias Radestock2009-09-031-30/+30
| | | | to match what we call the containing table
* rename #message_store_entry to #msg_locationMatthias Radestock2009-09-031-38/+38
| | | | to match what we call the containing table
* Switched to using a file to hold the disk_only data. Also found a bug where ↵Matthew Sackman2009-09-031-22/+27
| | | | vaporise was wiping out the disk_only data (both as a file, and when it was in mnesia). The result was that if the dq was in disk_only mode before being vaporised, it would refuse to start up again. Thus vaporise now pushes the queue back to ram_disk mode if necessary, after wiping out the contents of the mnesia table. Finally, all tests pass again.
* the disk queue sometimes needs much longer to shut down safely if it's ↵Matthew Sackman2009-09-031-1/+1
| | | | heavily loaded - up the limit to 5 seconds. However, I suspect something like 60 seconds is more likely to be realistic value
* use a record in file summaryMatthew Sackman2009-09-031-46/+66
|
* ensure that we fsync after copying messages across during compactionMatthew Sackman2009-09-031-1/+1
|
* commented on purpose of cacheMatthew Sackman2009-09-031-0/+11
|
* sort files for compaction numericallyMatthew Sackman2009-09-031-1/+1
|
* Guarantee that any oppressed process reporting 0 memory use will get liberated.Matthew Sackman2009-09-021-3/+4
|
* Made the disk queue start up in the same mode it was last running in.Matthew Sackman2009-09-022-20/+45
| | | | | | This is slightly grim because I have to store some values in the mnesia table which then have to survive all the start up logic, so there are a couple of annoying 1-line changes elsewhere. However, it does indeed work. There was also one bool() -> boolean() fix in the memory_manager.
* factor out sublist and disjoint testsMatthew Sackman2009-09-021-8/+10
|
* Shutdown III: Inline VaporisationMatthew Sackman2009-09-021-7/+3
|
* another attempt at stop/terminate/vaporisation/shutdown/suicide/deathsquadMatthew Sackman2009-09-021-17/+16
|
* renaming of variables.Matthew Sackman2009-09-021-21/+21
|
* A couple of cosmetics - grab_msg_id went away, but tidied up terminate and ↵Matthew Sackman2009-09-021-32/+38
| | | | vaporise - hopefully sufficiently that both Matthias and I are happy with the result\000
* made dq:delete_queue a call, not a cast. This eliminates a race condition ↵Matthew Sackman2009-09-021-4/+5
| | | | between queues being deleted and new queues with the same name being created and published to
* Fixed the mistake Matthias spotted in recovery from crash (comment #211). ↵Matthew Sackman2009-09-021-22/+34
| | | | Tested by deliberately breaking the compaction in a variety of ways and then coverage and startup and ensuring the correct code paths are taken.
* made type signatures almost as useful as commentsMatthew Sackman2009-09-023-11/+19
|
* refactoring of (no)reply1 to eliminate duplicationMatthew Sackman2009-09-011-25/+23
|
* specs for msg_fileMatthew Sackman2009-09-011-0/+16
|
* specs for rabbit_file_handle_cacheMatthew Sackman2009-09-011-0/+21
|
* The multiplier for the memory thresholds should not be on the number of ↵Matthew Sackman2009-09-011-3/+4
| | | | available tokes. In extremis, when available tokens is 0, this makes queues very likely to switch mode (bad, memory is tight), and when there are lots of tokens available, this makes queues less likely to switch mode (bad, memory is plentiful). Instead, we should be using constant offsets, based on the total number of tokens in the system.
* comments on magic 2.5 scaling of memory report, and removal of unnecessary ↵Matthew Sackman2009-09-011-6/+4
| | | | initial calls to report_memory
* Before the magic_marker_msg was introduced, if the queue was entirely on ↵Matthew Sackman2009-09-011-13/+89
| | | | | | disk already and was then told to go to disk_only mode, it would form a single requeue_next_n call, where the N would be the length of the queue. This would be detected by the disk_queue and become a no-op. Because of the introduction of the magic_marker_msg, that is no longer possible - we want the marker to go from the back of the queue to the front. As such, the N will at most be 1 less than the queue length, causing unnecessary work to be done. This patch removes unnecessary work by observing that we do not need to rotate the entire queue should we find that the queue consists of zero or more disk-stored msgs followed by zero or more ram-only stored messages. If this is the case, we only need to publish the latter ram-only messages, and have no need for the magic marker msgs at all. Furthermore, if there are no ram-only messages we have no work to do at all. The only situation in which we must rotate the entire queue is when we have ram-only messages followed by disk messages. In this case, we have to get the ram-only messages onto the disk queue before the disk messages, which requires the full rotation.
* Read through all of Matthias changes, no problems. In here, one cosmetic, ↵Matthew Sackman2009-09-012-9/+5
| | | | and one change so that rmf:read returns the MsgId off disk to be later matched. This is purely defensive.
* refactoring: move all the low level message file i/o into a separate moduleMatthias Radestock2009-08-292-116/+153
| | | | The details of how messages are encoded in files are opaque to disk_queue.
* remove unused constantMatthias Radestock2009-08-291-1/+0
| | | | This was added in revision b48919ed4e12 for no apparent reason
* banish ?FILE_PACKING_ADJUSTMENT from all but three functionsMatthias Radestock2009-08-291-33/+29
| | | | | | | | The details of the message packing are opaque to high level code, as they should be. The TotalSize that code sees now is the total size of the message on disk, including all packing adjustments, which is all that is ever needed to perform all the necessary file positioning etc at that level.
* cosmeticMatthias Radestock2009-08-291-2/+2
|
* cosmetic - mostly whitespace shufflingMatthias Radestock2009-08-291-32/+30
|
* refactoring: rename non-sensical read_file_hc_cache to read_file_handle_cacheMatthias Radestock2009-08-291-9/+9
|
* refactoring: (re)introduce {ok, ...} return on open_fileMatthias Radestock2009-08-291-15/+15
| | | | | | This allows us to replace the last remaining file:open with open_file. Also, extract binary mode settings and use them in both open_file and fh cache creation.
* cosmetic: remove whitespaceMatthias Radestock2009-08-291-21/+18
|
* Other is KOMatthias Radestock2009-08-291-1/+1
| | | | accidentally changed in previous commit
* refactoring: extract reading of stop byteMatthias Radestock2009-08-291-14/+17
|
* cosmeticMatthias Radestock2009-08-291-3/+2
|
* cosmeticMatthias Radestock2009-08-291-5/+3
|
View Lorry Controller Status