summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* quorum_queue_utils: wait_for_messages/2 now waits up to 30 secondsavoid-double-channel-count-decreaseJean-Sébastien Pédron2020-10-131-1/+1
| | | | | ... instead of 5 seconds. Hopefully, this should reduce the chance of failures in CI.
* rabbit_channel_tracking: Don't update channel tracking directly when a ↵Jean-Sébastien Pédron2020-10-131-2/+2
| | | | | | | | | | | | | | connection is closed Tracking is already taken care of when each channel is closed. Removing this concurrent call to `unregister_tracked()` fixes a double-decrease of the counter. The reason is the counter is updated outside of a transaction and is therefore sensitive to concurrent read/modify/write sequences. A transaction would be a more appropriate solution, but it may be too resources-intensive in the scenario of high connection/channel churn.
* per_user_connection_channel_tracking_SUITE: Wait for conditionsJean-Sébastien Pédron2020-10-131-29/+30
| | | | | | | | | ... instead of checking it just after the channel/connection was closed. The problem with closing those resources is that the metric may be updated after the close function returns. There the check is time-sensitive. Waiting for the expected counter value fixes the problem.
* many_node_ha_SUITE: Limit timetrap to 5 minutesJean-Sébastien Pédron2020-10-131-0/+5
|
* Travis CI: Update config from rabbitmq-commonJean-Sébastien Pédron2020-10-131-3/+3
|
* Fix start offset for first offset consumingArnaud Cogoluègnes2020-10-091-0/+1
|
* Merge pull request #2462 from rabbitmq/rabbitmq-management-844Michael Klishin2020-10-072-11/+72
|\ | | | | Several new functions for rabbit_alarms
| * Switch to new CT helpersrabbitmq-management-844Michael Klishin2020-10-061-4/+8
| |
| * Correct a Dialyzer specMichael Klishin2020-10-061-1/+1
| |
| * Introduce rabbit_alarm:get_local_alarms/{0,1}Michael Klishin2020-10-051-2/+20
| | | | | | | | Part of rabbitmq/rabbitmq-management#844
| * Introduce rabbit_alarm:format_as_map/1, format_as_maps/1Michael Klishin2020-10-051-4/+41
| |
| * Introduce rabbit_alarms:get_alarms/1Michael Klishin2020-10-051-4/+6
| |
* | Deleted queue should not crash channelkjnilsson2020-10-071-10/+25
| | | | | | | | | | | | This handles the case where the channel receives a message from a queue that has recently been deleted. This crash would only occur when running in a mixed versions cluster.
* | Fix mixed-versions test assertionkjnilsson2020-10-071-10/+15
| | | | | | | | To wait for both new and old confirm message formats.
* | Move rabbit_misc:confirm_to_sender/3kjnilsson2020-10-073-29/+30
| | | | | | | | out of rabbit_common as it is only used by modules in the server.
* | Backwards compatibilty fixeskjnilsson2020-10-073-11/+44
| | | | | | | | | | | | As some classic queue messages changed during the queue type refactoring we need to check the stream_queue feature flag to ensure we return the appropriate format to the interacting channel.
* | queue_type_SUITE setup fixkjnilsson2020-10-061-14/+14
| |
* | Fix credit flow related HA queue regressionkjnilsson2020-10-065-18/+32
|/ | | | When a down mirror would not clear credit flow.
* Workaround for dialyzer errorPhilip Kuryloski2020-10-051-1/+1
| | | | | | | | When erl_epmd:port_please/3 is called with a string(), dialyzer reports an error that atom() is the only allowed type. https://github.com/erlang/otp/blob/e8a9158ed2e686b1fa375cd0978e50d6bd70d088/lib/kernel/src/erl_epmd.erl#L136 Is suspected as the underlying issue, as the while the argument is ignored, the spec is not in alignment with the remainder of the module.
* GitHub Actions: Regen workflowsJean-Sébastien Pédron2020-10-052-633/+1401
|
* Dialyzer fix for rabbit_amqqueue:deliver/2kjnilsson2020-10-051-1/+2
|
* Merge pull request #2459 from rabbitmq/restore-rabbit_amqqueue-deliverKarl Nilsson2020-10-051-1/+6
|\ | | | | Restore rabbit_amqqueue:deliver/2
| * Restore rabbit_amqqueue:deliver/2restore-rabbit_amqqueue-deliverPhilip Kuryloski2020-10-051-1/+6
| | | | | | | | For rabbit_exchange_type_recent_history and any other plugins that use it
* | Backwards compatibility fixeskjnilsson2020-10-054-4/+28
| | | | | | | | For classic queues mostly.
* | rabbit_core_ff: Declare that `stream_queue` depends on `quorum_queue`Jean-Sébastien Pédron2020-10-051-0/+1
| | | | | | | | | | | | The `quorum_queue` feature flag introduces the new v2 #amqqueue{} record which allows to specify a queue type. The stream queue implementation relies on this new record.
* | amqqueue: Pass `Type` to amqqueue_v1:new/9Jean-Sébastien Pédron2020-10-051-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | amqqueue_v1:new/9 only accepts the classic queue type. Therefore, by passing the type, we ensure that the request to create a v1 record with a non-classic queue type will crash. Before this change, amqqueue:new/9 with a non-classic queue type would return "successfully" a classic queue record, effectively ignoring the queue type argument. This should help to detect missing dependency between feature flags. For instance, the `stream_queue` feature flag would depend on the `quorum_queue` feature flag because the latter introduced the v2 record (the one with the `type` record member). Without this change, it means that a user could request a new stream queue (after enabling the `stream_queue` feature flag), but a classic queue would be created instead. There would be no error. Now, with this change, the request will crash. The same user can report the bug: he enabled the `stream_queue` feature flag but the declare crashes.
* | rabbit_amqqueue: Broaden the scope of the "cannot declare because disabled ↵Jean-Sébastien Pédron2020-10-051-1/+1
|/ | | | | | | | | | feature flag" error message Initially, there was only the `quorum_queue` feature flag which made a new queue type available. Now we also have stream queues and they have their own feature flag. Therefore, rephrase the error message to not be quorum-queue-specific.
* Do not run queue_parallel agains stream queuekjnilsson2020-10-021-2/+1
| | | | As it does not support most of the features.
* Fix queue master locator regressionkjnilsson2020-10-021-1/+1
|
* Quorum Queue reject publish bug fixkjnilsson2020-10-023-37/+34
|
* Fix crash bug when consumers are removedkjnilsson2020-10-023-22/+24
| | | | | And the queue is shut down. This is a regression introduced by the queue type refactoring.
* Publish confirm fixeskjnilsson2020-10-011-22/+41
| | | | | Some subtle behavour around classic queue confirms got misunderstood in the refactoring.
* Merge pull request #2347 from rabbitmq/stream-queueKarl Nilsson2020-09-3050-1668/+6572
|\ | | | | Stream queue
| * Dialyzer fixstream-queuekjnilsson2020-09-301-1/+2
| |
| * Stream Queuekjnilsson2020-09-3050-1668/+6571
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an aggregated commit of all changes related to the initial implementation of queue types and on top of that the stream queue type. The varios commit messages have simply been included mostly un-edited below. Make rabbit_amqqueue:not_found_or_absent_dirty/1 visible For use in the stream plugin. Use bigger retention policy on max-age test Set coordinator timeout to 30s Handle coordinator unavailable error Handle operator policies as maps when checking if is applicable Add is_policy_applicable/2 to classic queues Ignore restart commands if the stream has been deleted It could happen that after termination some of the monitors are still up and trigger writer/replica restarts Policy support on stream queues Remove subscription events on stream coordinator Ensure old leaders are removed from monitors Introduce delay when retrying a failed phase Note that this ensures monitor is setup, there was a bug where no monitor was really started when re-trying the same phase Restart replicas after leader election instead of relying on old monitors Use timer for stream coordinator retries Fix stream stats for members/online Multiple fixes for replica monitoring and restart Ensure pending commands are appended at the end and re-run Ensure phase is reset with the state Remove duplicates from replica list Restart current phase on state_enter Remove unused import Ensure rabbit is running when checking for stream quorum Restart replicas Add a close/1 function to queue types So that we can get a chance of cleaning up resources if needed. Stream queues close their osiris logs at this point. fix compiler errors stream-queue: take retention into account When calculating ready messages metrics. Add osiris to the list of rabbit deps Retry restart of replicas Do not restart replicas or leaders after receiving a delete cluster command Add more logging to the stream coordinator Monitor subscribed processes on the stream coordinator Memory breakdown for stream queues Update quorum queue event formatter rabbit_msg_record fixes Refactor channel confirms Remove old unconfirmed_messages module that was designed to handle multiple queue fan in logic including all ha mirrors etc. Replaced with simpler rabbit_confirms module that handles the fan out and leaves any queue specific logic (such as confirms from mirrors) to the queue type implemention. Also this module has a dedicated test module. Which is nice. Backward compatibility with 3.8.x events Supports mixed version cluster upgrades Match specification when stream queue already exists Max age retention for stream queues Stop all replicas before starting leader election stream: disallow global qos remove IS_CLASSIC|QUORUM macros Ensure only classic queues are notified on channel down This also removes the delivering_queues map in the channel state as it should not be needed for this and just cause additional unecessary accounting. Polish AMQP 1.0/0.9.1 properties conversion Support byte in application properties, handle 1-bit representation for booleans. Use binary in header for long AMQP 1.0 ID Fix AMQP 1.0 to 0.9.1 conversion Fix test due to incorrect type Convert timestamp application properties to/from seconds AMQP 1.0 uses milliseconds for timestamp and AMQP 0.9.1 uses seconds, so conversion needed. Dialyzer fixes Handle all message-id types AMQP 1.0 is more liberal in it's allowed types of message-id and correlation-id - this adds headers to describe the type of the data in the message_id / correlation_id properties and also handles the case where the data cannot fit by again using headers. Resize stream coordinator cluster when broker configuration changes convert timestamp to and fro seconds user_id should be a binary message annotations keys need to be symbols stream-queue: default exchange and routing key As these won't be present for data written using the rabbitmq-stream plugin. Add exchange, routing key as message annotations To the AMQP 1.0 formatted data to enable roundtrip. Add osiris logging module config And update logging config test suite. Restart election when start of new leader fails The node might have just gone down so we need to try another one Only aux keeps track of phase now, as it might change if the leader election fails Stream coordinator refactor - all state is kept on the ra machine Ensure any ra cluster not a qq is not cleaned up Fixes to recovery and monitoring Add AMQP 1.0 common to dependencies Add rabbit_msg_record module To handle conversions into internal stream storage format. Use rabbitmq-common stream-queue branch Use SSH for osiris dependency Stream coordinator: delete replica Stream coordinator: add replica Stream coordinator: leader failover Stream coordinator: declare and delete Test consuming from a random offset Previous offsets should not be delivered to consumers Consume from stream replicas and multiple test fixes Use max-length-bytes and add new max-segment-size Use SSH for osiris dependency Basic cancel for stream queues Publish stream queues and settle/reject/requeue refactor Consume from stream queues Fix recovery Publish stream messages Add/delete stream replicas Use safe queue names Set retention policy for stream queues Required by the ctl command [#171207092] Stream queue delete queue fix missing callback impl Stream queue declare Queue type abstraction And use the implementing module as the value of the amqqueue record `type` field. This will allow for easy dispatch to the queue type implementation. Queue type abstraction Move queue declare into rabbit_queue_type Move queue delete into queue type implementation Queue type: dequeue/basic_get Move info inside queue type abstraction Move policy change into queue type interface Add purge to queue type Add recovery to the queue type interface Rename amqqueue quorum_nodes field To a more generic an extensible opaque queue type specific map. Fix tests and handle classic API response Fix HA queue confirm bug All mirrors need to be present as queue names. This introduces context linking allowing additional queue refs to be linked to a single "master" queue ref contining the actual queue context. Fix issue with events of deleted queues Also update queue type smoke test to use a cluster by default. correct default value of amqqueue getter Move classic queues further inside queue type interface why [TrackerId] Dialyzer fixes
* Merge pull request #2457 from rabbitmq/lrb-update-supervisorMichael Klishin2020-09-301-83/+0
|\ | | | | Supervisor2 tests have moved to rabbitmq-common
| * Move supervisor2_SUITE to rabbit_common (next to the module under test)Michael Klishin2020-09-301-128/+0
| |
| * erlfmt changeslrb-update-supervisorLuke Bakken2020-09-291-24/+25
| |
| * Add test for intrinsic supervisor2 typeLuke Bakken2020-09-292-83/+127
| | | | | | | | Part of rabbitmq/rabbitmq-common#411
* | Merge pull request #2456 from rabbitmq/lrb-fix-syslog-loggingMichael Klishin2020-09-301-1/+2
|\ \ | |/ |/| Explicitly start syslog application
| * configure_handler_backend/1: return a value expected by the callerMichael Klishin2020-09-301-1/+2
| |
| * Use application:ensure_all_started/1 to start sysloglrb-fix-syslog-loggingMichael Klishin2020-09-291-1/+1
| |
| * Explicitly start syslog applicationLuke Bakken2020-09-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Between RabbitMQ 3.8.3 and 3.8.4 logging to `syslog` broke because the `syslog` OTP application was no longer started when needed. This is probably due to changes in #2180. This change explicitly starts the `syslog` app if the backend is configured. Found while investigating this issue: https://groups.google.com/forum/?oldui=1#!topic/rabbitmq-users/-aqS2BiHm4k
* | Merge pull request #2451 from rabbitmq/qq-consumer-prioritiesMichael Klishin2020-09-294-26/+151
|\ \ | |/ |/| Quorum queue consumer priorities
| * Quorum queue consumer prioritiesqq-consumer-prioritieskjnilsson2020-09-254-26/+151
| | | | | | | | | | | | This switches the service queue inside rabbit_fifo from a normal queue to a priority queue such that consumers with a higher priority are favoured for service.
* | Merge pull request #2453 from rabbitmq/shrink-when-downMichael Klishin2020-09-291-0/+5
|\ \ | | | | | | Shrink quorum queue must succeed if the target node is down or rabbit is stopped
| * | Pass stricter Dialyzer on OTP 23Michael Klishin2020-09-291-2/+0
| | |
| * | Shrink quorum queue must succeed if the target node is down or rabbit is stoppedshrink-when-downdcorbacho2020-09-251-0/+7
| | | | | | | | | | | | | | | Avoids false negatives when shrink gets included in forget_cluster_node command, as at least the rabbit app is down
* | | Merge pull request #2455 from rabbitmq/rabbitmq-server-2421Michael Klishin2020-09-282-2/+5
|\ \ \ | | | | | | | | Exclude consumers suspected as down from the consumer count
| * | | Update rabbit_fifo consumers test to exclude suspected down ones from countdcorbacho2020-09-281-1/+1
| | | |
View Lorry Controller Status