This is an aggregated commit of all changes related to the
initial implementation of queue types and on top of that the stream
queue type. The varios commit messages have simply been included mostly
un-edited below.
Make rabbit_amqqueue:not_found_or_absent_dirty/1 visible
For use in the stream plugin.
Use bigger retention policy on max-age test
Set coordinator timeout to 30s
Handle coordinator unavailable error
Handle operator policies as maps when checking if is applicable
Add is_policy_applicable/2 to classic queues
Ignore restart commands if the stream has been deleted
It could happen that after termination some of the monitors are still
up and trigger writer/replica restarts
Policy support on stream queues
Remove subscription events on stream coordinator
Ensure old leaders are removed from monitors
Introduce delay when retrying a failed phase
Note that this ensures monitor is setup, there was a bug where no
monitor was really started when re-trying the same phase
Restart replicas after leader election instead of relying on old monitors
Use timer for stream coordinator retries
Fix stream stats for members/online
Multiple fixes for replica monitoring and restart
Ensure pending commands are appended at the end and re-run
Ensure phase is reset with the state
Remove duplicates from replica list
Restart current phase on state_enter
Remove unused import
Ensure rabbit is running when checking for stream quorum
Restart replicas
Add a close/1 function to queue types
So that we can get a chance of cleaning up resources if needed.
Stream queues close their osiris logs at this point.
fix compiler errors
stream-queue: take retention into account
When calculating ready messages metrics.
Add osiris to the list of rabbit deps
Retry restart of replicas
Do not restart replicas or leaders after receiving a delete cluster command
Add more logging to the stream coordinator
Monitor subscribed processes on the stream coordinator
Memory breakdown for stream queues
Update quorum queue event formatter
rabbit_msg_record fixes
Refactor channel confirms
Remove old unconfirmed_messages module that was designed to handle
multiple queue fan in logic including all ha mirrors etc. Replaced with
simpler rabbit_confirms module that handles the fan out and leaves any
queue specific logic (such as confirms from mirrors) to the queue type
implemention. Also this module has a dedicated test module. Which is
nice.
Backward compatibility with 3.8.x events
Supports mixed version cluster upgrades
Match specification when stream queue already exists
Max age retention for stream queues
Stop all replicas before starting leader election
stream: disallow global qos
remove IS_CLASSIC|QUORUM macros
Ensure only classic queues are notified on channel down
This also removes the delivering_queues map in the channel state as it
should not be needed for this and just cause additional unecessary
accounting.
Polish AMQP 1.0/0.9.1 properties conversion
Support byte in application properties, handle 1-bit representation for
booleans.
Use binary in header for long AMQP 1.0 ID
Fix AMQP 1.0 to 0.9.1 conversion
Fix test due to incorrect type
Convert timestamp application properties to/from seconds
AMQP 1.0 uses milliseconds for timestamp and AMQP 0.9.1 uses seconds, so
conversion needed.
Dialyzer fixes
Handle all message-id types
AMQP 1.0 is more liberal in it's allowed types of message-id and
correlation-id - this adds headers to describe the type of the data in
the message_id / correlation_id properties and also handles the case
where the data cannot fit by again using headers.
Resize stream coordinator cluster when broker configuration changes
convert timestamp to and fro seconds
user_id should be a binary
message annotations keys need to be symbols
stream-queue: default exchange and routing key
As these won't be present for data written using the rabbitmq-stream
plugin.
Add exchange, routing key as message annotations
To the AMQP 1.0 formatted data to enable roundtrip.
Add osiris logging module config
And update logging config test suite.
Restart election when start of new leader fails
The node might have just gone down so we need to try another one
Only aux keeps track of phase now, as it might change if the leader election fails
Stream coordinator refactor - all state is kept on the ra machine
Ensure any ra cluster not a qq is not cleaned up
Fixes to recovery and monitoring
Add AMQP 1.0 common to dependencies
Add rabbit_msg_record module
To handle conversions into internal stream storage format.
Use rabbitmq-common stream-queue branch
Use SSH for osiris dependency
Stream coordinator: delete replica
Stream coordinator: add replica
Stream coordinator: leader failover
Stream coordinator: declare and delete
Test consuming from a random offset
Previous offsets should not be delivered to consumers
Consume from stream replicas and multiple test fixes
Use max-length-bytes and add new max-segment-size
Use SSH for osiris dependency
Basic cancel for stream queues
Publish stream queues and settle/reject/requeue refactor
Consume from stream queues
Fix recovery
Publish stream messages
Add/delete stream replicas
Use safe queue names
Set retention policy for stream queues
Required by the ctl command
[#171207092]
Stream queue delete queue
fix missing callback impl
Stream queue declare
Queue type abstraction
And use the implementing module as the value of the amqqueue record
`type` field. This will allow for easy dispatch to the queue type
implementation.
Queue type abstraction
Move queue declare into rabbit_queue_type
Move queue delete into queue type implementation
Queue type: dequeue/basic_get
Move info inside queue type abstraction
Move policy change into queue type interface
Add purge to queue type
Add recovery to the queue type interface
Rename amqqueue quorum_nodes field
To a more generic an extensible opaque queue type specific map.
Fix tests and handle classic API response
Fix HA queue confirm bug
All mirrors need to be present as queue names. This introduces context
linking allowing additional queue refs to be linked to a single "master"
queue ref contining the actual queue context.
Fix issue with events of deleted queues
Also update queue type smoke test to use a cluster by default.
correct default value of amqqueue getter
Move classic queues further inside queue type interface
why
[TrackerId]
Dialyzer fixes