summaryrefslogtreecommitdiff
path: root/cpp/src/qpid/cluster
Commit message (Collapse)AuthorAgeFilesLines
* QPID-2702 --max-connections does not work when using cluster.Alan Conway2010-07-231-2/+11
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@967215 13f79535-47bb-0310-9956-ffa450edef68
* Race condition in cluster+management, inconsistent errors like:Alan Conway2010-07-232-23/+46
| | | | | | | | | | | | "confirmed < (2097+0) but only sent < (2096+0)" Management messages are generated if a managed objects properties have changed since the last update. Properties of the cluster object (members and status) were sometimes being changed outside the delivery context which could create inconsistencies in the cluster. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@966933 13f79535-47bb-0310-9956-ffa450edef68
* Fix bug in cluster with authentication: nodes exit with "unauthorized-access"Alan Conway2010-07-204-1/+16
| | | | | | | | | | Adding a node to a cluster on which authentication is enabled and on which there are existing connections authenticated with mechanisms other than anonymous, may result in nodes exiting the cluster with inconsistent authorisation errors. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@965979 13f79535-47bb-0310-9956-ffa450edef68
* Fix cluster-safe assertion in ~Connection.Alan Conway2010-07-081-0/+6
| | | | | | | | Don't trigger cluster-safe asserts in broker::~Connection as it can be called from an IO threads during broker shutdown. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@961814 13f79535-47bb-0310-9956-ffa450edef68
* Defer delivery of messages in cluster-unsafe context.Alan Conway2010-07-052-3/+42
| | | | | | | | | | | | | | | Messages enqueued in a cluster-safe context are synchronized across the cluster. However some messages are delivered in a cluster-unsafe context, for example raising a link established event occurs the connection thread of the establishing connection. This fix deferrs such messages by multicasting them so they can be re-delived in a cluster safe context. See https://bugzilla.redhat.com/show_bug.cgi?id=611543 git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@960681 13f79535-47bb-0310-9956-ffa450edef68
* Fix sporadic cluster core on exit.Alan Conway2010-07-012-1/+1
| | | | | | | | Fix a race condition that allows a cluster::Connection to be deleted via the Cluster's local map before it was fully initialized. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@959752 13f79535-47bb-0310-9956-ffa450edef68
* Fix regression in r956882, sporadic failures of ↵Alan Conway2010-06-242-15/+10
| | | | | | client_test.cpp:testBadClientData git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@957640 13f79535-47bb-0310-9956-ffa450edef68
* Fix cluster broker crashes when management is active.Alan Conway2010-06-224-26/+49
| | | | | | | | | | | | Cluser brokers were exiting with errors "modified cluster state outside cluster context" and "confirmed < (50+0) but only sent < (49+0)" Fix was to: - delay completion of incoming update till update connection closes. - delay addding new connections to managment until connection is announced. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@956882 13f79535-47bb-0310-9956-ffa450edef68
* Bug 603835 - cluster_tests.test_management failing.Alan Conway2010-06-165-33/+48
| | | | | | | | | | | Clean up connections causing extra connection objects in the mangement agent map. - update connection was not being closed. - connections belonging to members that left the cluster were not fully cleaned up Also fixed test errors making failover_soak fail sporadically. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@955370 13f79535-47bb-0310-9956-ffa450edef68
* This change is a partial fix for the problem of long cluster failovers. ↵Michael Goulish2010-06-151-1/+3
| | | | | | | | | | | This change addresses part of that problem: long newbie-broker catchup times. At this time, it looks as though there are two distinct mechanisms causing these long catchup times. They roughly divide into a population of times less than 20 seconds, and a population of much longer times (up to hundreds of seconds). This fix addresses only the shorter times. In a test of 25 failovers before and after the change, it reduced those times by an average of nearly 50%. A T-test on the times indicates that the likelihood of the observed change happening randomly is less than 2%. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@954895 13f79535-47bb-0310-9956-ffa450edef68
* Fix cluster-safe assertion in connection negotiation.Alan Conway2010-06-091-0/+2
| | | | | | | | See https://bugzilla.redhat.com/show_bug.cgi?id=602347. In a cluster, raise the management connect event when processing cluster.announce. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@953147 13f79535-47bb-0310-9956-ffa450edef68
* Cluster handle connection-negotiation phase in local broker.Alan Conway2010-06-083-132/+95
| | | | | | | | | | | | | | | | | The connection negotiation phase up to the "open" or "open-ok" frame establishes whether/what encryption to use for the rest of the connection. With this patch a cluster broker completes the initial negotiation with its local clients and only then begins multicasting to other brokers. The local broker decrypts if necessary and multicasts in the clear. This replaces a problematic locking scheme that was formerly in place which caused deadlocks. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@952692 13f79535-47bb-0310-9956-ffa450edef68
* Fixed sporadic failure of cluster_tests.py sasl_testAlan Conway2010-06-041-1/+2
| | | | | | | - added missing initializer for cluster::Connection::inConnectionNegotiation - brokertest.py: fixed formatting of log excerpt in exceptions. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@951452 13f79535-47bb-0310-9956-ffa450edef68
* QPID-2004: Send disconnected event to any handles still registered after ↵Gordon Sim2010-06-011-2/+4
| | | | | | shutdown to ensure they can clean themselves up git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@950205 13f79535-47bb-0310-9956-ffa450edef68
* Fixed assertion error in cluster_tests.py test_failover.Alan Conway2010-05-312-1/+2
| | | | | | | Added missing call to expandd in cluster/RetractClient.cpp git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@949767 13f79535-47bb-0310-9956-ffa450edef68
* Fixed: authentication with bad credentials causes cluster broker to exit.Alan Conway2010-05-271-2/+12
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@948969 13f79535-47bb-0310-9956-ffa450edef68
* Fix issues with cluster+securityAlan Conway2010-05-271-15/+15
| | | | | | | | - was using "none" not empty string for no ID. - was multicasting secure id for update and shadow connections. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@948967 13f79535-47bb-0310-9956-ffa450edef68
* QPID-2631: For blocking Bounds::expand() calls, only increase the current ↵Gordon Sim2010-05-271-2/+14
| | | | | | count when there is space. In SessionImpl::send() expand bounds before queueing frame. Expand bounds for all frames sent (including connection frames and cluster specific frames). git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@948936 13f79535-47bb-0310-9956-ffa450edef68
* Fix "mismatched cluster-id" errors during start up.Alan Conway2010-05-252-21/+28
| | | | | | | | | Intermittent failure when starting a persistent cluster with all clean stores. Some brokers fail with: critical Unexpected error: Cluster-ID mismatch. Stores belong to different clusters. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@948143 13f79535-47bb-0310-9956-ffa450edef68
* Fix broker core dump during start-up caused by un-initialized mAgent pointer.Alan Conway2010-05-211-0/+1
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@947081 13f79535-47bb-0310-9956-ffa450edef68
* Fix for "Assertion `!mcastSentButNotReceived' failed." in cluster tests.Alan Conway2010-05-171-15/+22
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@945383 13f79535-47bb-0310-9956-ffa450edef68
* Cluster + SecurityMichael Goulish2010-05-146-10/+227
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ----------------------------------- * initial observation of a problem was a 2% failure rate in perftests of 20,000 messages against a cluster with security enabled. Problem was occasional receit of encrypted frames before the security codec had been enabled. This is fixed with locking in cluster code (no new locks in broker code) and a callback that is fired by broker::ConnectionHandler::Handler to tell the cluster code when the opening handshake has finished. This was never a problem in the non-clustered broker before because everything happened in a single thread. * the brokers that "shadow" the connection must not have null authenticators rather than real ones, so that they go through all the motions but don't do anythig. Only the directly-connected broker can perform the security handshake. * once the directly-connected broker receives the real user ID from its callback, it mcasts that ID to all other brokers. Otherwise the shadowing brokers will al think that the user ID is "anonymous". Check this by doing a substantial perftest, and using qpid-stat -c localhost:PORT to confirm that the brokers all have the same userID for the same connection. * the user ID, negotiated during the Sasl security startup, is communicated from the directly connected broker to all other cluster brokers. * If security is *not* being used, then this code should *not* tell the brokers anything about the userID -- or it will step on the value that is being set by other code pathways. * test program at cpp/src/tests/cluster_authentication_soak is not yet fully automated -- run it with something like "sudo ./cluster_authentication_soak 500" git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@944158 13f79535-47bb-0310-9956-ffa450edef68
* Delay generating URL in cluster till global constructors to handle ↵Alan Conway2010-05-121-10/+13
| | | | | | multi-protocol URLs. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@943488 13f79535-47bb-0310-9956-ffa450edef68
* Support for multiple protocols in qpid::Url.Alan Conway2010-05-111-0/+1
| | | | | | | | | | - simplified qpid::Address to hold (protocol,host,port) triples. - protocol plugins call Url:addProtocol to add tags to Url parser. - use Address::protocol when establishing connections. - ssl_test: tests using URL to connect. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@943130 13f79535-47bb-0310-9956-ffa450edef68
* Correct brokertest.retry logic.Alan Conway2010-05-061-1/+1
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@941852 13f79535-47bb-0310-9956-ffa450edef68
* Code cleanup Ted Ross2010-04-232-48/+0
| | | | | | | | | | - Removed IdAllocator (it's no longer needed) - Cleaned up the calls to ManagementAgent::addObject to handle durable objects - Removed the deferred call to addObject for durable objects - Removed unneeded calls to self._checkClosed() in qmf.console.Agent git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@937516 13f79535-47bb-0310-9956-ffa450edef68
* Cluster management improvements:Ted Ross2010-04-231-8/+1
| | | | | | | | | | | 1) Enable all management methods via QMFv2 for clusters 2) Disable all management methods via QMFv1 for clusters 3) The broker-resident management agent can handle both v1 and v2 method calls 4) qmf.console (Python) now works with new and old brokers by detecting whether the broker can handle v2 commands git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@937472 13f79535-47bb-0310-9956-ffa450edef68
* QPID-2527: Remove Thread::id member as its uses are better implemented by ↵Andrew Stitcher2010-04-211-3/+3
| | | | | | | | comparison operators. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@936537 13f79535-47bb-0310-9956-ffa450edef68
* Tidy up unnecessary includes in ConnectionSettings and fix consequencesGordon Sim2010-04-141-0/+1
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@933971 13f79535-47bb-0310-9956-ffa450edef68
* Update cluster store status with a single atomic write() operation.Alan Conway2010-04-013-78/+78
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@930055 13f79535-47bb-0310-9956-ffa450edef68
* Cluster: remove un-necessary code for config-seq.Alan Conway2010-03-317-43/+13
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@929616 13f79535-47bb-0310-9956-ffa450edef68
* Joining a cluster: don't push an empty store.Alan Conway2010-03-301-0/+1
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@929278 13f79535-47bb-0310-9956-ffa450edef68
* Fix hanging failover_soak test.Alan Conway2010-03-302-4/+3
| | | | | | | | | Explictly turn off PollableQueue bypass mode during initialize. Implicitly turning it off in PollableQueue::start() led to incorrectly turning off bypass in a PRE_INIT client when an offer was processed. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@929276 13f79535-47bb-0310-9956-ffa450edef68
* Cluster logging improvements: log config changes in the deliver thread.Alan Conway2010-03-304-38/+50
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@929274 13f79535-47bb-0310-9956-ffa450edef68
* Remove dead PeriodicTimer code.Alan Conway2010-03-302-156/+0
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@929273 13f79535-47bb-0310-9956-ffa450edef68
* Raise ClusterTimer lateness threshold to reduce noisy warnings.Alan Conway2010-03-292-1/+4
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@928779 13f79535-47bb-0310-9956-ffa450edef68
* Fix "Ignoring frame while closing connection" warnings in a cluster broker ↵Alan Conway2010-03-251-1/+4
| | | | | | giving an update. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@927385 13f79535-47bb-0310-9956-ffa450edef68
* More generous overrun threshold for cluster timer to avoid excessive warnings.Alan Conway2010-03-251-1/+5
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@927383 13f79535-47bb-0310-9956-ffa450edef68
* Work-around for race condition in the IO layer.Alan Conway2010-03-151-0/+6
| | | | | | | | | | | | Added a 100ms sleep in UpdateClient between calling Connection::close() and destroying the Connection object. This appears to work around the race condition described in https://bugzilla.redhat.com/show_bug.cgi?id=568831. which was causing brokers to seg fault sporadically at the end of giving an update. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@923414 13f79535-47bb-0310-9956-ffa450edef68
* New cluster member pushes store when joining an active cluster.Alan Conway2010-03-128-63/+178
| | | | | | | | | | | | | | | | | | | Previously a broker with a clean store would not be able to join an active cluster because the shtudown-id did not match. This commit ensures that when a broker joins an active cluster, it always pushes its store regardless of status. Clean/dirty status is only compared when forming an initial cluster. This change required splitting initialization into two phases: PRE_INIT: occurs in the Cluster ctor during early-initialize. This phase determines whether or not to push the store. INIT: occurs after Cluster::initialize and does the remaining initialization chores. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@922412 13f79535-47bb-0310-9956-ffa450edef68
* QPID-2436: Fix cluster update of remote agents.Alan Conway2010-03-083-7/+16
| | | | | | | | | The v2key of cluster agents was not being passed as part of a cluster update. This meant they were not being associated with the correct shadow connections on the updatee. This caused inconsistencies that shut down the new broker. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@920414 13f79535-47bb-0310-9956-ffa450edef68
* Minor cleanup: removed unused parameter of initial-status in cluster.xml.Alan Conway2010-03-052-6/+3
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@919662 13f79535-47bb-0310-9956-ffa450edef68
* Don't generate debug snapshot messages unless debug logging enabled.Alan Conway2010-03-052-9/+8
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@919523 13f79535-47bb-0310-9956-ffa450edef68
* QPID-2412: Support for EXTERNAL mechanism on client-authenticated SSL ↵Gordon Sim2010-03-055-28/+39
| | | | | | | | | | | | connections. On SSL connection where the clients certificate is authenticated (requires the --ssl-require-client-authentication option at present), the clients identity will be taken from that certificate (it will be the CN with any DCs present appended as the domain, e.g. CN=bob,DC=acme,DC=com would result in an identity of bob@acme.com). This will enable the EXTERNAL mechanism when cyrus sasl is in use. The client can still negotiate their desired mechanism. There is a new option on the ssl module (--ssl-sasl-no-dict) that allows the options on ssl connections to be restricted to those that are not vulnerable to dictionary attacks (EXTERNAL being the primary example). git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@919487 13f79535-47bb-0310-9956-ffa450edef68
* Amended cluster error message for decoding failures.Alan Conway2010-03-031-1/+1
| | | | | | | | New message says "Error decoding events, may indicate a broker version mismatch" since version mismatch is the most likely causes of a decoding error. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@918679 13f79535-47bb-0310-9956-ffa450edef68
* Fix: decoding error causes cluster broker to exit.Alan Conway2010-03-011-4/+16
| | | | | | | | | | An error decoding a connection was causing the broker::Connection object to be deleted in the connection IO thread rather than the cluster deliver thread, causing an exit with "critical Modified cluster state outside of cluster context" git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@917734 13f79535-47bb-0310-9956-ffa450edef68
* Fix cluster abort on shutdown in ClusterTimer::fire.Alan Conway2010-02-263-2/+7
| | | | | | | | The cluster destructor was not deleting the ClusterTimer set on the broker, so this timer had a dangling pointer to the cluster. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@916751 13f79535-47bb-0310-9956-ffa450edef68
* Last member of a cluster always has clean store.Alan Conway2010-02-253-3/+15
| | | | | | | | | | When a cluster is reduced to a single broker, it marks its store as clean regardless of how it is shut down. If we're down to a single member we know we want to use its store to recover as there are no others. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@916475 13f79535-47bb-0310-9956-ffa450edef68
* Removed unecessary include of Msg.h in Exception.hAndrew Stitcher2010-02-251-0/+1
| | | | git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@916453 13f79535-47bb-0310-9956-ffa450edef68
* Consistent connection names across a cluster.Alan Conway2010-02-055-31/+67
| | | | | | | | | - use the same host:port for connections and their shadows. - add shadow property to managment connection to identify shadows. - updated qpid-stat and qpid-cluster to filter on shadow property. git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@907123 13f79535-47bb-0310-9956-ffa450edef68