From f6c5da0288770ea204e6cddc5b9dba1e50fa696a Mon Sep 17 00:00:00 2001 From: Gordon Sim Date: Thu, 8 Jul 2010 09:31:56 +0000 Subject: Restructuring of chapter on Qpid Messaging API git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@961673 13f79535-47bb-0310-9956-ffa450edef68 --- doc/book/src/Programming-In-Apache-Qpid.xml | 1017 +++++++++++++-------------- 1 file changed, 502 insertions(+), 515 deletions(-) (limited to 'doc/book/src/Programming-In-Apache-Qpid.xml') diff --git a/doc/book/src/Programming-In-Apache-Qpid.xml b/doc/book/src/Programming-In-Apache-Qpid.xml index e504fed0d5..31b3fd3c9f 100644 --- a/doc/book/src/Programming-In-Apache-Qpid.xml +++ b/doc/book/src/Programming-In-Apache-Qpid.xml @@ -1245,7 +1245,7 @@ spout - -content "$(cat rdu.xml | sed -e 's/70/45/')" xml/weather one of: unreliable, at-least-once, at-most-once, exactly-once - + Reliability indicates the level of reliability that the sender or receiver. unreliable and at-most-once are currently @@ -1255,7 +1255,14 @@ spout - -content "$(cat rdu.xml | sed -e 's/70/45/')" xml/weather a message is not lost, but duplicates may be received. exactly-once guarantees that a message is not lost, and is delivered precisely - once. + once. Currently only unreliable + and at-least-once are supported. + If at-most-once is requested, + unreliable will be used and for durable messages on + durable queues there is the possibility that messages + will be redelivered; if exactly-once is requested, + at-most-once will be used and the application needs to + be able to deal with duplicates. @@ -1416,53 +1423,84 @@ options := map -
- Logging - - To simplify debugging, Qpid provides a logging facility - that prints out messaging events. +
+ Sender Capacity & Replay + + The send method of a sender has an optional second parameter + that controls whether the send call is synchronous or not. A + synchronous send call will block until the broker has confirmed + receipt of the message. An asynchronous send call will return + before the broker confirms receipt of the message, allowing for + example further send calls to be made without waiting for a + roundtrip to the broker for each message. This is desirable where + increased throughput is important. + + The sender maintains a list of sent messages whose receipt + has yet to be confirmed by the broker. The maximum number of such + messages that it will hold is defined by the capacity of the + sender, which can be set by the application. If an application + tries to send with a sender whose capacity is already fully used + up, the send call will block waiting for capacity regardless of + the value of the sync flag. + + The sender can be queried for the available space (i.e. the + unused capacity), and for the current count of unsettled messages + (i.e. those held in the queue pending confirmation by the + server). When the unsettled count is zero, all messages on that + sender have been successfully sent. + + If the connection fails and is transparently reconnected + (see for details on how to control + this feature), the unsettled messages for each sender over that + connection will be re-transmitted. This provides a transparent + level of reliability. This feature can be controlled through the + link's reliability as defined in the address (see + ). At present only + at-least-once guarantees are offered. +
-
- Logging in C++ - The Qpidd broker and C++ clients can both use environment - variables to enable logging. Use QPID_LOG_ENABLE to set the - level of logging you are interested in (trace, debug, info, - notice, warning, error, or critical): +
+ Receiver Capacity i.e. Prefetch - -$ export QPID_LOG_ENABLE="warning+" - + By default, a receiver requests the next message from the + server in response to each fetch call, resulting in messages being + sent to the receiver one at a time. As in the case of sending, it + is often desirable to avoid this roundtrip for each message. This + can be achieved by allowing the receiver + to prefetch messages in anticipation of + fetch calls being made. The receiver needs to be able to store + these prefetched messages, the number it can hold is controlled by + the receivers capacity. - The Qpidd broker and C++ clients use QPID_LOG_OUTPUT to - determine where logging output should be sent. This is either a - file name or the special values stderr, stdout, or syslog: +
- -export QPID_LOG_TO_FILE="/tmp/myclient.out" - +
+ Acknowledging Received Messages + + Applications that receive messages should acknowledge their + receipt by calling the session's acknowledge method. As in the + case of sending messages, acknowledged transfer of messages to + receivers provides at-least-once reliability meaning that the loss + of the connection, a client crash or even in the case of durable + messages a broker restart does not result in lost messages. Some + cases may not require this however and the reliability can be + controlled through a link property in the address options (see + ). + + The acknowledge call acknowledges all messages received on + the session (i.e. all message that have been returned from a fetch + call on a receiver created on that session). + + The acknowledge call also support an optional parameter + controlling whether the call is synchronous or not. A synchronous + acknowledge will block until the server has confirmed that it has + received the acknowledgement. In the asynchronous case, when the + call returns there is not yet any guarantee that the server has + received and processed the acknowledgement. The session may be + queried for the number of unsettled acknowledgements; when that + count is zero all acknowledgements made for received messages have + been successful. -
- -
- Logging in Python - - The Python client library supports logging using the standard Python logging module. The easiest way to do logging is to use the basicConfig(), which reports all warnings and errors: - - -from logging import basicConfig -basicConfig() - - - Qpidd also provides a convenience method that makes it easy to specify the level of logging desired. For instance, the following code enables logging at the DEBUG level: - - -from qpid.log import enable, DEBUG -enable("qpid.messaging.io", DEBUG) - - - For more information on Python logging, see http://docs.python.org/lib/node425.html. For more information on Qpid logging, use $ pydoc qpid.log. - -
@@ -1470,12 +1508,11 @@ enable("qpid.messaging.io", DEBUG) Receiving Messages from Multiple Sources A receiver can only read from one source, but many - programs need to be able to read messages from many sources, - preserving the original sequence of the messages. In the Qpid - Messaging API, a program can ask a session for the next - receiver; that is, the receiver that is responsible for - the next available message. The following example shows how this - is done in C++, Python, and .NET C#. + programs need to be able to read messages from many sources. In + the Qpid Messaging API, a program can ask a session for + the next receiver; that is, the receiver that is + responsible for the next available message. The following + example shows how this is done in C++, Python, and .NET C#. Note that to use this pattern you must enable prefetching @@ -1527,413 +1564,102 @@ session.Acknowledge();
- Request / Response - Request / Response applications use the reply-to property, - described in , to allow a server - to respond to the client that sent a message. A server sets up a - service queue, with a name known to clients. A client creates a - private queue for the server's response, creates a message for a - request, sets the request's reply-to property to the address of - the client's response queue, and sends the request to the - service queue. The server sends the response to the address - specified in the request's reply-to property. - - - Request / Response Applications in C++ - - This example shows the C++ code for a client and server - that use the request / response pattern. - - The server creates a service queue and waits for a - message to arrive. If it receives a message, it sends a - message back to the sender. + Transactions - Sometimes it is useful to be able to group messages + transfers - sent and/or received - on a session into atomic + grouping. This can be done be creating the session as + transactional. On a transactional session sent messages only + become available at the target address on commit. Likewise any + received and acknowledged messages are only discarded at their + source on commit -Message request = receiver.fetch(); -const Address& address = request.getReplyTo(); // Get "reply-to" from request ... -if (address) { - Sender sender = session.createSender(address); // ... send response to "reply-to" - Message response("pong!"); - sender.send(response); - session.acknowledge(); -} - ]]> + Note that this currently is only true for + messages received using a reliable mode + e.g. at-least-once. Messages sent by a broker to a receiver in + unreliable receiver will be discarded immediately regardless of + transctionality. - The client creates a sender for the service queue, and - also creates a response queue that is deleted when the - client closes the receiver for the response queue. In the C++ - client, if the address starts with the character - #, it is given a unique name. + . + + Transactions + C++: " << response.getContent() << std::endl; -]]> - - The client sends the string ping to - the server. The server sends the response - pong back to the same client, using the - replyTo property. - - +Connection connection(broker); +Session session = connection.createTransactionalSession(); +... +if (smellsOk()) + session.commit(); +else + session.rollback(); + ]]> -
-
- Maps in Message Content - - Many messaging applications need to exchange data across - languages and platforms, using the native datatypes of each - programming language. - - The Qpid Messaging API supports maps in message - content. + - Unlike JMS, there is not a specific message type for - map messages. +
- These maps are supported in each language using - the conventions of the language. In Java, we implement the - MapMessage interface - - Note that the Qpid JMS client supports - MapMessages whose values can be nested maps or lists. This is - not standard JMS behaviour. +
+ Connection Options - ; in Python, we - support dict and - list in message content; in C++, we - provide the Variant::Map and - Variant::List classes to represent maps - and lists. In all languages, messages are encoded using AMQP's - portable datatypes. + + Aspects of the connections behaviour can be controlled through + specifying connection options. For example, connections can be + configured to automatically reconnect if the connection to a + broker is lost. - - Because of the differences in type systems among - languages, the simplest way to provide portable messages is to - rely on maps, lists, strings, 64 bit signed integers, and - doubles for messages that need to be exchanged across languages - and platforms. - - -
- Qpid Maps in Python + + Specifying Connection Options in C++ and Python - In Python, Qpid supports the dict and list types directly in message content. The following code shows how to send these structures in a message: + In C++, these options can be set using Connection::setOption() or by passing in a set of options to the constructor. The options can be passed in as a map or in string form: - - Sending Qpid Maps in Python - +Connection connection("localhost:5672", "{reconnect: true}"); +try { + connection.open(); + !!! SNIP !!! + ]]> +or - The following table shows the datatypes that can be sent in a Python map message, - and the corresponding datatypes that will be received by clients in Java or C++. - + - - Python Datatypes in Maps - - - - Python Datatype - → C++ - → Java - - - - boolboolboolean - intint64long - longint64long - floatdoubledouble - unicodestringjava.lang.String - uuidqpid::types::Uuidjava.util.UUID - dictVariant::Mapjava.util.Map - listVariant::Listjava.util.List - - -
+ In Python, these options can be set as attributes of the connection or using named arguments in + the Connection constructor: -
+ +or + + See the reference documentation for details in each language. + -
- Qpid Maps in C++ - - - In C++, Qpid defines the the - Variant::Map and - Variant::List types, which can be - encoded into message content. The following code shows how to - send these structures in a message: - - - Sending Qpid Maps in C++ - - - - The following table shows the datatypes that can be sent - in a C++ map message, and the corresponding datatypes that - will be received by clients in Java and Python. - - - C++ Datatypes in Maps - - - - C++ Datatype - → Python - → Java - - - - boolboolboolean - uint16int | longshort - uint32int | longint - uint64int | longlong - int16int | longshort - int32int | longint - int64int | longlong - floatfloatfloat - doublefloatdouble - stringunicodejava.lang.String - qpid::types::Uuiduuidjava.util.UUID - Variant::Mapdictjava.util.Map - Variant::Listlistjava.util.List - - -
-
- - -
-
- Performance - - Clients can often be made significantly faster by using asynchronous operations, batching acknowledgements and setting the capacity of receivers to allow prefetch. The size of a sender's replay buffer can also affect performance. - - -
- Asynchronous Operations - - The send method on a sender takes an optional second - argument that controls whether the call is synchronous or - not - - In C++ the send() is asynchronous by default. In python however it is synchronous by default. - - . A synchronous send will block until the broker confirms - receipt of the message. Higher throughput is achieved if the - call returns before this point, allowing further send calls to - be made. The sender can be queried to find out how many - unsettled messages there are. Unsettled messages are those - that have been sent but not yet acknowledged by the broker. An - unsettled count of zero means that receipt of all messages - sent using that sender have been confirmed as received by the - broker. - - - The acknowledge call on a session may also be - synchronous or asynchronous. The session can be queried for - the unsettled acknowledgements also, i.e. the number of - messages received on the session that have been acknowledged - by the application but for which the acknowledgements have not - been confirmed by the broker. -
- -
- Batching Acknowledgements - - Many of the simple examples we have shown retrieve a message and immediately acknowledge it. Because each acknowledgement results in network traffic, you can imporve efficiency and increase performance by acknowledging messages in batches. For instance, an application can receive and process a number of related messages, then acknowledge the entire batch, or an application can acknowledge after a certain number of messages have been received or a certain time period has elapsed. -
- - -
- Prefetch - - By default, a receiver retrieves the next message from the server, one message at a time, in response to each fetch() call. This provides intuitive results when writing and debugging programs, but does not provide optimum performance. By allowing the client to prefetch messages in anticipation of fetch() calls, round-trips to the broker can be avoided and the throughput for a receiver can be increased. To enable prefetching, set the capacity of the receiver to the size of the desired input buffer; for many applications, a capacity of 100 performs well. - - - - - Prefetch - - C++ - -Receiver receiver = session.createReceiver(address); -receiver.setCapacity(100); -Message message = receiver.fetch(); - - - - -
- -
- Sizing the Replay Buffer - - In order to guarantee delivery, a sender automatically - keeps messages in a replay buffer until the messaging broker - acknowledges that they have been received. The replay buffer - is held in memory, and is never paged to disk. For most - applications, the default size of the replay buffer works - well. A large replay buffer requires more memory, a small - buffer can slow down the client because it can not send new - messages if the replay buffer is full, and must wait for - existing sends to be acknowledged. - - - Sizing the Replay Buffer - C++ - - -Sender sender = session.createSender(address); -sender.setCapacity(100); - - - -
- -
- -
- Connection Options - - - Aspects of the connections behaviour can be controlled through - specifying connection options. For example, connections can be - configured to automatically reconnect if the connection to a - broker is lost. - - - - Specifying Connection Options in C++ and Python - - In C++, these options can be set using Connection::setOption() or by passing in a set of options to the constructor. The options can be passed in as a map or in string form: - - - -or - - - - In Python, these options can be set as attributes of the connection or using named arguments in - the Connection constructor: - - - -or - - - - See the reference documentation for details in each language. - - - The following table lists the supported connection options. + The following table lists the supported connection options. Connection Options @@ -2061,64 +1787,317 @@ try: -
- Reliability - - When creating a sender or a receiver, you can specify a reliability option in the address string. For instance, the following specifies at-least-once as the reliability mode for a sender: - - -Sender = session.createSender("topic;{link:{reliability:at-least-once}}"); - +
+ Maps in Message Content + + Many messaging applications need to exchange data across + languages and platforms, using the native datatypes of each + programming language. - The recognised modes are unreliable, at-most-once, at-least-once, and exactly-once. These modes govern the transfer of messages. Currently only unreliable and at-least-once are supported. + The Qpid Messaging API supports maps in message + content. - If at-most-once is requested, unreliable will be used and for durable messages on durable queues there is the possibility that messages will be redelivered; if exactly-once is requested, at-most-once will be used and the application needs to be able to deal with duplicates. + Unlike JMS, there is not a specific message type for + map messages. - + These maps are supported in each language using + the conventions of the language. In Java, we implement the + MapMessage interface + + Note that the Qpid JMS client supports + MapMessages whose values can be nested maps or lists. This is + not standard JMS behaviour. - The unreliable mode means that - messages may be lost in the event of a client, connection or - broker failure. The broker will not wait for unreliable - receivers to acknowledge receipt before discarding - messages. Likewise an unreliable sender will not wait for - broker acknowledgement before considering the message - sent. - - The at-least-once mode ensures that - messages are not lost in the event of client, connection or - broker failure - To survive broker failure a message must be durably recorded. - . Duplicates of a message may delivered - however. The broker will not dequeue messages it sends to - receivers in at-least-once mode until they are - acknowledged. Messages sent by senders configure with - at-least-once mode are kept in a replay buffer until the - broker acknowledges receipt; in the event of a broker or - connection failure, messages in the replay buffer are resent - upon reconnection. - - + ; in Python, we + support dict and + list in message content; in C++, we + provide the Variant::Map and + Variant::List classes to represent maps + and lists. In all languages, messages are encoded using AMQP's + portable datatypes. + -
+ + Because of the differences in type systems among + languages, the simplest way to provide portable messages is to + rely on maps, lists, strings, 64 bit signed integers, and + doubles for messages that need to be exchanged across languages + and platforms. + -
- Cluster Failover +
+ Qpid Maps in Python - The messaging broker can be run in clustering mode, which provides high reliability through replicating state between brokers in the cluster. If one broker in a cluster fails, clients can choose another broker in the cluster and continue their work. Each broker in the cluster also advertises the addresses of all known brokers + In Python, Qpid supports the dict and list types directly in message content. The following code shows how to send these structures in a message: -This is done via the amq.failover exchange in AMQP 0-10 + + Sending Qpid Maps in Python + +content = {'Id' : 987654321, 'name' : 'Widget', 'percent' : 0.99} +content['colours'] = ['red', 'green', 'white'] +content['dimensions'] = {'length' : 10.2, 'width' : 5.1,'depth' : 2.0}; +content['parts'] = [ [1,2,5], [8,2,5] ] +content['specs'] = {'colors' : content['colours'], + 'dimensions' : content['dimensions'], + 'parts' : content['parts'] } +message = Message(content=content) +sender.send(message) + ]]> + - In C++, the FailoverUpdates class provides this functionality: - - Tracking cluster membership - - In C++: + The following table shows the datatypes that can be sent in a Python map message, + and the corresponding datatypes that will be received by clients in Java or C++. + - +
+ Python Datatypes in Maps + + + + Python Datatype + → C++ + → Java + + + + boolboolboolean + intint64long + longint64long + floatdoubledouble + unicodestringjava.lang.String + uuidqpid::types::Uuidjava.util.UUID + dictVariant::Mapjava.util.Map + listVariant::Listjava.util.List + + +
+ +
+ + + + +
+ Qpid Maps in C++ + + + In C++, Qpid defines the the + Variant::Map and + Variant::List types, which can be + encoded into message content. The following code shows how to + send these structures in a message: + + + Sending Qpid Maps in C++ + + + + The following table shows the datatypes that can be sent + in a C++ map message, and the corresponding datatypes that + will be received by clients in Java and Python. + + + C++ Datatypes in Maps + + + + C++ Datatype + → Python + → Java + + + + boolboolboolean + uint16int | longshort + uint32int | longint + uint64int | longlong + int16int | longshort + int32int | longint + int64int | longlong + floatfloatfloat + doublefloatdouble + stringunicodejava.lang.String + qpid::types::Uuiduuidjava.util.UUID + Variant::Mapdictjava.util.Map + Variant::Listlistjava.util.List + + +
+
+ + + + +
+ The Request / Response Pattern + Request / Response applications use the reply-to property, + described in , to allow a server + to respond to the client that sent a message. A server sets up a + service queue, with a name known to clients. A client creates a + private queue for the server's response, creates a message for a + request, sets the request's reply-to property to the address of + the client's response queue, and sends the request to the + service queue. The server sends the response to the address + specified in the request's reply-to property. + + + Request / Response Applications in C++ + + This example shows the C++ code for a client and server + that use the request / response pattern. + + The server creates a service queue and waits for a + message to arrive. If it receives a message, it sends a + message back to the sender. + + + + The client creates a sender for the service queue, and + also creates a response queue that is deleted when the + client closes the receiver for the response queue. In the C++ + client, if the address starts with the character + #, it is given a unique name. + + " << response.getContent() << std::endl; +]]> + + The client sends the string ping to + the server. The server sends the response + pong back to the same client, using the + replyTo property. + + + +
+ + +
+ Performance Tips + + + + Consider prefetching messages for receivers (see + ). This helps eliminate roundtrips + and increases throughput. Prefetch is disabled by default, + and enabling it is the most effective means of improving + throughput of received messages. + + + Send messages asynchronously. Again, this helps + eliminate roundtrips and increases throughput. The C++ and + .NET clients send asynchronously by default, however the + python client defaults to synchronous sends. + + + Acknowledge messages in batches (see + ). Rather than + acknowledging each message individually, consider issuing + acknowledgements after n messages and/or after a particular + duration has elapsed. + + + Tune the sender capacity (see + ). If the capacity is too low the + sender may block waiting for the broker to confirm receipt + of messages, before it can free up more capacity. + + +
+ +
+ Cluster Failover + + The messaging broker can be run in clustering mode, which provides high reliability through replicating state between brokers in the cluster. If one broker in a cluster fails, clients can choose another broker in the cluster and continue their work. Each broker in the cluster also advertises the addresses of all known brokers + +This is done via the amq.failover exchange in AMQP 0-10 + +. A client can use this information to dynamically keep the list of reconnection urls up to date. + + In C++, the FailoverUpdates class provides this functionality: + + + Tracking cluster membership + + In C++: + + ... Connection connection("localhost:5672"); connection.setOption("reconnect", true); @@ -2145,48 +2124,56 @@ try:
-
- Transactions - Sometimes it is useful to be able to group messages - transfers - sent and/or received - on a session into atomic - grouping. This can be done be creating the session as - transactional. On a transactional session sent messages only - become available at the target address on commit. Likewise any - received and acknowledged messages are only discarded at their - source on commit - Note that this currently is only true for - messages received using a reliable mode - e.g. at-least-once. Messages sent by a broker to a receiver in - unreliable receiver will be discarded immediately regardless of - transctionality. +
+ Logging - . + To simplify debugging, Qpid provides a logging facility + that prints out messaging events. - - Transactions - C++: - - - +
+ Logging in C++ + The Qpidd broker and C++ clients can both use environment + variables to enable logging. Use QPID_LOG_ENABLE to set the + level of logging you are interested in (trace, debug, info, + notice, warning, error, or critical): -
+ +$ export QPID_LOG_ENABLE="warning+" + + + The Qpidd broker and C++ clients use QPID_LOG_OUTPUT to + determine where logging output should be sent. This is either a + file name or the special values stderr, stdout, or syslog: + +export QPID_LOG_TO_FILE="/tmp/myclient.out" + +
+ +
+ Logging in Python + + The Python client library supports logging using the standard Python logging module. The easiest way to do logging is to use the basicConfig(), which reports all warnings and errors: + + +from logging import basicConfig +basicConfig() + + + Qpidd also provides a convenience method that makes it easy to specify the level of logging desired. For instance, the following code enables logging at the DEBUG level: + + +from qpid.log import enable, DEBUG +enable("qpid.messaging.io", DEBUG) + + + For more information on Python logging, see http://docs.python.org/lib/node425.html. For more information on Qpid logging, use $ pydoc qpid.log. + +
+
-- cgit v1.2.1