Prior work and proposed standard

There is a console standard proposal, designed by iMatix together with JPMC and tested in production use by JPMC.

Note that the approach taken by this proposal is to define a schema-based hierarchical property system, similar to that used by JMX, Microsoft management console (MMC), etc. Rather than try to define a standard set of broker objects, we define an abstracted model where a hierarchy of objects each have properties and methods that can be inspected, modified, and executed by clients. The proposed standard implements this model using XML language messages, which are transported to and from the broker using AMQP. Clients can depend on fixed schemas, or can be fully dynamic. The Console Markup Language (CML) that this proposal defines implements both the schema, and object/property/method access.

The current schema implemented by OpenAMQ is not part of the standard proposal but serves as a useful example of known use cases, driven by the operational requirements of at least one significant application.

The primary use cases are: monitoring largest queues; monitoring blocked clients; killing blocked clients; viewing all queues for a client; viewing all clients for a shared queue; purging over-full queues; viewing and modifying the broker's configuration.

There have been some comments on other lists that this work is "not standards based". However, since the design is properly based on an XML schema, this seems an inaccurate criticism. It would be worth checking if there are existing XML schemas that solve the same problem; if so, we can obviously use these in preference to CML.

Goals

MB --->

• Decoupling of management tool implementation with AMQP implementations such that any management console implementation will interoperate with any AMQP implementation.  This is a essential reqt from the CS perspective as we've seen too much coupling in the past of management and messaging products (i.e. limitations of management tool choices based on particular messaging vendor product, unsupported management/messaging combinations requiring management vendor commitment to delivering reqd functionality, usually as a far off future date and possibly at a cost to the customer, inconsistement management feel across disparate commercial messaging solutions due to particulars of messaging product)
• Definition of a standard wire format and semantic model supporting a rich set of functions for:
  • Querying the broker (and possibly clients, either directly or via the broker) to obtain statistical information, current configuration settings, etc.
  • Execute commands against the broker (and possibly clients) to change state, shutdown, terminate client connections, etc.
  • support an event drive model for management related events (i.e. change of broker state, creation and deletion of AMQP objects, thresholds being met, etc.)
• Possibly explore integration of AMQP management with system and network management products (e.g. HP OpenView framework (i.e. as an SPI), Tivoli, CA, etc.)
• Possibly explore extension points within the management protocol such that an AMQP vendor could implement management extensions that would be queryable by a a management console.
  
  
  

Use Cases

Proposed Modes [MB]

• Request/Reply
  • Query information about the Server or an object within the Server
  • Execute a command against the Server or an object within the Server (causes some action to take place)

• Server generated Events
  • Various Events occurring within the Server or against objects within the Server would generate pre-defined Events (messages) through a pub/sub function
  • Events would be published by the Server using well-known "Topics" (routing-keys) based on pre-defined "collections" (forming a hierarchical namespace of possible Events)

• Watchlists
  • The ability to register a "watch" against a particular Server object and property
  • A watch could be expressed as a rule (or set of rules) specifying a property, a relative operator, and a value. Possibly, multiple expressions could be combined using boolean ANDing and ORing to form complex rules
  • When a watch condition is met, an Event could be published by the Server
  • While registering a watch, an associated Topic could be specified to enable one or more subscribers to receive the Event
    

Manage Server

• Shutdown
• Quiesce
• Query Server information (summary snapshot of Server) [MB]
  • Name given to Server
  • Server state (e.g. initializing, active, shutting down, quiesced)
  • Server vendor
  • Server version
  • Server start time, uptime (days/hours/seconds)
  • host machine name running Server
  • Security identifier associated with Server
  • Server listener URL(s) (e.g. port number(s))
  • Memory max available
  • Memory currently in use
  • Max connections allowed
  • Current number of connections
  • Current number of Exchanges
  • Current number of Channels
  • Current number of Queues
  • Current number of "producers" (number of clients producing msgs)
  • Current number of "consumers" (number of clients consuming msgs)
  • Current number of messages/second (both inbound and outbound)
  • Current number of bytes/second (both inbound and outbound)
  • Current number of pending messages and stated in number of bytes
  • I/O rates (both network and disk, inbound and outbound)
  • Current "Trace" and "Statistics Gathering" settings
  • Persistent store URL (e.g. file directory and file names, DBMS connection string, etc.)
  • Log file URL
  • Fault Tolerance info (i.e. active/not-active, this Server's mode, alternate Server URL, etc.)
• Events: [MB]
  • Memory usage reaches some pre-defined threshold
    

Broker Configuration

• Set max connections
• Set max memory
  

Manage Exchanges

• Display static wiring information. It should help in answering questions around;
  • (a) What messages does a client publish or subscribe ?
  • (b) Which client publishes or subscribes to a given message ?
• Add/Delete an Exchange [MB]
• Suspend an Exchange (e.g. disallow further usage, pause all activity against an Exchange) [MB]
• Display attributes associated with Exchange (e.g. Type) [MB]
• Display Exchange statistics [MB]
  • Number of msgs processed by Exchange since Server up (received, acknowledged)
  • Number of msgs received that were dropped (i.e. no bindings against the Exchange matched msg routing-key)
  • Number of clients current using the Exchange
• Enumerate all ACLs associated with the Exchange [MB]
• Events: [MB]
  • An Exchange was declared
  • An Exchange has been deleted
  • An Exchange has received a msg
  • An Exchange has matched a binding and has been forwarded a msg to a queue or queues
  • An Exchange has been bound to a queue
    

Manage Virtual Hosts [MB]

• Enumerate Virtual Hosts
• Create/delete a Virtual Host
• Suspend a Virtual Host
• Events:
  • TBD
    

Manage Queues

• Enumerate all queues [MB]
• Display queue statistics
  • Number of msgs/bytes per second, inbound and outbound
  • Pending msgs within a queue
  • Timestamp of first and last msg on a queue
  • Number of clients connected to the queue
  • Number of Exchanges bound to the queue
  • Size of queue (nbr of msgs, bytes occupied) [MB]
• Pause / Unpause all read activity on a queue
• Pause / Unpause all write activity to a queue
• Create a queue (overlap with Queue class?) [MB]
• Purge queue (all msgs; could also be selective: first msg on queue(e.g.idios to poison msg), range of msgs) [MB]
• Enumerate all consumers connected to a queue [MB]
• Enumerate all Exchanges bound to a queue [MB]
• Enumerate all security ACLs associated with queue (also a feature of Security) [MB]
  

Manage Message Queues

• View / set attributes associated with a queue
• View / change reaping parameters
• Events:
  • A qeueue has been created
  • A queue has been destroyed
  • A new msg has been enqueued
  • A msg has been dequeued
  • A msg has been acked (both producer and consumer acks)
  • A pre-defined queue threshold (by nbr of msgs, by size in bytes) has been reached
    

Manage Clients

• Allow/Disallow new client connections
  • NOTE: Requirement - admins can always log in
• Disconnect client
• Pause a client (inbound, outbound, or both) [MB]
• Disconnect all non-administrator clients
• Enumerate clients
• Display client activity
• Display host information about the client [MB]
• Display client connection type (e.g. TCP, UDP (unicast or multicast), SCTP, Infiniband(udapl,verbs)) [MB]
• Display pending messages for the client (by Channel, by Queue)
• Display time last msg was read by client from Queue
• Highlight slow consumers
  • NOTE: use Queue msg statistics (e.g. #pending) with the preceding two attributes to ascertain slow or non-consuming consumers [MB]
• Events: [MB]
  • A client has connected to the Server
  • A client has disconnected from the Server
  • A client has been disconnected from the Server due to an Exception
  • A client has been blocked from sending a msg due to the Server limited resources
  • A client has been disallowed from connecting to the Server due to max connections being reached
    

Manage Channels [MB]

• Display all channels associated with a client connection
• Display transactional quality of channel
• Events:
  • A Channel has been opened
  • A Channel has been closed
  • Transaction events ??
    

Manage Messages in Queues

• View outstanding messages (with search)
• View archived messages (with search)
• Display statistics Enqueue/Dequeue rates. Calculate estimated time to empty queue
• Events: [MB]
  • TBD
    

"Park" and "Unpark" Messages

• Take an outstanding message and move it aside, without deleting or dequeuing it.
• Resubmit "Parked" messages back onto the queue
• Change priority on a parked event.
• Resubmit to either the head or tail of the queue.
• Option to resubmit with the same message identity (default), or a new identity
• Delete events from the queue (do we really mean 'delete')

• Change Message Priority
  

AMQP Broker Security

• Add/Change/Delete a user/principal [MB]
• Add/Change/Delete a "role" (realm?) [MB]
• Add/Change/Delete an ACL (e.g. authorize a user/principal to a Queue or Exchange) [MB]
• Configure a security source (URL to local store, LDAP, identity system, entitlements system) [MB]
• Events: [MB]
  • Client connection rejected due to authentication failure
  • Client declare of an Exchange failed due to authorization failure
  • Client publication on an Exchange failed due to authorization failure against the routing-key
    • NOTE: would be relevant for JMS: JMS supports the notion in the pub/sub model of publication security against the Topic
  • Client "consume" on a Queue failed due to authorization failure
  • Client "consume" on a Queue against a particular routing-key failed due to authorization failure on the routing-key
    • NOTE: would be relevant for JMS for pub/sub security
• Unauthorized "admin" function against the Server (details TBD)
  

Tracing / Debugging [MB]

• Display current Tracing settings
• Change Tracing settings (e.g. by Server wide, by Exchange, by Queue, by client connection, etc.)
• Q: Should the protocol enable tracing to be activated within a particular client endpoint via the management function? i.e. remote tracing activation via AMQP rather than end user required activation?
• Events:
  • Tracing has been activated against an object
  • Tracing has been deactivated against an object
    

Abstraction Considerations

• Portable to all broker-side languages and platforms
• Accessible to any client-side technology
• Dynamic, so the broker schema can change with no client changes
• Interoperable, so a single management client can work with arbitrary servers

[MB: Comments]

• The native management functionality should enable the JMS model to be mappable upon it (i.e. the notion of JMS Connections, Sessions, Producers, Consumers, Destinations, etc. should map to AMQP constructs)
• Should the management protocol of AMQP specify a different protocol from the client defined one? A backdoor channel may be required in circumstances where the Server no longer can accept client connections due to resource limitations (e.g. max connections, max memory, etc.). A catch-22 may result where a Server is short on resources, but you can't get in with an admin connection to correct the situation because the admin uses the very same resources that a client connection would use.
• Should there be a basic set of management functionality with the possibility to extend the management model (extension can be queried) by specific implementations (i.e. to offer extended value-added features with a particular implementation), or would this break interoperability?

Don't read this junk yet, bashing it into shape --- JOH

• Message holding is different from parking -- holding is done by an application, parking by admins
• Message Hold Facility (push back until released)
• View Held Messages
• Release held messages

• Replay from archive based on header content, date/time,

• Message Relaying to Other Brokers

• Snoop 'n' Save Tool to analyse the life cycle of a message transported by Metro Transport between STS components. For a valid message this tool displays information about publishers, destinations, size, transport times, etc. between components.

• For un-reaped messages, this tool also provides the ability to snoop the message and to save the message for replay into a development environment