Alexis Richardson-2 wrote
Max - I have looked at your blog. Please tell us more about what you
want to achieve since then we can help you.
Allow me to give a somewhat long-winded introduction of our use of Messaging.
Maybe we are exemplary for a new kind of less enterprisy user of this class of
software. If you are bored, skip to the bottom.
I'm CIO at HUDORA, an SME being Germany's biggest supplier of Inline-Skates
and quite big in some other sportive product ranges. Our main challenge are
increasingly complex processes with our customers in the hyper competitive
European retail market (WalMart gave up in germany a year ago!). We are
determined to meet the future with custom developed, agile, highly adaptive
software. So much for the Buzzword Bingo.
In our IT landscape we are relatively legacy free (basically only the
telephone System and a big hulking AS/400 left). All new developments are done
in Python. Sometimes we write some Java if we can't resist the use of
libraries like XSLF-FO/FOP or JasperReports. We are considering Erlang to
implement a new Warehouse Managment System (WMS) since Warehouse Management has
some interesting concurrency issues. The Python programs have to run on
FreeBSD (our preferred Platform) but also on Linux and Windows to access
specialized Libraries and Interfaces like ODBC drivers etc.
We started to use messaging (without the label "messaging") and RPC to bind
systems and platforms together. In that course we have implemented UDP unicast
and multicast based solutions, used the Spread Toolkit, used PyRO RPC and a
PostgreSQL based Queue approach. By the Apache podcast I got introduced to
ApacheMQ and the world of JMS and learned that the JMS (or AMQP or whatever)
crowd solved most of the problems we were tackling with our home grown
solutions. So we stared using ActiveMQ for new projects and to replace the
home grown protocols.
We are not the financial service industry and so we have much lower
requirements for messaging :-) I estimate the highest cost for a lost message
would be 250E usually much lower. And the latency is also not that relevant: a
few seconds are fine for most of our queues. Also the volume is low: at the
moment we are well below 50.000 messages a day. Queues and topics are all we
need (for now) and we are happy the have a single message type along the lines
of BytesMessage. But we like to send big messages. So for example we would
like to use the message system to distribute scanned bitmaps for OCRing.
We build a lot of software ourselves and being a family owned business for the
last 90 years we intend to use a pice of software for some decades (at least
we try to design it that way). Therefore especially for the software where
other software builds upon we are extremely fond of the concept of reliability.
For us this does not necessarily means, 99.9999999% uptime but being "solid"
in the sense of: works deterministic and according to specification; can be
understood, fixed and tested easily. This is mainly to effectively use our
sparse coding and administration resources: if we can rely of all parts
holding up to their specification it is much more easy to integrate them and
to concentrate on the part you are developing.
It is important that "works according to specification" does not mean a huge
documentation. On the contrary - the lighter the specification, the easier it
is to implement and to test that it is implemented correctly (you probably see
my issues with AMQP there). So we generally build with a "less is more"
attitude and use agile methods.
It turned out out that our attitude towards software didn't work well with
ActiveMQ. I was not able to get my own python based client code work reliable
with ActiveMQ and it's Stomp implementation. I also was not able to get other
Python-Stomp libraries to work reliable. In fact the unit tests failed for
about 1 in 1000 messages - but not always in the same place. Maybe ActiveMQ's
Stomp implementation is broken maybe Stomp is just extremely hard to implement
right. I toyed with the idea of implementing ActiveMQ's OpenWire Protocol but
I found the documentation lacking to start an implementation from scratch -
also the protocol is more complex than necessary for our needs.
Shopping for an other message broker than ActiveMQ turns out the be a thorny
way: Most are imbedded within a bigger software stack (e.g JBoss). Seemingly
everything is meant for people with a Java background and often it is only
usable by from Java. For example see RabbitMQ which is written in Erlang but
has no Way to send and receive messages from Erlang. (Or at least had not
until the work by Ben Hood.)
Just as a side note: while we are are perfectly willing to pay for software
related services, we wary with what Stallman calls "proprietary software": we
found that systems with a "free" code base work better for us. On the other
hand proprietary software puts upon us burdens which are not in our own best
interest: E.g. windows XP "activation" or other licensing and registration
schemes.
What we are looking for
* reliable & stable broker, client libs, protocol.
* JMS Queue and Topic like behavior. In addition RPC. No transactions needed.
* Clients in Python and Java with a strong probability that in the next
24 months there will be additional clients in Ruby, C, Perl and PHP.
* Clients with a well documented, stable interface. Something along the lines
of send_bytemsg('/queue/TEST', databytes, headers),
receive_send_bytemsg('/queue/TEST'), rpc('/queue/TEST', params)
* Alternatively to already implemented clients a protocol simple enough
to enable us to implement well tested and documented clients ourself in
reasonable time.
Regards
Maximillian
Locked
