frank's homepage

Here's a lesson I learned recently: Don't use JMS over HTTP if you want to have anything close to high-throughput, at least not if you are using JBossMQ. This may also be the case for other providers, depending on their HTTP client implementation.

And here's why: When a JMS over HTTP client is subscribed to a JMS destination it is actually polling the server. It will connect, receive a message, close the connection, reconnect, get the next message, etc. That's because HTTP doesn't have persistent connections or server-client callbacks the way a binary protocol might have. The client needs to reconnect to the server with a new HTTP request every time it wants to check for messages.

If you are receiving a lot of messages this will result in very frequent HTTP requests [1]. This causes memory and threading problems on the server as it spins up new threads to handle the requests.

In my case this is made worse by the fact that the client will very frequently close an existing JMS consumer and create a new consumer with a different JMS selector. What happens in this case is that the new consumer will use a new outgoing port for its HTTP connection [2]. As the client rapidly creates new consumers and makes connections it will use up more and more ports. Windows is slow in cleaning up relinquished ports, so under heavy load when receiving a lot of messages and creating new consumers the client will eventually fail to connect when all ports are used up [3]. Making so many frequent HTTP connections on the client also causes memory and threading issues.

Luckily this issue is easily addressed by switching to a different JMS protocol. By using the JBossMQ UIL2 protocol only a single port and persistent connection is used for JMS. This allows the client to rapidly receive messages and create/close consumers without problems.

I thought this was an interesting problem since initially the implications of using JMS over HTTP weren't clear to me. The original idea of going over HTTP was to avoid opening an additional port on the server.

Notes:

[1] It is possible to set a property that will cause the client to wait before reconnecting to get the next message. However, this is not desirable if you want the client to process messages as quickly as possible.

[2] Using a new outgoing port may happen even if you're receiving messages using the same consumer, without closing/creating new consumers. I didn't test that case.

[3] On Windows the port limit can be increased to work around this part of the problem: KB196271