In my previous post, I provided a quick overview of three basic concepts of RabbitMQ: exchange, routing keys and queues.
In this post, I will explain how to wire it all together.
What are the different types of exchanges?
There are four types: fanout, direct, topic and headers. They allow for increasing levels of filtering.
An exchange will dispatch its messages to queues that have the correct bindings defined.
Wait, what is a binding?
A binding is the connection between an exchange and a queue. A binding defines when a message that has been posted to an exchange will be put on a certain queue.
A binding consists of of three things:
- the exchange
- the routing key
- the queue
The important thing here is the combination of the exchange and the routing key. These will decide whether or not the message is put on the queue.
Fanout
In the case of a fanout exchange, the routing key is ignored. All messages put on the exchange that is mentioned in the binding will end up in the queue.
Direct
If the binding mentions a direct exchange, the routing key of the message must match the routing key of the binding exactly. Only then will the message be put on the queue.
Topic
If the binding mentions a topic exchange, it gets a little more complex, but also more powerful. It allows you to filter out certain messages more granularly.
A binding to a direct exchange only allows messages on the queue if their routing key is exactly the same as the routing key of the binding. But in case of a topic exchange you can use wildcards.
This requires you to send messages to the exchange with a routing key that consists of multiple words, separated by dots, for example customer.purchase
or premiumcustomer.purchase
.
You could then use this to route all purchases to a queue, by defining a binding with a routing key of *.purchase
.
If a certain queue only needs the purchases of premium customers, you could use the routing key premiumcustomer.purchase
.
You could definitely achieve the same with direct exchanges and multiple bindings, but you would need a binding for every routing key, making management more and more complex when more routing keys come into existence.
For example, say you have these routing keys:
customer.purchase
premiumcustomer.purchase
customer.registration
customer.feedback
customer.upgrade
If you wanted a queue to receive all customer
messages, you would need 4 bindings on a direct exchange. With a topic exchange, you only need one binding, with customer.*
as a routing key.
One final remark. There are two wildcards: *
and #
. The *
will match exactly one word, whereas the #
will match zero or more words.
So premiumcustomer.#
will match premiumcustomer
, premiumcustomer.purchase
, premiumcustomer.purchase.payment
, etc.
Headers
That last type of exchange is the headers exchange. When you send out a message, you can include headers with the message. These are just key-value pairs, much like HTTP headers.
If you bind a queue to a headers exchange you can provide arguments to the binding. These arguments are also key-value pairs. RabbitMQ will then match those arguments to the headers. If they match, the message will go to the queue. If not, the queue doesn’t receive the message.
There is one extra argument that you need to add to the binding: the special “x-match” argument. If you set the value of this argument to “all”, then all the headers should match with the key-value pairs of the binding. If you set it to “any”, then one is enough.
As an example, picture a message with headers type: report
and format: pdf
. Here’s a table of queues, the binding arguments and whether or not the queue will receive the message:
Binding Arguments | Receives Message? |
type: report format: pdf x-match: all | Yes |
type: report format: docx x-match: all | No |
type: report format: pdf x-match: any | Yes |
This type of exchange allows for some creative filtering. The strength lies in the fact that you can add all kinds of headers to your messages, e.g. user id’s, regions, etc. The order in which you put them isn’t important. In a topic exchange, report.pdf
isn’t the same as pdf.report
.
This is a simple example, but if you have more than 10 headers for example, this could prove to become complicated with a topic exchange.
On the other hand, you can only match “all” or “any” of the headers. You couldn’t implement a topic exchange binding like user.*.registration.*.failed
in a headers exchange.
Conclusion
RabbitMQ configuration isn’t that hard, but it allows a lot of flexibility. The key is in the bindings and how it is defined by the routing key and the type of exchange. My suggestion would be that simple projects can and should start with direct exchanges and only move to topic exchanges once there is a need for it (YAGNI, remember?).