Building distributed systems is hard work. Consensus algorithms like [Paxos](http://en.wikipedia.org/wiki/Paxos_(computer_science) or the new hotness raft can be tricky to understand and even trickier to implement. Distributed locking mechanisms can be shoehorned into Redis (see the “Locking” section in the SETNX command docs) or memcached, but these solutions are hacks and not fault tolerant, making them less than ideal.
But there is a prebaked solution: Apache Zookeeper. If you’ve never directly used Zookeeper, there’s still a good chance that you have used it indirectly. It is integrated with systems like Hadoop, Akka, Kafka and many others.
Zookeeper lets you store data in what it calls znodes. Znodes are named with filesystem-like paths, for
/path/to/node. Znodes are set up in a tree structure. So
/foo is the parent of
Unlike a typical filesystem, you can store data in both
/foo/bar. Zookeeper is, in many
ways, just a fault tolerant key value store.
However, zookeeper has a few things that make it unique.
- Zookeeper lets you set up ephemeral znodes, which are destroyed when the client that created them disconnects. This can be used for things like keeping a list of living servers.
- Zookeeper lets you watch certain nodes and get updates when their value changes or when new children are added to them. This can be used for keeping track of living servers and discovering new servers as they become live.
- Zookeeper lets you create nodes with a sequential suffix. This allows you to create queues in zookeeper or set up a chain of automatic failover.
If you’re looking for a few examples of what you can do by leveraging these features, the official zookeeper docs have some good examples.