Apache Kafka

Between each major operation data is buffered.

Alpaca uses apache-kafka as a proven/reliable/scalable solution.

It can be downloaded here: https://kafka.apache.org/