Callysto—Stream Processing in Rust

Callysto is a Rust framework for stream processing,
which benefits from Rust's concurrency model and
optimized performance.

High Performance

2 Million messages processed concurrently in ~3 seconds. Get more work done in less time.*

durability

Even your application crashes, you will have all your messages in your persistent store of choice. Can recover from messages instantly from storage.

distributed

You can have message collections shared between multiple Callysto instances using distributed WAL or distributed object-storage system.

GitHub

Source Code

Crates.IO

Crate Page

Docs.rs

Documentation

Discord

Community Server

Shared State / Memory

Share state between concurrent processor instances. Callysto's lock-free nature allows high-frequency updates on the shared memory your processor can use. Callysto designed towards having services as first-class pattern. Everything in Callysto is a service or a task. For more information about the distinction between services and tasks follow the link below.

learn more

Fast IO system and Web interface

Callysto uses io_uring by default on Linux. Io_uring is new fast proactive IO system for Linux. Callysto works with thread per core model on top of Nuclei runtime. It also includes epoll interface in linux for backward compatibility. Latencies are tested on real production deployment environment.

* tested on Intel Xeon W-2295 with K8S service deployment with multi-node Kafka cluster.

learn more

Distributed Tables, Sinks and more..

Different sinks to write your data back to various storage systems. In addition to that distributed tables for using aggregated data across instances for computation purposes. Cron Jobs, single use tasks and many more features that allows using writing distributed services easily.

learn more