Stream processing made easy

Configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time and historical streaming data.
Powering the best data teams
gojek
midtrans
mapan
moka
zoomcar
gojek
midtrans
mapan
moka
zoomcar
gojek
midtrans
mapan
moka
zoomcar
gojek
midtrans
mapan
moka
zoomcar

Framework

Modular architecture

With Dagger, you don't need to write custom applications to process data as a stream. Instead, you can write SQL to do the processing and analysis on streaming data.

Reliable & consistent processing

Provides built-in support for fault-tolerant execution that is consistent and correct regardless of data size, cluster size, processing pattern or pipeline complexity.

Robust recovery mechanism

Checkpoints, Savepoints & State-backup ensure that even in unforeseen circumstances, clusters & jobs can be brought back within minutes.

SQL and more

Define business logic in a query & kick-start your streaming job; but it is not just that, there is support for user-defined functions & pre-defined transformations.

Scale

Dagger scales in an instant, both vertically and horizontally for high performance streaming sink and zero data drops.

Extensibility

Add your own sink or source to dagger with a clearly defined interface or choose from already provided ones.

Flexibility

Add custom business logic in form of plugins (UDFs, Transformers, Preprocessors and Post Processors) independent of the core logic.
architecture

Key Features

Built for scale

Stream processing platform for transforming, aggregating and enriching data in real-time mode with ease of operation & unbelievable reliability. Dagger can deployd in VMs or cloud-native environment to makes resource provisioning and deployment simple & straight-forward, the only limit to your data processing is your imagination.

Aggregations

Supports Tumble & Slide for time-windows. Longbow feature supports large windows upto 30-day.

SQL Support

Query writing made easy through formatting, suggestions, auto-completes and template queries.

Stream Enrichment

Enrich streamed messages from HTTP endpoints or database sources to bring offline & reference data context to real-time processing.

Observability

Always know what’s going on with your deployment with built-in monitoring of throughput, response times, errors and more.

Analytics Ecosystem

Dagger can transform, aggregate, join and enrich data in real-time for operational analytics using InfluxDB, Grafana and others.

Stream Transformations

Convert messages on the fly for a variety of use-cases such as feature engineering.

Community

Want to get involved?

Join the community on Slack and talk to maintainers to answer your questions.
Explore and contribute to Raystack data platform on Github.

© 2023 Raystack Foundation, Inc.oss