StreamsHub

There is a rich ecosystem of open source projects providing event-driven and real time data infrastructure. The Apache Foundation alone has a wide selection of projects in this space including; Kafka, Flink, Spark, Pulsar, Beam, Paimon, Pinot to name a few. The Linux and Cloud Native Computing Foundations also have their own open source offerings. Navigating this ecosystem is often difficult and knowing how to deploy, run and combine these projects can be daunting.

StreamsHub aims to provide a curated set of open source projects, templates and tools to help infrastructure engineers create services that support event-driven architectures (EDA), running on Kubernetes, and for data-scientists and analysts to create data-pipelines and query real-time data. Where there is a gap in the current open source offering, that would benefit from being filled, StreamsHub will aim to host a project to fill it (for example the Flink SQL runner allowing Standalone SQL query deployments), with the aim of pushing the solution up to the main projects.