Apache Airflow

Origin

Airflow originated at Airbnb in 2014 and became an Apache project in 2016. It orchestrates complex Data Pipelines as code using Directed Acyclic Graphs (DAGs).

Benefits

Pipelines as Python code, versionable
Wide selection of operators
Established standard in Data Engineering

Drawbacks

Less suitable for streaming than specialized tools
Scaling requires Executor selection (Celery, Kubernetes)

Selected projects

All projects

Energy Industry

Data Foundation for Price and Scenario Management in Energy Sales

Supported the further development and operation of a central data solution for price control, planning, and scenario analyses in an energy sales environment. The solution processes extensive business and technical data flows, checks their quality, and consolidates them into reliable source data for operational and medium-term decisions. This included the modeling and automation of data pipelines, the provision of relevant KPIs, and ensuring a verifiable data foundation for reporting and management.

Learn more

Planning a project with Apache Airflow?

We bring experienced experts to your project. Talk to us.

Request a call