What we build

Challenge

Many all-in-one BI solutions promise quick results. In practice, development costs run high, the solution hits limits as the company grows, or use cases can't be modelled cleanly – and in the end, gaps are bridged with unversioned scripts.

Solution

Instead of buying a BI platform, we rely on proven cloud services. AWS Step Functions orchestrate the load processes – with no ongoing maintenance costs. Snowflake is the de facto standard for cloud data warehousing and extremely cheap at low usage: one or two refreshes of the data model per day are often enough, and reports then run from the QuickSight or Power BI cache – the warehouse then costs around €100 a month. Unlike a classic database, it scales up seamlessly as the company grows. Amazon QuickSight fits in seamlessly and is priced low. With AI support, new data sources are connected quickly today.

Challenge

A central data warehouse holds a lot of sensitive data in one place. With SaaS solutions, an external operator comes on top, who has to be secured contractually and technically.

Solution

All components run in the company's AWS account. Credentials are stored in AWS Secrets Manager and rotated automatically. Roles and permissions in Snowflake are defined as code and can be checked against the actual state at any time. Every load run is logged.

Challenge

Many industry solutions offer no usable interface. But the data sits in a database – such as Oracle or Microsoft SQL Server – that can be reached on the company network.

Solution

A small Python application runs where the database can be reached. It exports the data with the database's command-line tools and uploads it to Amazon S3, from where it flows straight into the data warehouse. The exporter is controlled through tasks stored in S3 – it checks every minute whether there is something to do. We know the typical pitfalls from several systems, so new variants are built quickly.

Challenge

A new data warehouse is only accepted if its figures match the familiar reports. But how a backend system's standard reports calculate is often undocumented.

Solution

We load the original – a standard report, an Excel export or a PDF invoice – into the data warehouse and adjust the data model until the figures match. The reconciliation is built permanently into the data model, and deviations appear in a dashboard. Once one to two years of history show only minor deviations throughout, the old report can be set aside or the legacy system retired.

Challenge

Over time, data sources, models and dashboards grow. Without monitoring, errors go unnoticed; without documentation, soon nobody knows where a figure comes from or what a dashboard is for.

Solution

Every load run is monitored, and errors trigger an alert immediately. A web app lets the team start processes themselves. AI documents the data sources and the data model and describes every dashboard. This documentation is also what lets AI assistants find their way around the system.