Outcome-Driven Data Architecture: A Spotify Listener Personality Case Study

Cloud-native systems are often discussed in terms of infrastructure and platforms, but the same principles apply just as strongly to data architecture. In this talk, I show how outcome-driven design — a core cloud-native principle— can be applied to analytics pipelines by defining the final product first and engineering backward from it.

Using a Spotify listener personality system as a case study, I demonstrate how daily listening events can be transformed into meaningful behavioral signals. The pipeline is implemented with Apache Airflow running in Docker for orchestration and Google Cloud Platform services such as BigQuery and Cloud Storage for storage and analytics. This architecture enables reproducible local development, containerized and stateless execution, and a clear separation of concerns between ingestion, orchestration, and analytics.

The project originated from a simple question — understanding my own listening behavior on Spotify — but evolved into a broader exploration of data system design. Through this case study, I highlight a common pitfall in data engineering: starting with technology choices rather than product goals. The talk makes the case for clarifying the desired outcome first and letting it drive architectural and tooling decisions, resulting in simpler, more maintainable, and more cloud-native data pipelines.