An exploration of the journey of a data product from prototype to complex data product operation in the holiday market.
Speaker
Andres Baravelle
Agenda
Introductions.
Webinar
Q&A
Synopsis
What started as a quick prototype for ingesting competitor holiday prices turned into a fully-fledged, multi-layered data product - streaming, batch, CI/CD-enabled, and just occasionally responsible for mini DoS.
In this talk, we'll walk through the journey of scaling a Price Intelligence system in Python and Spark, from humble beginnings in a docker container to Delta Live Tables, data contracts, automated checks, and operational chaos (and recovery).
Along the way, we'll explore what we learnt the hard way.
- Data product development
- Scaling for operation
- Automating updates
- Managing fails
About the speaker
Andres works at the intersection of analytics and data engineering, building cloud-native products for online holiday retailer On the Beach. With over 25 years’ experience spanning academia and industry, his core expertise is at the intersection of data, web and security. While in academia, he taught web technologies, data management and security, and designed and led multiple BSc and MSc programmes over the years.
His research on dark web marketplaces shone a light on illegal on-line trade in drugs, weapons and forged documents. In industry, he has led large scale projects for Fortune 500, FTSE-listed, and government organisations. He is passionate about modern analytics engineering practices and about helping teams turn raw data into actionable insights.
Our events are for adults aged 16 years and over.
BCS is a membership organisation. If you enjoy this event, please consider joining BCS. You’ll be very welcome. You’ll receive access to many exclusive career development tools, an introduction to a thriving professional community and also help us Make IT Good For Society. Join BCS today
This event is brought to you by: Data Management specialist group
