Voxxed Days Bristol 2018
on Thursday 25 October
Blanca Garcia Gil is a senior software engineer at BBC. Her current work is focused on highly scalable big data pipelines. She is very interested in automation (testing and deployment) and observability. While having spent most of her time as a Java developer she's also enjoys tinkering with Python and Bash when running data experiments.
See also https://www.bbc.co.uk
The BBC Audience Platform Data team collects, transforms and delivers billions of events each day from audience interactions with mobile apps and web sites such as BBC News, BBC Sport and iPlayer.
We will share our experience of the 7 critical steps we took to migrate from Spark clusters with Map-Reduce to a serverless architecture. We will cover the good, the bad and the ugly, including lessons learnt along the way. These steps are also applicable to new architectures.
We have been running in production for more than a year and have achieved our goal of real-time analysis at up to 25000 transactions per second. This allows us to meet the demands of even the biggest news days, as well as seeing significant savings on our cloud infrastructure.