Tracking commercial aircraft in near real-time using a Raspberry Pi, Kafka and Vertica (followed by AGM)

When: 12th Oct 2017, 18:00 - 12th Oct 2017, 21:00
Where: Amba Hotel Charing Cross, Thames Room, The Strand, London, WC2N 5HX
Town/City: London
Organiser: BCS Advanced Programming Specialist Group
Price: Free
Further Information: Further Information

Please note: This is a different venue to our usual.
Please allow additional time to travel to the venue and locate the room.

This presentation discusses a project that has been developing over a number of months in conjunction with the London Big Data and Machine Learning Meetup.

The primary goal of this project was to develop a simple, end-to-end example of how streaming data can be sourced, decoded and transformed prior to being loaded into a Vertica database. From here, the project looks at ways in which the data can be used to demonstrate many of the in-built features of Vertica, including integration to the Apache Kafka messaging bus, supporting third-party visualisation tools and measuring and preparing data in readiness for undertaking advanced analytics and machine learning.

The secondary goal was to make this project accessible to anyone who wanted to follow this for themselves. This would be achieved by identifying hardware and software that are both easy to source and affordable!

There are many components to this project including, using Automatic Dependent Surveillance Broadcast (ADS-B) to track the position, speed and other metrics of commercial aircraft, Raspberry Pi computers to perform data capture and decoding, a custom-built Extract Transform and Load (ETL) function, Apache Kafka, integration between Kafka and Vertica, simple data visualisation using geospatial coordinates and a Google Maps API, time series gap filling and interpolation, outlier detection and sessionization to name but a few!

This project is only part of the way through its journey, with participants encouraged to come up with ideas on its direction as it continues to evolve.

The meeting will be followed by the APSG AGM and a buffet with wine.

Speaker biography:
From the early 1980s, Mark Whalley BSc (Hons) worked with Michael Stonebraker's Ingres RDBMS and then a number of column-store big data analytic technologies.

In 2016, he joined HPE Big Data Platform as a Systems Engineer specialising in Vertica and Vertica SQL in Hadoop, and from September 2017 followed Vertica as it transitioned to Micro Focus.

In December 2016 Mark founded the Big Data and Machine Learning (BDML) Meetup group in London. This group has grown to 1,500 Members in its first year and tries to meet once per month. Mark has since established a Cambridge BDML Meetup group, and is co-founder of a group in Munich.