ETL is a simple process until it isn’t. When you’re dealing with billions of rows, how can you choose the synchronization strategy that best fits your needs?
In this talk, we dive down the rabbit hole of incremental time-series ETL. We consider the trade-offs with each strategy and how to choose the right one for you. Are your priorities perfect accuracy? Bandwidth? Run-time? Or something in between?
To put these strategies to the test, we will build several time-series ETL streams using the open source framework Meerschaum. We’ll dive into a new Meerschaum Compose project and write several plugins to ingest outside data sources. Lastly, we’ll wrap things up with a secure Docker image for easy deployment that Cloud Ops will love.
From beginners to experts, there’s something here for everyone!
Target Audience: Intermediate
Presenter: Bennett Meares
The Carolina Code Conference is a polyglot conference that takes place every August in Greenville, SC. Bennett made some heads spin showing our audience how time-series data really works!
This was the only TimescaleDB related talk from this year’s event but all talks have been shared on YouTube for those interested.