A success story:LAIKA
“TimescaleDB makes it significantly easier to correlate data and visualize various environments. We were doing all these things before, but TimescaleDB makes it faster and better.”
- Mahlon Smith, Senior Technologist, LAIKA
Why LAIKA needs TimescaleDB
Founded in 2005, the U.S.-based studio is best known for its critically acclaimed feature films, including Paranorman, The Boxtrolls, the Academy Award-nominated Coraline, and, most recently, Golden Globe-winning Missing Link. With their large quantities of render farms, workstations, and virtual machines, LAIKA’s devices produce enormous amounts of IT metrics and time-series data.
LAIKA requires large amounts of data processing and storage while creating and editing their animated content. LAIKA’s relatively small infrastructure team decided that their best option for data management was an in-house system that would not require much manual servicing. However, managing a sophisticated database system can be quite difficult if you do not choose the right technologies for your use case.
Migrating to a more flexible architecture for time-series data
The team at LAIKA reported that they “missed the boat on Timescale initially,” and instead evaluated OpenTSDB and InfluxDB. Ultimately, they ended up selecting InfluxDB, but said they “regretted it shortly thereafter” since InfluxDB did not prove to be scalable nor resource-efficient for their workload. The infrastructure team felt that with a different system, they could handle a much bigger workload than InfluxDB managed at the time without having to change their existing hardware.
After about 8 months of using InfluxDB, they discovered TimescaleDB and started aggressively migrating their data. The team reported a notable performance boost when it came to performing complex queries after the transition.
Since the team at LAIKA had years and years of data stored in PostgreSQL, they were excited to add TimescaleDB which allowed them to finally perform cross-join queries to answer complex questions of their data. TimescaleDB allowed the team to consolidate metrics from all of LAIKA’s host machines into one central, high-performance, reliable time-series database.
“We could foresee that our internal database, InfluxDB, was not scalable nor resource-efficient for our workload.
Since we are huge fans of PostgreSQL, TimescaleDB was a natural solution. The transition allowed us to retire complex internal applications and immediately allowed us to correlate licensing data with farm usage. We are very pleased with TimescaleDB's ability to serve as a sophisticated database system that does not require much manual servicing.”
- Mahlon Smith, Senior Technologist, LAIKA
Upgrading existing infrastructure based on PostgreSQL
Prior to the addition of TimescaleDB, all LAIKA’s technologies were rolled up in-house and most evolved around PostgreSQL. With PostgreSQL, the team found themselves with a fairly complex system of trigger-based partitioning for syslog and other similar data.
After moving to TimescaleDB, the team is able to leverage automatic partitioning, which partitions your chunks automatically for you behind a hypertable. A hypertable looks and feels just like a PostgreSQL table but allows for more efficient storage. Since the team already knew how to manage PostgreSQL, getting started with TimescaleDB was a seamless experience.
When it comes to constraint exclusion, TimescaleDB’s optimizations reduce the number of chunks access to reduce compute cycles/disc access and ultimately speeds up queries.
Since time-series data has unique properties, it requires specific features to manage and special storage requirements. Fortunately, the team at LAIKA realized this early on and implemented TimescaleDB as a solution which seamlessly integrated with their existing infrastructure.
Overview of how LAIKA deploys TimescaleDB
Leveraging time-series specific features for increased developer productivity
TimescaleDB has many built-in features for time-series analytics, and the team at LAIKA takes full advantage of these features to enhance the performance of their database. For example, LAIKA makes heavy use of
time_bucket()and TimescaleDB’s internalization of function results (
now()), etc. They approximate
time_bucketwith some fairly complicated
generate_series()self joins but report that
time_bucketmakes the whole process easier.
Because TimescaleDB is built on a PostgreSQL foundation, the team at LAIKA is able to expand on its capabilities using tools from the rich, proven PostgreSQL ecosystem. In conjunction with TimescaleDB, the team is using Grafana, OpenLDAP, Ruby, RabbitMQ, Syslog-NG, Asterisk, Netdata, and a few other technologies that are confidential. (To learn more about the integration between Netdata and TimescaleDB, visit this blog post.)
“What are your goals? What problems are you trying to solve? Do you need relational queries on your data? Are you storing data hinged on time? Number one, you should be using PostgreSQL. After that, TimescaleDB is a natural fit.”
- Mahlon Smith, Senior Technologist, LAIKA
TimescaleDB was LAIKA’s Missing Link
With the addition of TimescaleDB, LAIKA was able to retire complex internal systems and utilize a new, flexible infrastructure. Additionally, the team can now concentrate on new tasks instead of worrying about the stability and scalability of their database.
As LAIKA continues to create award-winning films, their team can confidently build and maintain the infrastructure they need, with the help of TimescaleDB.