A success story:LAIKA

LAIKA is an award-winning stop-motion animation studio specializing in feature films, commercial content, music videos, and short films. Their database infrastructure team recognized the need for a performant, reliable time-series database that also worked with their current infrastructure centered around PostgreSQL. TimescaleDB allows the team at LAIKA to consolidate a variety of metrics from different sources on all their host machines into one central, performant, reliable time-series database.

UserMahlon Smith
TitleSenior Technologist
CompanyLAIKA
IndustryEntertainment
Use casesIT & DevOps Monitoring

“TimescaleDB makes it significantly easier to correlate data and visualize various environments. We were doing all these things before, but TimescaleDB makes it faster and better.”

- Mahlon Smith, Senior Technologist, LAIKA

Why LAIKA needs TimescaleDB

Founded in 2005, the U.S.-based studio is best known for its critically acclaimed feature films, including Paranorman, The Boxtrolls, the Academy Award-nominated Coraline, and, most recently, Golden Globe-winning Missing Link. With their large quantities of render farms, workstations, and virtual machines, LAIKA’s devices produce enormous amounts of IT metrics and time-series data.

LAIKA requires large amounts of data processing and storage while creating and editing their animated content. LAIKA’s relatively small infrastructure team decided that their best option for data management was an in-house system that would not require much manual servicing. However, managing a sophisticated database system can be quite difficult if you do not choose the right technologies for your use case.

Migrating to a more flexible architecture for time-series data

The team at LAIKA reported that they “missed the boat on Timescale initially,” and instead evaluated OpenTSDB and InfluxDB. Ultimately, they ended up selecting InfluxDB, but said they “regretted it shortly thereafter” since InfluxDB did not prove to be scalable nor resource-efficient for their workload. The infrastructure team felt that with a different system, they could handle a much bigger workload than InfluxDB managed at the time without having to change their existing hardware.

After about 8 months of using InfluxDB, they discovered TimescaleDB and started aggressively migrating their data. The team reported a notable performance boost when it came to performing complex queries after the transition.

Since the team at LAIKA had years and years of data stored in PostgreSQL, they were excited to add TimescaleDB which allowed them to finally perform cross-join queries to answer complex questions of their data. TimescaleDB allowed the team to consolidate metrics from all of LAIKA’s host machines into one central, high-performance, reliable time-series database.

“We could foresee that our internal database, InfluxDB, was not scalable nor resource-efficient for our workload.

Since we are huge fans of PostgreSQL, TimescaleDB was a natural solution. The transition allowed us to retire complex internal applications and immediately allowed us to correlate licensing data with farm usage. We are very pleased with TimescaleDB's ability to serve as a sophisticated database system that does not require much manual servicing.”

- Mahlon Smith, Senior Technologist, LAIKA

Upgrading existing infrastructure based on PostgreSQL

Prior to the addition of TimescaleDB, all LAIKA’s technologies were rolled up in-house and most evolved around PostgreSQL. With PostgreSQL, the team found themselves with a fairly complex system of trigger-based partitioning for syslog and other similar data.

After moving to TimescaleDB, the team is able to leverage automatic partitioning, which partitions your chunks automatically for you behind a hypertable. A hypertable looks and feels just like a PostgreSQL table but allows for more efficient storage. Since the team already knew how to manage PostgreSQL, getting started with TimescaleDB was a seamless experience.

When it comes to constraint exclusion, TimescaleDB’s optimizations reduce the number of chunks access to reduce compute cycles/disc access and ultimately speeds up queries.

Since time-series data has unique properties, it requires specific features to manage and special storage requirements. Fortunately, the team at LAIKA realized this early on and implemented TimescaleDB as a solution which seamlessly integrated with their existing infrastructure.

Overview of how LAIKA deploys TimescaleDB

At a high level, this is how their setup works: they have a traditional 2-machine primary-replica setup, writes all hit the primary, reads are balanced via a pooler.
The replica is a WAL replayer and a replicant streamer. They deploy by updating the replica’s libraries, then updating the primary, and executing the “ALTER” command then cascades to the replica.

Leveraging time-series specific features for increased developer productivity

TimescaleDB has many built-in features for time-series analytics, and the team at LAIKA takes full advantage of these features to enhance the performance of their database. For example, LAIKA makes heavy use of

time_bucket()
and TimescaleDB’s internalization of function results (
now()
), etc. They approximate
time_bucket
with some fairly complicated
generate_series()
self joins but report that
time_bucket
makes the whole process easier.

Because TimescaleDB is built on a PostgreSQL foundation, the team at LAIKA is able to expand on its capabilities using tools from the rich, proven PostgreSQL ecosystem. In conjunction with TimescaleDB, the team is using Grafana, OpenLDAP, Ruby, RabbitMQ, Syslog-NG, Asterisk, Netdata, and a few other technologies that are confidential. (To learn more about the integration between Netdata and TimescaleDB, visit this blog post.)

“What are your goals? What problems are you trying to solve? Do you need relational queries on your data? Are you storing data hinged on time? Number one, you should be using PostgreSQL. After that, TimescaleDB is a natural fit.”

- Mahlon Smith, Senior Technologist, LAIKA

Identify with LAIKA's use case?
Contact our team to learn more

I'd like to know more