Can someone tell me whether TimescaleDB is a good fit for my use case?

1

Hey all,

Can someone tell me whether TimescaleDB is a good fit for my use case?

I have a table called “Trades”. It contains two columns. (1) trade_time and (2) price.

I need to check whether future price will go up or down from the current row. For example, the current row is 1-Jan-2022. And the price is $100.

1% is $1. If the price hits $101 first, then the price is up. So the result is 1.

If the price hits $99 first, then the price is down. So the result is 0.

I have the Trades data in a 20 GB csv file. It contains 650 Million rows.

This is a one-time task. So I’m not looking for persistence. I just want the task to be finished quickly.

Thanks.

2

Hi John,

(answered on Slack as well, but replying here for visibility)

Considering that this is a one-time task, you might want to use Pandas vectorized operations (looping over rows one by one will be slow) or Dask to process your dataset.

If you do go down the database route, then you could use the LAG() or LEAD() window functions within PostgreSQL (and TimescaleDB) to compare the rows. But using a database would make more sense if you want to store this data long-term, aggregate, run queries etc…