Time series

Ingest and query time series data with Redis

Discord Github

The Redis time series data type lets you store real-valued data points along with the time they were collected. You can combine the values from a selection of time series and query them by time or value range. You can also compute aggregate functions of the data over periods of time and create new time series from the results. When you create a time series, you can specify a maximum retention period for the data, relative to the last reported timestamp, to prevent the time series from growing indefinitely.

Time series support very fast reads and writes, making them ideal for applications such as:

  • Instrument data logging
  • System performance metrics
  • Financial market data
  • Internet of Things (IoT) sensor data
  • Smart metering
  • Quality of service (QoS) monitoring

Redis time series are available in Redis Open Source, Redis Software, and Redis Cloud. See Install Redis Open Source or Install Redis Enterprise for full installation instructions.

Create a time series

You can create a new empty time series with the TS.CREATE command, specifying a key name. Alternatively, if you use TS.ADD to add data to a time series key that does not exist, it is automatically created (see Adding data points below for more information about TS.ADD).

The timestamp for each data point is a 64-bit integer value. The value represents a Unix timestamp, measured in milliseconds since the Unix epoch. When you create a time series, you can specify a maximum retention period for the data, relative to the last reported timestamp. A retention period of zero means the data does not expire.

You can also add one or more labels to a time series when you create it. Labels are name-value pairs where both the name and value are strings. You can use the names and values to select subsets of all the available time series for queries and aggregations.

Add data points

You can add individual data points with TS.ADD, but you can also use TS.MADD to add multiple data points to one or more time series in a single command. (Note that unlike TS.ADD, TS.MADD doesn't create any new time series if you specify keys that don't exist.) The return value is an array containing the number of samples in each time series after the operation. If you use the * character as the timestamp, Redis will record the current Unix time, as reported by the server's clock.

Query data points

Use TS.GET to retrieve the data point with the highest timestamp in a time series. This returns both the timestamp and the value.

Use TS.RANGE to retrieve data points from a time series that fall within a given timestamp range. The range is inclusive, meaning that samples whose timestamp equals the start or end of the range are included. You can use - and + as the start and end of the range, respectively, to indicate the minimum and maximum timestamps in the series. The response is an array of timestamp-value pairs returned in ascending order by timestamp. If you want the results in descending order, use TS.REVRANGE with the same parameters.

Both TS.RANGE and TS.REVRANGE also let you filter results. Specify a list of timestamps to include only samples with those exact timestamps in the results (you must still specify timestamp range parameters if you use this option). Specify a minimum and maximum value to include only samples within that range. The value range is inclusive and you can use the same value for the minimum and maximum to filter for a single value.

Query multiple time series

The TS.GET, TS.RANGE, and TS.REVRANGE commands also have corresponding TS.MGET, TS.MRANGE, and TS.MREVRANGE versions that operate on multiple time series. TS.MGET returns the data point with the highest timestamp from each time series, while TS.MRANGE and TS.MREVRANGE return data points from a range of timestamps in each time series.

The parameters are mostly the same except that the multiple time series commands don't take a key name as the first parameter. Instead, you specify a filter expression to include only time series with specific labels. (See Creating a time series above to learn how to add labels to a time series.) The filter expressions use a simple syntax that lets you include or exclude time series based on the presence or value of a label. See the description in the TS.MGET command reference for details of the filter syntax. You can also request that data points be returned with all their labels or with a selected subset of them.

Aggregation

A time series can become large if samples are added very frequently. Instead of dealing with individual samples, it is sometimes useful to split the full time range of the series into equal-sized "buckets" and represent each bucket by an aggregate value, such as the average or maximum value.

For example, if you expect to collect more than one billion data points in a day, you could aggregate the data using buckets of one minute. Since each bucket is represented by a single value, this reduces the dataset size to 1,440 data points (24 hours x 60 minutes = 1,440 minutes).

The range query commands let you specify an aggregation function and bucket size. The available aggregation functions are:

  • avg: Arithmetic mean of all values
  • sum: Sum of all values
  • min: Minimum value
  • max: Maximum value
  • range: Difference between the highest and the lowest value
  • count: Number of values
  • first: Value with lowest timestamp in the bucket
  • last: Value with highest timestamp in the bucket
  • std.p: Population standard deviation of the values
  • std.s: Sample standard deviation of the values
  • var.p: Population variance of the values
  • var.s: Sample variance of the values
  • twa: Time-weighted average over the bucket's timeframe (since RedisTimeSeries v1.8)

For example, the example below shows an aggregation with the avg function over all five data points in the rg:2 time series. The bucket size is 2ms, so there are three aggregated values with only one value used to calculate the average for the last bucket.

Bucket alignment

The sequence of buckets has a reference timestamp, which is the timestamp where the first bucket in the sequence starts. By default, the reference timestamp is zero. For example, the following commands create a time series and apply a min aggregation with a bucket size of 25 milliseconds at the default zero alignment.

The diagram below shows the aggregation buckets and their alignment to the reference timestamp at time zero.

Value:        |      (1000)     (2000)     (3000)     (4000)     (5000)     (6000)     (7000)
Timestamp:    |-------|10|-------|20|-------|30|-------|40|-------|50|-------|60|-------|70|--->

Bucket(25ms): |_________________________||_________________________||___________________________|
                           V                          V                           V
                  min(1000, 2000)=1000      min(3000, 4000)=3000     min(5000, 6000, 7000)=5000

You can also align the buckets to the start or end of the query range. For example, the following command aligns the buckets to the start of the query range at time 10.

The diagram below shows this arrangement of buckets.

Value:        |      (1000)     (2000)     (3000)     (4000)     (5000)     (6000)     (7000)
Timestamp:    |-------|10|-------|20|-------|30|-------|40|-------|50|-------|60|-------|70|--->

Bucket(25ms):          |__________________________||_________________________||___________________________|
                                    V                          V                           V
                        min(1000, 2000, 3000)=1000      min(4000, 5000)=4000     min(6000, 7000)=6000

Aggregation across timeseries

By default, the results from TS.MRANGE and TS.MREVRANGE are grouped by time series. However, you can use the GROUPBY and REDUCE options to group them by label and apply an aggregation over elements that have the same timestamp and the same label value (this feature is available from RedisTimeSeries v1.6 onwards).

For example, the following commands create four time series, two for the UK and two for the US, and add some data points. The first TS.MRANGE command groups the results by country and applies a max aggregation to find the maximum sample value in each country at each timestamp. The second TS.MRANGE command uses the same grouping, but applies an avg aggregation.

Compaction

Aggregation queries let you extract the important information from a large data set into a smaller, more manageable set. If you are continually adding new data to a time series as it is generated, you may need to run the same aggregation regularly on the latest data. Instead of running the query manually each time, you can add a compaction rule to a time series to compute an aggregation incrementally on data as it arrives. The values from the aggregation buckets are stored in a separate time series, leaving the original series unchanged.

Use TS.CREATERULE to create a compaction rule, specifying the source and destination time series keys, the aggregation function, and the bucket duration. Note that the destination time series must already exist when you create the rule and also that the compaction will only process data that is added to the source series after you create the rule.

For example, you could use the commands below to create a time series along with a compaction rule to find the minimum reading in each period of 3ms.

Adding data points within the first 3ms (the first bucket) doesn't produce any data in the compacted series. However, when you add data for time 4 (in the second bucket), the compaction rule computes the minimum value for the first bucket and adds it to the compacted series.

The general strategy is that the rule does not add data to the compaction for the latest bucket in the source series, but will add and update the compacted data for any previous buckets. This reflects the typical usage pattern of adding data samples sequentially in real time (an aggregate value typically isn't correct until its bucket period is over). But note that earlier buckets are not "closed" when you add data to a later bucket. If you add or delete data in a bucket before the latest one, the compaction rule will still update the compacted data for that bucket.

Delete data points

Use TS.DEL to delete data points that fall within a given timestamp range. The range is inclusive, meaning that samples whose timestamp equals the start or end of the range are deleted. If you want to delete a single timestamp, use it as both the start and end of the range.

Use time series with other metrics tools

In the RedisTimeSeries GitHub organization, you can find projects that help you integrate RedisTimeSeries with other tools, including:

  1. Prometheus, a read/write adapter to use RedisTimeSeries as the backend database.
  2. Grafana 7.1+, using the Redis Data Source.
  3. Telegraf. Download the plugin from InfluxData.
  4. StatsD, Graphite exports using graphite protocol.

More information

The other pages in this section describe RedisTimeSeries concepts in more detail. See also the time series command reference.

RATE THIS PAGE
Back to top ↑