Periscope Data
The world’s fastest visualization platform for data analysts.

The High-Performance SQL Blog

Generate Series in Redshift and MySQL

May 20, 2014

‍A lot of the charts and tables made in Periscope are time series, and the queries behind them are often easier when you can join and aggregate against a list of dates. Not having a complete list of dates causes gaps in the results, changing them in a misleading way:

Postgres has a great function for generating a list of dates (see Use generate_series to get continuous results), and making a list of the last 60 days with generate_series is easy:

select now()::date - generate_series(0, 59)

Accomplishing the same thing in Redshift and MySQL requires a little more work.

Date Series from a Numbers Table

The simplest alternative to generate_series is to create a table containing a continuous list of numbers, starting at 0, and select from that table. (If you have a table with a sequential id column and never delete rows from it, you can just select the id column from that table instead of creating a new numbers table).

select n from numbers;

Returns this list of rows: 0, 1, 2, 3…

Now that you have a numbers table, convert each number into a date:

Redshift:

select (getdate()::date - n)::date from numbers

MySQL:

select date_sub(date(now()), interval n day) from numbers

A numbers table is more convenient than a dates table since it never needs to be refreshed with new dates.

Redshift: Date Series using Window Functions

If you don’t have the option to create a numbers table, you can build one on the fly using a window function. All you need is a table that has at least as many rows as the number of dates desired. Using a window function, number the rows in any table to get a list of numbers, and then convert that to a list of dates:

select row_number() over (order by true) as n
from users limit 60

And now creating the list of dates directly:

select (
    getdate()::date - row_number() over (order by true)
  )::date as n
from users limit 60

MySQL: Date Series using Variables

With variables in MySQL, we can generate a numbers table by treating a select statement as a for loop:

set @n:=-1;
select (select @n:= @n+1) n
from users limit 60

And now creating the list of dates directly:

set @n:=date(now() + interval 1 day);
select (select @n:= @n - interval 1 day) n
from users limit 60

Now that we’ve made a list of dates, aggregating and joining data from other tables for time series charts is a breeze!


Want to discuss this article? Join the Periscope Data Community!
Haven't tried Periscope Data yet?
Start a trial and we’ll send you one of our famous coffee mugs.
Read More
Haven’t tried Periscope Data yet?

Subscribe to our Newsletter