Common Statistical Operations
Medians in SQL
Finding the min, max, and avg price of an order in your purchases table is easy. But what about the median price? Medians are much more difficult for the database to compute, so there usually isn’t a built-in function like the following:
-- not going to work!
select median(price) from purchases
Why is Median Hard?
For the standard aggregate functions (min, max, count, etc.) it’s possible to get the result in a single pass over the data, storing the current min/max/count along the way. If the column is indexed, the minimum or maximum value can be found without looking at any rows!
Median cannot work like that because it’s algorithmically much more complicated. The median is the middle value. The data needs to be sorted so that the middle record can be found.
Finding the Median with Different Databases
Since median is harder than min/max/count and not a standard aggregate function, we’ll need to do a little more work to calculate it. Here’s how on several different databases:
Median on Redshift
The Redshift team recently released a median window function, making it one of the easiest syntaxes to find the median with:
select median(price) over () as median
Note the limit 1: Since median is a window function and not an aggregate function, it’ll return one value for each row in the table.
Median on Postgres
If you like defining your own functions in Postgres, the Postgres Wiki has a definition for median. We’ll do it in SQL and get Postgres to help us find the middle value by numbering all the rows with the row_number() window function.
First, a CTE to sort and number all of the rows, with a count that’ll help later on:
with ordered_purchases as (
row_number() over (order by price) as row_id,
(select count(1) from purchases) as ct
Then we find the middle one or two rows and average their values:
select avg(price) as median
where row_id between ct/2.0 and ct/2.0 + 1
The where clause ensures that we’ll get the two middle values if there is an even number of values, and the single middle number if there is an odd number of values because between is inclusive of its bounds.
Median on MySQL
MySQL might not have window functions, but it does have variables, and we’ll use them to achieve the same result.
First we’ll set two variables, one for the row count and one to act as the row_id from before:
set @ct := (select count(1) from purchases);
set @row_id := 0;
And just like before, we average the middle one or two values:
select avg(price) as median
from (select * from purchases order by price)
where (select @row_id := @row_id + 1)
between @ct/2.0 and @ct/2.0 + 1
The @row_id := @row_id + 1 syntax simply increments the @row_id counter for each row. Unlike Postgres we don’t need to build up a temporary result set of rows with row_id because variables let us compute the row_id on the fly.
Bonus Round: Mode
No post with avg and median is complete without mode! The mode (defined as the most frequent value) of a column can be found with a simple group and count:
select price as mode
group by 1
order by count(1) desc
This query gives us the price with the greatest count() in the table, also known as the mode.
And now you know how to compute median in SQL for Redshift, Postgres and MySQL databases!