Sean Cook
Sean has chased the best tech from San Diego to NYC to SF working in GIS, data science and marketing. You can find him alternatively creating dazzling new analyses and driving content strategy at Periscope Data.

What You Need To Know About SQL's GROUP BY

May 10, 2017

A Brief Tutorial

Group by is one of the most frequently used SQL clauses. It allows you to collapse a field into its distinct values. This clause is most often used with aggregations to show one value per grouped field or combination of fields.

Consider the following table

We can use a group by and aggregates to collect multiple types of information. For example, a group by can quickly tell us the number of countries on each continent.

-- How many countries are in each continent?
select
  
continent
  , count(*)
from 
  countries
group by 
  continent

Keep in mind when using GROUP BY:

  • Group by X means put all those with the same value for X in the same row.
  • Group by X, Y put all those with the same values for both X and Y in the same row.

More Interesting Things About GROUP BY

1. Aggregations Can Be Filtered Using The HAVING Clause

You will quickly discover that the where clause cannot be used on an aggregation. For instance

select 
  continent
  , max(area)
from
  countries
where 
  max(area) >= 1e7
group by 
  1

will not work, and will throw an error. This is because the where statement is evaluated before any aggregations take place. The alternate having is placed after the group by and allows you to filter the returned data by an aggregated column.

select
 
continent
  , max(area)
from 
  countries
group by 
  1
having
  max(area) >= 1e7 -- Exponential notation can keep code clean!

Using having, you can return the aggregate filtered results!

2. You Can Often GROUP BY Column Number

In many databases you can can group by column number as well as column name. Our first query could have been written

select 
 
 continent
  , count(*)
from 
  base
group by 
  1

and returned the same results. This is called ordinal notation and its use is debated. It predates column based notation and was SQL standard until the 1980s. 

  • It is less explicit, which can reduce legibility for some users. 
  • It can be more brittle. A query select statement can have a column name changed and continue to run, producing an unexpected result.

On the other hand, it has a few benefits.

  • SQL coders tend towards a consistent pattern of selecting dimensions first, and aggregates second. This makes reading SQL more predictable.
  • It is easier to maintain on large queries. When writing long ETL statements, I have had group by statements that were many, many lines long. I found this difficult to maintain.
  • Some databases allow using an aliased column in the group by.  This allows a long case statement to be grouped without repeating the full statement in the group by clause. Using ordinal positions can be cleaner and prevent you from unintentionally grouping by an alias that matches a column name in the underlying data. For example, the following query will return the correct values:

-- How many countries use a currency called the dollar?
select
  
case when currency = 'Dollar' then currency
    else 'Other'
  
end as currency --bad alias
  
, count(*)
from
  countries
group by
  1

But this will not, and will segment by the base table's currency field while accepting the new alias column labels:

select
  
case when currency = 'Dollar' then currency 
    else 'Other' 
  end as currency --bad alias
  
, count(*)
from 
  countries
group by 
  currency

This is 'expected' behavior, but remain vigilant.

A common practice is to use ordinal positions for ad-hoc work and column names for production code. This will ensure you are being completely explicit for future users who need to change your code.

3. The Implicit GROUP BY

There is one case where you can take an aggregation without using a group by. When you are aggregating the full table there is an implied group by. This is known as the <grand total> in SQL standards documentation.

-- What is the largest and average country size in Europe?
select
  
max(area) as largest_country
  , avg(area) as avg_country_area
from 
  countries
where 
  continent = 'Europe'

4. GROUP BY Treat Null as Groupable Value, and that is Strange.

When your data set contains multiple null values, group by will treat them as a single value and aggregate for the set.

This does not conform to the standard use of null, which is never equal to anything including itself.

select null = null
-- returns null, not True

From the SQL standards guidelines in SQL:2008

Although the null value is neither equal to any other value nor not equal to any other value — it is unknown whether or not it is equal to any given value — in some contexts, multiple null values are treated together; for example, the <group by> treats all null values together.

5. MySQL Allows you to GROUP BY without Specifying all your Non-Aggregate Columns

In MySQL, unless you change some database settings, you can run queries like only a subset of the select dimensions grouped, and still get results. As an example, in MySQL this will return an answer, populating the state column with a randomly chosen value from those available.

select
  country
  , state
  , count(*)
from
  countries
group by
  country



That's all for today! Group by is a commonly used keyword, but hopefully you now have a clearer understanding of some of it's more nuanced uses.

Tags:
Haven't tried Periscope Data yet?
Start a trial and we’ll send you one of our famous coffee mugs.
Read More
Haven’t tried Periscope Data yet?