Advanced SQL

Why We’re Launching R and Python Support

sql python r image

As the complexity and volume of data grows, the data space is buzzing with change as everyone seeks to find the best new ways to make sense of their data. Conversations are dominated by talk of new methods of data storage, fancy warehouses and new kinds of tech like Spark, Athena and Spectrum for processing data.

But we don’t often talk about the languages used to analyze these massive quantities of data. As the technology continues to improve for storage and processing, data teams are working quietly behind the scenes to improve their methods of analysis. We’ve seen first-hand here at Periscope Data that our customers are getting more sophisticated as their field matures, evolving to work with bigger data sets and integrating new techniques into their workflow. This means more than just finding new ways to innovate with SQL – advanced languages like R and Python have become a more critical part of their everyday analysis.

Python has been one of the most popular languages for development, but for those in data environments the current workflows with Python are entirely disjointed. Python development environments don't typically run SQL on databases, which means you’d have to query the data elsewhere, bring that data into the environment where Python runs, then take the output and migrate it back to a place where others can consume the results.

That’s why we’re rolling out a workflow that brings together the most powerful languages needed for data analysis today, SQL, Python and R, within Periscope Data. We’re excited to give data teams the tools needed to complete far more analysis in less time, all without leaving our platform. Check out more information about what we’re announcing here.   

As always, our vision is to help make data teams into superheroes. And while our platform already enables them to build amazing visualizations and provide a single source of truth for data, there’s a whole series of superpowers that can’t be done with just SQL. Like a quarterback who stalls in the red zone, we see many analysts who bring their teams 80 yards down the field but stop just short of getting into the end zone with the perfect analysis they need.  

This functionality is all about enabling them to add that finishing touch, be it clustering, regressions or predictions, that delivers exactly what their stakeholders need to make important business decisions. These tasks are much simpler to pull off in R or Python, and come with a whole new set of customization options for visualization.

There are some who may have become accustomed to conducting R or Python analysis by connecting SQL to a Jupyter Notebook, or via an API, but we believe those solutions aren’t outcome focused. Using a notebook can make it really difficult to connect analysis back to your original data set, to keep data consistently fresh, or to get reviews from your colleagues and share results across the organization.

We know our customers are hungry for a unified data editor to do all their analysis in one place, so we’re streamlining this functionality into the same workflow. This is about bringing the power of data science to data teams – instead of forcing data teams to learn new ways to work, we’re enabling more complex analysis without changing the ways they operate.

This exciting addition to our platform will open up a whole new world of value for data teams. The combination of SQL with R and Python will be truly game-changing in the things it will allow your team to do. We can’t wait to see what you’re able to create!

Tom O’Neill is the Chief Technology Officer at Periscope Data.

Want to discuss this article? Join the Periscope Data Community!

Tom O'Neill
Periscope Data was started in Tom's apartment, where he built the first version of the product in a weekend. He leads Periscope Data's engineering efforts, and holds the coveted customer-facing bugfix record. (10 minutes!)