Combining SQL + Python for an Improved Data Workflow
Data teams have evolved a lot in the last five years. With radically cheaper storage and an exponential increase in data available for analysis, modern data teams are finding answers to questions that were recently considered impossible to even approach.
As companies mature in their usage of data, they’re finding that a lot of traditional tools and process using aren’t adequate anymore. Advanced data questions require advanced data models and languages to transform and visualize information in a way that uncovers new insights.
Five years ago, SQL was enough to handle all of a company’s data requests, but today’s playing field requires the use of languages like R and Python to perform deeper analyses.
Mature data teams get the most out of a dataset with a team full of generalists who have skills with multiple languages. Using a powerful data platform, they can transform data in new ways and perform a new range of advanced analysis. Using SQL for the heavy data lifting and Python/R for the polishing, the data workflow is evolving to give more mature data companies a serious competitive advantage.
To show how SQL and Python work together to answer complex data questions, I did a live coding demo at Big Data Ignite a couple of weeks ago. In only 15 minutes, I used Periscope Data to turn a SQL table into five unique new visualizations, illustrating progressively complex insights.
See it all (the live coding in SQL and Python starts at 20:00) in the video below: