As I wrote in pretty much all my articles about this tool, Spark is super easy to use, as much as SQL. But it doesn’t matter how many hours I spend in writing code, I am just not able to permanently store Spark APIs in my brain (someone would say that my memory is like RAM, small and volatile (: ).

Whether you want a quick-start introduction to Spark SQL and you are impatient of writing your first query, or you are just like me and you need a cheat-sheet, I believe that you’ll find this article useful.

The idea of this post is to cover all the major functions/features of Spark SQL, and in the snippets you’ll always have the original SQL query and its translation in PySpark.

#spark #python #data-science #towards-data-science #programming