Combine 2 data frames with different columns in spark

Combine 2 data frames with different columns in spark

I have 2 dataframes:

I have 2 dataframes:

df1 :

Id    purchase_count   purchase_sim
12       100               1500
13       1020              1300
14       1010              1100
20       1090              1400
21       1300              1600

df2:

Id     click_count      click_sim
12       1030              2500
13       1020              1300
24       1010              1100
30       1090              1400
31       1300              1600

I need to get the combined data frame with results as :

Id     click_count      click_sim     purchase_count   purchase_sim
12       1030              2500            100               1500
13       1020              1300            1020              1300
14       null              null            1010              1100
24       1010              1100            null              null
30       1090              1400            null              null
31       1300              1600            null              null
20       null              null            1090              1400
21       null              null            1300              1600                                     

I can't use union because of different column names. Can some one suggest me a better way to do this ?

Angular 9 Tutorial: Learn to Build a CRUD Angular App Quickly

What's new in Bootstrap 5 and when Bootstrap 5 release date?

Brave, Chrome, Firefox, Opera or Edge: Which is Better and Faster?

How to Build Progressive Web Apps (PWA) using Angular 9

What is new features in Javascript ES2020 ECMAScript 2020

Apache Spark Tutorial For Beginners - Apache Spark Full Course

This video on Apache Spark Tutorial For Beginners - Apache Spark Full Course will help you learn the basics of Big Data, what Apache Spark is, and the architecture of Apache Spark. Yyou will understand how to install Apache Spark on Windows and Ubuntu. You will look at the important components of Spark, such as Spark Streaming, Spark MLlib, and Spark SQL. You will get an idea about implement Spark with Python in PySpark tutorial and look at some of the important Apache Spark interview questions

Why learn Apache Spark in 2020?

This video on "Apache Spark in 2020" will provide you with the detailed and comprehensive knowledge about the current IT Job trends based on Apache Spark and why learn Apache Spark in 2020? What is new in Apache Spark? What is Apache Spark? Top 5 Reasons to learn Spark. Salary trends of Spark Developer. Components of Spark. Skills required by Spark Developer. Companies using Apache Spark

PySpark Tutorial For Beginners | Apache Spark With Python Tutorial

PySpark Tutorial For Beginners | Apache Spark With Python Tutorial will help you understand what PySpark is, the different features of PySpark, and the comparison of Spark with Python and Scala. Learn the various PySpark contents - SparkConf, SparkContext, SparkFiles, RDD, StorageLevel, DataFrames, Broadcast and Accumulator. You will get an idea about the various Subpackages in PySpark. You will look at a demo using PySpark SQL to analyze Walmart Stocks data