Apache Spark

Shuffling in Apache Spark

Senthil Nayagan
Senthil Nayagan           
  -  0 Mins Read
Shuffling in Apache Spark

Writing in progress: If you have any suggestions for improving the content or notice any inaccuracies, please email me at [email protected]. Thanks!

What is shuffling in Spark?

Comments

comments powered by Disqus

Explore more like this

apache-spark shuffling
Partitions and Bucketing in Spark

Partitions and Bucketing in Spark

Partitioning and bucketing are used to improve the reading of data by reducing the cost of shuffles, the need for serialization, and the amount of network traffic.
Senthil Nayagan
Senthil Nayagan
Jul 25, 2022 - 13 Mins Read
Need for Caching in Apache Spark

Need for Caching in Apache Spark

Caching is one of Spark's optimization strategies for reusing computations. It stores interim and partial results so they'll be utilised in subsequent computation stages.
Senthil Nayagan
Senthil Nayagan
Jul 24, 2022 - 2 Mins Read