Recent Posts

Understand the Spark Deployment Modes

5 minute read

Spark deployment modes Besides running Spark application in local mode (used only for testing), spark applications can run in different cluster managers: ...

Top reasons why you should shift to spark

less than 1 minute read

Fast, in-memory (100x faster) or disk (2-10x faster). See Daytona GraySort contest and Official Result. Usability: rich APIs (Scala, Java, Python), conc...

Introduction to Spark SQL

3 minute read

With Spark and RDD core API, we can do almost everything with datasets. Developers define the steps of how to retrieve the data by applying functional transf...

How can two applications share RDDs

1 minute read

Problem The application isolation in current Spark’s architecture results in the impossibility of sharing data (mostly RDDs) between different applications w...