Databricks Spark Knowledge Base

The contents contained here is also published in Gitbook format.

• Best Practices
  • Avoid GroupByKey
  • Don't copy all elements of a large RDD to the driver
  • Gracefully Dealing with Bad Input Data
• General Troubleshooting
  • Job aborted due to stage failure: Task not serializable:
  • Missing Dependencies in Jar Files
  • Error running start-all.sh - Connection refused
  • Network connectivity issues between Spark components
• Performance & Optimization
  • How Many Partitions Does An RDD Have?
  • Data Locality
• Spark Streaming
  • ERROR OneForOneStrategy

This content is covered by the license specified here.