From the course: Complete Guide to Databricks for Data Engineering
Join today to access over 24,700 courses taught by industry experts.
Understand job clusters - Databricks Tutorial
From the course: Complete Guide to Databricks for Data Engineering
Understand job clusters
- [Instructor] Job cluster is one of the cluster that works on its own. You must be surprised how, let's just see and understand what is job cluster and how exactly it works. Job cluster is a type of a cluster that is very dedicated work for a specific job and a notebook. The job cluster automatically starts to execute that notebook and once that notebook or the job get finished, it'll turn off or terminate the cluster automatically. The job cluster is in general, is temporary, lightweight and optimized for a specific task execution. The job cluster's important feature is it is dedicated for the jobs. So let's say if you have a 10 job, you can think of creating a 10 different job cluster. And these are a very lightweight job cluster works for every job. They get started to execute the job. Once the job get finished, they automatically get terminated. It's a temporary, unlike to your interactive cluster, which is always there, job cluster is temporary. Once you start and stop it, you…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.