Free Databricks Associate Developer for Apache Spark practice questions

10 free Databricks Associate Developer for Apache Spark practice questions with the correct answer and a full explanation for each, taken from the CertStash pack of 342 questions. Work through them, then open each answer to check your reasoning.

Get all 342 questions (US$39) · Download these 10 as a PDF

Question 1

Which of the following describes the Spark driver?

  1. The Spark driver is responsible for performing all execution in all execution modes, it is the entire Spark application.
  2. The Spare driver is fault tolerant, if it fails, it will recover the entire Spark application.
  3. The Spark driver is the coarsest level of the Spark execution hierarchy, it is synonymous with the Spark application.
  4. The Spark driver is the program space in which the Spark application’s main method runs coordinating the Spark entire application.
  5. The Spark driver is horizontally scaled to increase overall processing throughput of a Spark application.
Show answer and explanation

Correct answer: D. The Spark driver is the program space in which the Spark application’s main method runs coordinating the Spark entire application.

Spark application’s main method runs coordinating the Spark entire application. The Spark driver is the program space where the Spark application's main method runs, and it coordinates the entire Spark application. It acts as the central coordinator that manages the overall execution flow, schedules tasks, and maintains the state of the application. While the driver is crucial to Spark applications, it is not itself fault-tolerant (eliminating B), does not perform all execution (A is incorrect), and cannot be horizontally scaled (E is wrong). Option C is somewhat accurate but less precise than D, which captures the driver's essential role in coordinating execution through the main method.

Why the other options are wrong

  • A. The driver coordinates execution but does not perform all execution itself, executors on nodes handle the actual task execution.
  • B. The Spark driver is not fault-tolerant; if the driver fails, the entire Spark application typically fails.
  • C. While the driver is the coarsest level, option D is more complete by explaining its actual function in the application.
  • E. The Spark driver cannot be horizontally scaled; it runs as a single process that coordinates the application.

Question 2

Which of the following describes the relationship between nodes and executors?

  1. Executors and nodes are not related.
  2. Anode is a processing engine running on an executor.
  3. An executor is a processing engine running on a node.
  4. There are always the same number of executors and nodes.
  5. There are always more nodes than executors.
Show answer and explanation

Correct answer: C. An executor is a processing engine running on a node.

An executor is a processing engine that runs on a node. Executors are JVM processes launched on worker nodes that perform the actual computation and data processing tasks. Each node can host multiple executors, and multiple nodes collectively form the Spark cluster. This relationship is fundamental to Spark's distributed architecture.

Why the other options are wrong

  • A. Nodes and executors are directly related within Spark's execution model.
  • B. A node is hardware; an executor is a process running on that hardware, not the reverse.
  • D. There is no requirement for equal numbers of executors and nodes; one node can run multiple executors.
  • E. There is no requirement that nodes outnumber executors; the relationship is flexible based on configuration.

Question 3

Which of the following will occur if there are more slots than there are tasks?

  1. The Spark job will likely not run as efficiently as possible.
  2. The Spark application will fail, there must be at least as many tasks as there are slots.
  3. Some executors will shut down and allocate all slots on larger executors first.
  4. More tasks will be automatically generated to ensure all slots are being used.
  5. The Spark job will use just one single slot to perform all tasks.
Show answer and explanation

Correct answer: A. The Spark job will likely not run as efficiently as possible.

When there are more slots available than tasks to execute, some slots will remain idle and unused. This means the available computing resources are not fully utilized, resulting in suboptimal job efficiency. The job will still run successfully, but it will not achieve the best possible performance because parallel processing capacity is wasted. Spark does not fail the application, shut down executors, or automatically generate extra tasks to fill slots.

Why the other options are wrong

  • B. Spark jobs run successfully even with more slots than tasks; there is no requirement for equal numbers.
  • C. Spark does not automatically shut down executors when slots exceed tasks.
  • D. Spark does not automatically generate additional tasks to fill available slots.
  • E. Spark will use multiple slots if they are available; it does not force execution to a single slot.

Question 4

Which of the following is the most granular level of the Spark execution hierarchy?

  1. Task
  2. Executor
  3. Node
  4. Job
  5. Slot
Show answer and explanation

Correct answer: A. Task

The Spark execution hierarchy from coarsest to most granular is: Application > Job > Stage > Task. A task is the smallest unit of work that is executed by a single executor on a single partition. It represents the finest level of granularity in the execution model. Slots are abstractions representing execution capacity, and executors and nodes are containers that host tasks and slots.

Why the other options are wrong

  • B. An executor is a container that runs multiple tasks; it is coarser than a task.
  • C. A node is hardware infrastructure; it is coarser than a task.
  • D. A job is a collection of tasks triggered by an action; it is coarser than a task.
  • E. A slot is a unit of execution capacity; it is coarser than a task in terms of the execution hierarchy.

Question 5

Which of the following statements about Spark jobs is incorrect?

  1. Jobs are broken down into stages.
  2. There are multiple tasks within a single job when a DataFrame has more than one partition.
  3. Jobs are collections of tasks that are divided up based on when an action is called.
  4. There is no way to monitor the progress of a job.
  5. Jobs are collections of tasks that are divided based on when language variables are defined.
Show answer and explanation

Correct answer: D. There is no way to monitor the progress of a job.

The statement that 'there is no way to monitor the progress of a job' is incorrect. Spark provides multiple ways to monitor job progress, including the Spark UI web interface, logging output, and programmatic APIs. Jobs are correctly described as being broken down into stages (A), containing multiple tasks when a DataFrame has multiple partitions (B), and being triggered by actions (C). Option E is also somewhat poorly worded but refers to the fact that language variables do not define job boundaries, actions do. Option D is definitively false.

Why the other options are wrong

  • A. Jobs are indeed broken down into stages, which is a correct statement.
  • B. Multiple tasks within a job for multi-partition DataFrames is correct.
  • C. Jobs are correctly defined as collections of tasks triggered by action calls.
  • E. While poorly worded, this attempts to distinguish jobs from variable definitions, which is a defensible distinction.

Question 6

Which of the following operations is most likely to result in a shuffle?

  1. DataFrame.join()
  2. DataFrame.filter()
  3. DataFrame.union()
  4. DataFrame.where()
  5. DataFrame.drop()
Show answer and explanation

Correct answer: A. DataFrame.join()

DataFrame.join() is most likely to result in a shuffle operation. Joins typically require data from different partitions to be co-located, which necessitates shuffling data across the network to group related rows together. Filter, where, union, and drop are narrow transformations that operate on data within existing partitions without requiring data movement across partitions.

Why the other options are wrong

  • B. Filter is a narrow transformation that does not require a shuffle.
  • C. Union combines DataFrames without requiring rows to be redistributed across partitions.
  • D. Where is functionally equivalent to filter and is a narrow transformation without shuffle.
  • E. Drop (removing columns) is a narrow transformation that does not require a shuffle.

Question 7

The default value of spark.sql.shuffle.partitions is 200.

Which of the following describes what that means?

  1. By default, all DataFrames in Spark will be spit to perfectly fill the memory of 200 executors.
  2. By default, new DataFrames created by Spark will be split to perfectly fill the memory of 200 executors.
  3. By default, Spark will only read the first 200 partitions of DataFrames to improve speed.
  4. By default, all DataFrames in Spark, including existing DataFrames, will be split into 200 unique segments for parallelization.
  5. By default, DataFrames will be split into 200 unique partitions when data is being shuffled.
Show answer and explanation

Correct answer: E. By default, DataFrames will be split into 200 unique partitions when data is being shuffled.

The spark.sql.shuffle.partitions parameter specifies the default number of partitions to use when data is shuffled. When a shuffle operation occurs (such as during a join or aggregation), Spark will redistribute the data into 200 partitions by default. This applies specifically to shuffle operations, not to all DataFrame operations or all existing DataFrames. It is a configuration for controlling parallelism during expensive shuffle operations.

Why the other options are wrong

  • A. This configuration does not relate to filling executor memory or affecting all DataFrames indiscriminately.
  • B. This setting applies only during shuffle operations, not to all new DataFrames created.
  • C. This is not about reading only the first 200 partitions; it is about repartitioning during shuffle.
  • D. This setting applies only during shuffle operations, not to all DataFrames including existing ones.

Question 8

Which of the following is the most complete description of lazy evaluation?

  1. None of these options describe lazy evaluation
  2. A process is lazily evaluated if its execution does not start until it is put into action by some type of trigger
  3. A process is lazily evaluated if its execution does not start until it is forced to display a result to the user
  4. A process is lazily evaluated if its execution does not start until it reaches a specified date and time
  5. A process is lazily evaluated if its execution does not start until it is finished compiling
Show answer and explanation

Correct answer: B. A process is lazily evaluated if its execution does not start until it is put into action by some type of trigger

Lazy evaluation is a computational strategy where execution does not begin until it is triggered by an action. In Spark, transformations on DataFrames are lazily evaluated, they are not executed when defined but only when an action (such as collect(), show(), or write()) forces the computation to actually run. This allows Spark to optimize the entire computation graph before executing it. The trigger is the action, which is a more general and accurate description than limiting it only to displaying results to users.

Why the other options are wrong

  • A. Option B does describe lazy evaluation.
  • C. While displaying results is one trigger, lazy evaluation is triggered by any action, not just user display.
  • D. Lazy evaluation has nothing to do with date and time scheduling.
  • E. Lazy evaluation is not related to compilation; Spark uses lazy evaluation at runtime.

Question 9

Which of the following DataFrame operations is classified as an action?

  1. DataFrame.drop()
  2. DataFrame.coalesce()
  3. DataFrame.take()
  4. DataFrame.join()
  5. DataFrame.filter()
Show answer and explanation

Correct answer: C. DataFrame.take()

DataFrame.take() is an action because it returns data to the driver and triggers execution of the computation graph. Actions are operations that return results to the user or write data to storage, forcing Spark to evaluate all preceding transformations. The other options are transformations: drop(), coalesce(), join(), and filter() all return new DataFrames without triggering execution.

Why the other options are wrong

  • A. DataFrame.drop() is a transformation that returns a new DataFrame without executing the plan.
  • B. DataFrame.coalesce() is a transformation that reduces partition count without executing the plan.
  • D. DataFrame.join() is a transformation that returns a new DataFrame without executing the plan.
  • E. DataFrame.filter() is a transformation that returns a filtered DataFrame without executing the plan.

Question 10

Which of the following DataFrame operations is classified as a wide transformation?

  1. DataFrame.filter()
  2. DataFrame.join()
  3. DataFrame.select()
  4. DataFrame.drop()
  5. DataFrame.union()
Show answer and explanation

Correct answer: B. DataFrame.join()

DataFrame.join() is a wide transformation because it requires data from multiple partitions to be shuffled across the network. Wide transformations cause a shuffle operation where data is redistributed across partitions based on join keys, requiring network I/O and repartitioning. The other options are narrow transformations: filter(), select(), and drop() operate within partitions independently, and union() combines DataFrames without shuffling.

Why the other options are wrong

  • A. DataFrame.filter() is a narrow transformation operating independently on each partition.
  • C. DataFrame.select() is a narrow transformation operating independently on each partition.
  • D. DataFrame.drop() is a narrow transformation operating independently on each partition.
  • E. DataFrame.union() is a narrow transformation that combines partitions without shuffling data.

That was 10 of 342.

The full Databricks Associate Developer for Apache Spark pack has all 342 questions, each with the answer, the explanation and why the other options are wrong, plus a questions-only copy for timed runs. US$39, paid once, with free monthly updates and a pass-or-your-money-back guarantee.

Get the full pack