Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Practice Test Questions and Answers (2026) | Cert Empire Practice Questions

Correct code block:

transactionsDf.filter(transactionsDf.storeId.isNotNull()).groupBy("storeId").pivot("productId",

[2,

3]).mean("predError").orderBy("storeId")

Output of correct code block:

+-------+----+----+

|storeId| 2| 3|

+-------+----+----+

|

2| 6.0|null|

|

3|null|null|

|

25| 3.0| 3.0|

+-------+----+----+

This Question: is quite

convoluted and requires you to think hard about the correct order of operations. The pivot method

also

makes

an

appearance

-

a

method

that

you

may

not

know

all

that much

about (yet).

At the first position in all answers is code block 4, so the Question: is essentially just about the

ordering of the remaining 4 code blocks.

The Question: states that the returned DataFrame should be sorted by column storeId. So, it should

make sense to have code block 3 which includes the orderBy operator at the very end of

the code

block. This leaves you with only two answer options.

Now, it is useful to know more about the context of pivot in PySpark. A common pattern is groupBy,

pivot, and then another aggregating function, like mean. In the documentation linked below you

can see that pivot is a method of pyspark.sql.GroupedData - meaning that before pivoting, you have

to use groupBy. The only answer option matching this requirement is the one in which code

block 2 (which includes groupBy) is stated before code block 5 (which includes pivot).

More info: pyspark.sql.GroupedData.pivot — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 3,

If an action including an accumulator fails during execution and Spark manages to restart the action

and complete it successfully, only the successful attempt will be counted in the accumulator.

Correct, when Spark tries to rerun a failed action that includes an accumulator, it will only update the

accumulator if the action succeeded.

Accumulators are immutable.

No. Although accumulators behave like write-only variables towards the executors and can only be

read by the driver, they are not immutable.

All accumulators used in a Spark application are listed in the Spark UI.

Incorrect. For scala, only named, but not unnamed, accumulators are listed in the Spark UI. For

pySpark, no accumulators are listed in the Spark UI – this feature is not yet implemented.

Accumulators are used to pass around lookup tables across the cluster.

Wrong – this is what broadcast variables do.

Accumulators can be instantiated directly via the accumulator(n) method of the pyspark.RDD

module.

Wrong, accumulators are instantiated via the accumulator(n) method of the sparkContext, for

example: counter = spark.sparkContext.accumulator(0).

More info: python - In Spark, RDDs are immutable, then how Accumulators are implemented? - Stack

Overflow, apache spark - When are accumulators truly reliable? - Stack Overflow, Spark – The

Definitive Guide, Chapter 14

transactionsDf.dropna(thresh=4)

Correct. Note that by only working with the thresh keyword argument, the first how keyword

argument is ignored. Also, figuring out which value to set for thresh can be difficult, especially when

under pressure in the exam. Here, I recommend you use the notes to create a "simulation" of what

different values for thresh would do to a DataFrame. Here is an explanatory image why thresh=4 is

the correct answer to the question:

transactionsDf.dropna(thresh=2)

Almost right. See the comment about thresh for the correct answer above.

transactionsDf.dropna("any")

No, this would remove all rows that have at least one missing value.

transactionsDf.drop.na("",2)

No, drop.na is not a proper DataFrame method.

transactionsDf.dropna("",4)

No, this does not work and will throw an error in Spark because Spark cannot understand the first

argument.

More info: pyspark.sql.DataFrame.dropna — PySpark 3.1.1 documentation (https://bit.ly/2QZpiCp)

Static notebook | Dynamic notebook: See test 1,

Correct code block: transactionsDf.join(itemsDf, itemsDf.itemId == transactionsDf.productId, "outer") Static notebook | Dynamic notebook: See test 1, ( Databricks import instructions) (<a href="https://flrs.github.io/spark_practice_tests_code/#1/33.html" rel="noopener noreferrer" target="_blank" style="color: rgb(29, 78, 216);">https://flrs.github.io/spark_practice_tests_code/#1/33.html , <a href="https://bit.ly/sparkpracticeexams_import_instructions)" rel="noopener noreferrer" target="_blank" style="color: rgb(29, 78, 216);">https://bit.ly/sparkpracticeexams_import_instructions)

Correct code block:

transactionsDf.repartition(14, "storeId", "transactionDate").count()

Since we do not know how many partitions DataFrame transactionsDf has, we cannot safely use

coalesce, since it would not make any change if the current number of partitions is smaller than 14.

So, we need to use repartition.

In

the

Spark

documentation,

the

call

structure

for

repartition

is

shown

like

this:

DataFrame.repartition(numPartitions, *cols). The * operator means that any argument after

numPartitions

will

be

interpreted as column. Therefore, the brackets need to be removed.

Finally, the Question: specifies that

after the execution the DataFrame should be divided. So, indirectly this Question: is asking us to

append

an

action

to

the

code

block.

Since

.select()

is a transformation. the only possible choice here is .count().

More info: pyspark.sql.DataFrame.repartition — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1,

Although the Question: suggests using

a method called unique() here, that method does not actually exist in PySpark. In PySpark, it is called

distinct().

But

then,

this

method

is

not

the

right

one to use here, since with distinct() we could filter out unique values in a specific column.

However, we want to return the entire rows here. So the trick is to use dropDuplicates with the

subset keyword parameter. In the documentation for dropDuplicates, the examples show that subset

should be used with a list. And this is exactly the key to solving this question: The productId column

needs to be fed into the subset argument in a list, even though it is just a single column.

More info: pyspark.sql.DataFrame.dropDuplicates — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1,

Answering this Question: correctly depends

on whether you understand the arguments to the DataFrame.sample() method (link to the

documentation

below).

The

arguments

are

as

follows:

DataFrame.sample(withReplacement=None, fraction=None, seed=None).

The first argument withReplacement specified whether a row can be drawn from the DataFrame

multiple times. By default, this option is disabled in Spark. But we have to enable it here, since the

question asks for a row being able to appear more than once. So, we need to pass True for this

argument.

About replacement: "Replacement" is easiest explained with the example of removing random items

from a box. When you remove those "with replacement" it means that after you have taken an

item out of the box, you put it back inside. So, essentially, if you would randomly take 10 items out of

a box with 100 items, there is a chance you take the same item twice or more times. "Without

replacement" means that you would not put the item back into the box after removing it. So, every

time you remove an item from the box, there is one less item in the box and you can never take the

same item twice.

The second argument to the withReplacement method is fraction. This referes to the fraction of

items that should be returned. In the Question: we are asked for 150 out of 1000 items – a

fraction of 0.15.

The last argument is a random seed. A random seed makes a randomized processed repeatable. This

means that if you would re-run the same sample() operation with the same random seed, you

would get the same rows returned from the sample() command. There is no behavior around the

random seed specified in the question. The varying random seeds are only there to confuse you!

More info: pyspark.sql.DataFrame.sample — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1,

Caching means storing a copy of a partition on an executor, so it can be accessed quicker by

subsequent operations, instead of having to be recalculated. cache() is a lazily-evaluated method of

the

DataFrame. Since count() is an action (while filter() is not), it triggers the caching process.

More info: pyspark.sql.DataFrame.cache — PySpark 3.1.2 documentation, Learning Spark, 2nd

Edition, Chapter 7

Static notebook | Dynamic notebook: See test 2,

Correct code block:

transactionsDf.printSchema()

This is more of a knowledge Question: that you

should just memorize or look up in the provided documentation during the exam. You can get more

info

about

DataFrame.printSchema()

in

the

documentation (link below). However - it is a plain simple method without any arguments.

One

answer

points

to

an

alternative

of

printing

the

schema:

You

could

also

use

print(transactionsDf.schema). This will give you readable, but not nicely formatted, description of the

schema.

More info: pyspark.sql.DataFrame.printSchema — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1,

Correct code block:

transactionsDf.select("storeId", "predError").collect()

collect() is a method of the DataFrame object.

More info: pyspark.sql.DataFrame.collect — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 2,

Correct code block:

find_most_freq_letter_udf = udf(find_most_frequent_letter)

itemsDf.withColumn("most_frequent_letter", find_most_freq_letter_udf("itemName"))

Spark should use the previously registered find_most_freq_letter_udf method here – but it is not

doing that in the original codeblock. There, it just uses the non-UDF version of the Python method.

Note that typically, we would have to specify a return type for udf(). Except in this case, since the

default return type for udf() is a string which is what we are expecting here. If we wanted to return

an integer variable instead, we would have to register the Python function as UDF using

find_most_freq_letter_udf = udf(find_most_freq_letter, IntegerType()).

More info: pyspark.sql.functions.udf — PySpark 3.1.1 documentation

Output of correct code block:

+-------------------+

|

supplier|

+-------------------+

|Sports Company Inc.|

+-------------------+

Key to managing this Question: is understand

which operator to use to do the opposite of an operation – the ~ (not) operator. In addition, you

should know that there is no unique() method.

Static notebook | Dynamic notebook: See test 1,

More

info:

Running

Spark:

an

overview

of

Spark’s

runtime

architecture

-

Manning

(https://bit.ly/2RPmJn9)

Slots are the communication interface for executors and are used for receiving commands and

sending results to the driver.

Wrong, executors communicate with the driver directly.

Slots are dynamically created and destroyed in accordance with an executor's workload.

No, Spark does not actively create and destroy slots in accordance with the workload. Per executor,

slots are

made available

in accordance

with how

many cores per executor (property

spark.executor.cores) and how many CPUs per task (property spark.task.cpus) the Spark

configuration calls for.

A slot is always limited to a single core.

No, a slot can span multiple cores. If a task would require multiple cores, it would have to be

executed through a slot that spans multiple cores.

In Spark documentation, "core" is often used interchangeably with "thread", although "thread" is the

more accurate word. A single physical core may be able to make multiple threads available. So, it

is better to say that a slot can span multiple threads.

To optimize I/O performance, Spark stores data on disk in multiple slots.

No – Spark stores data on disk in multiple partitions, not slots.

More info: Spark Architecture | Distributed Systems Architecture

Result of correct code block:

+-------------------+

|attributes_exploded|

+-------------------+

|

winter|

|

cooling|

+-------------------+

To solve this question, you need to know about explode(). This operation helps you to split up arrays

into single rows. If you did not have a chance to familiarize yourself with this method yet, find

more examples in the documentation (link below).

Note that explode() is a method made available through pyspark.sql.functions – it is not available as

a method of a DataFrame or a Column, as written in some of the answer options.

More info: pyspark.sql.functions.explode — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 2,

Correct code block:

transactionsDf.select(["transactionId", "predError", "value", "f"])

The DataFrame.select returns specific columns from the DataFrame and accepts a list as its only

argument. Thus, this is the correct choice here. The option using col(["transactionId", "predError",

"value", "f"]) is invalid, since inside col(), one can only pass a single column name, not a list. Likewise,

all columns being specified in a single string like "transactionId, predError, value, f" is not valid

syntax.

filter and where filter rows based on conditions, they do not control which columns to return.

Static notebook | Dynamic notebook: See test 2,

Executors stop upon application completion by default.

Correct. Executors only persist during the lifetime of an application.

A notable exception to that is when Dynamic Resource Allocation is enabled (which it is not by

default). With Dynamic Resource Allocation enabled, executors are terminated when they are idle,

independent of whether the application has been completed or not.

An executor can serve multiple applications.

Wrong. An executor is always specific to the application. It is terminated when the application

completes (exception see above).

Each node hosts a single executor.

No. Each node can host one or more executors.

Executors store data in memory only.

No. Executors can store data in memory or on disk.

Executors are launched by the driver.

Incorrect. Executors are launched by the cluster manager on behalf of the driver.

More info: Job Scheduling - Spark 3.1.2 Documentation, How Applications are Executed on a Spark

Cluster | Anatomy of a Spark Application | InformIT, and Spark Jargon for Starters. This blog is to

clear some of the… | by Mageswaran D | Medium

More info: pyspark.sql.DataFrame.withColumnRenamed — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 2,

Decrease values for the properties spark.default.parallelism and spark.sql.partitions

No, these values need to be increased.

Increase values for the properties spark.sql.parallelism and spark.sql.partitions

Wrong, there is no property spark.sql.parallelism.

Increase values for the properties spark.sql.parallelism and spark.sql.shuffle.partitions

See above.

Increase values for the properties spark.dynamicAllocation.maxExecutors, spark.default.parallelism,

and spark.sql.shuffle.partitions

The property spark.dynamicAllocation.maxExecutors is only in effect if dynamic allocation is enabled,

using

the

spark.dynamicAllocation.enabled

property.

It

is

disabled

by

default.

Dynamic

allocation can be useful when to run multiple applications on the same cluster in parallel. However,

in this case there is only a single application running on the cluster, so enabling dynamic

allocation would not yield a performance benefit.

More info: Practical Spark Tips For Data Scientists | Experfy.com and Basics of Apache Spark

Configuration Settings | by Halil Ertan | Towards Data Science (https://bit.ly/3gA0A6w ,

https://bit.ly/2QxhNTr)

DataFrame.select()

Correct, DataFrame.select() is a transformation. When the command is executed, it is evaluated lazily

and returns an RDD when it is triggered by an action.

DataFrame.foreach()

Incorrect, DataFrame.foreach() is not a transformation, but an action. The intention of foreach() is to

apply code to each element of a DataFrame to update accumulator variables or write the

elements to external storage. The process does not return an RDD - it is an action!

DataFrame.first()

Wrong. As an action, DataFrame.first() executed immediately and returns the first row of a

DataFrame.

DataFrame.count()

Incorrect. DataFrame.count() is an action and returns the number of rows in a DataFrame.

DataFrame.show()

No, DataFrame.show() is an action and displays the DataFrame upon execution of the command.