Databricks Updated Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Exam Questions and Answers by renesmae

Explanation

Correct code block:

itemsDf.join(transactionsDf, itemsDf.itemId==transactionsDf.transactionId)

The join statement is incomplete.

Correct! If you look at the documentation of DataFrame.join() (linked below), you see that the very first argument of join should be the DataFrame that should be joined with. This first argument is

missing in the code block.

The join method is inappropriate.

No. By default, DataFrame.join() uses an inner join. This method is appropriate for the scenario described in the question.

The join expression is malformed.

Incorrect. The join expression itemsDf.itemId==transactionsDf.transactionId is correct syntax.

The merge method should be used instead of join.

False. There is no DataFrame.merge() method in PySpark.

The union method should be used instead of join.

Wrong. DataFrame.union() merges rows, but not columns as requested in the question.

More info: pyspark.sql.DataFrame.join — PySpark 3.1.2 documentation, pyspark.sql.DataFrame.union — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 3, QUESTION NO: 44 (Databricks import instructions)

Explanation

The issue with all answers that have "broadcast" as very last argument is that "broadcast" is not a valid join type. While the entry with "right_outer" is a valid statement, it is not a broadcast join. The

item where broadcast() is wrapped around the equality condition is not valid code in Spark. broadcast() needs to be wrapped around the name of the small DataFrame that should be broadcast.

More info: Learning Spark, 2nd Edition, Chapter 7

Static notebook | Dynamic notebook: See test 1, QUESTION NO: 34 (Databricks import instructions)

tion and explanation?

Explanation

This QUESTION NO: targets your knowledge about how to chain filtering conditions. Each filtering condition should be in parentheses. The correct operator for "or" is the pipe character (|) and not

the word or. Another operator of concern is the equality operator. For the purpose of comparison, equality is expressed as two equal signs (==).

Static notebook | Dynamic notebook: See test 2, QUESTION NO: 21 (Databricks import instructions)

Explanation

Filtering out distinct rows based on columns is achieved with the dropDuplicates method, not the distinct method which does not take any arguments.

The second argument of the join() method only accepts strings if they are column names. The SQL-like statement "itemsDf.itemId==transactionsDf.productId" is therefore invalid.

In addition, it is not necessary to specify how="inner", since the default join type for the join command is already inner.

More info: pyspark.sql.DataFrame.join — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 2, QUESTION NO: 53 (Databricks import instructions)