PySpark apply function to column

PySpark apply function to column in dataframe to get desired transformation as output. In this post, we will see 2 of the most common ways of applying function to column in PySpark. First is applying spark built-in functions to column […]

Read more

PySpark handle scientific number

What is scientific notation or exponent number ? Recently I was working on PySpark process in which requirement was to apply some aggregation on big numbers. The result in output was accurate however it was in exponential format or scientific […]

Read more