Raj, Author at SQL & Hadoop

SQL on Hadoop with Hive, Spark & PySpark on EMR & AWS Glue

Hadoop & Hive – Introduction for Beginners

Hadoop is a very popular framework for data storage and data processing. So it suffice two main purposes: Distributed Data Storage using HDFS ( Hadoop Distributed File System) Data processing using Map-Reduce. In Hadoop everything is in File format. It […]

SQL on HadoopBy Raj1 comment

SQL on RDBMS to SQL on Hadoop/Cloud

We all have been using SQL on RDBMS for so long now. The time has come when we shall switch to SQL on Hadoop/Cloud. SQL (Structured Query Language) help us in communicating with any RDBMS like Teradata, Oracle, Netezza, SQL […]

SQL on HadoopBy Raj0 comments

Raj on PySpark-How to Generate MD5 of entire row with columnsMarch 9, 2023
Done. Please check now.
Anand on PySpark-How to Generate MD5 of entire row with columnsFebruary 25, 2023
can you please make the video available to learn
Raj on Free Online SQL to PySpark ConverterAugust 9, 2022
Thank you for sharing this. I will give it a try as well.
John K-W on Free Online SQL to PySpark ConverterAugust 8, 2022
Might be interesting to add a PySpark dialect to SQLglot https://github.com/tobymao/sqlglot https://github.com/tobymao/sqlglot/tree/main/sqlglot/dialects
Meena M on Spark Dataframe WHEN caseJuly 28, 2022
try something like df.withColumn("type", when(col("flag1"), lit("type_1")).when(!col("flag1") && (col("flag2") || col("flag3") || col("flag4") || col("flag5")), lit("type2")).otherwise(lit("other")))