Skip to Main Content
Apache Spark Quick Start Guide
book

Apache Spark Quick Start Guide

by Shrey Mehrotra, Akash Grade
January 2019
Beginner to intermediate content levelBeginner to intermediate
154 pages
4h 31m
English
Packt Publishing
Content preview from Apache Spark Quick Start Guide

DataFrame operations and associated functions

DataFrames support untyped transformations with the following operations:

  • printSchema: This prints out the mapping for a Spark DataFrame in a tree structure. The following code will give you a clear idea of how this operation works:
//Scalaimport spark.implicits._// Print the schema in a tree formatsales_df.printSchema()//Javaimport static org.apache.spark.sql.functions.col;// Print the schema in a tree formatsales_df.printSchema();#Python# Print the schema in a tree formatsales_df.printSchema()

The output you get should look like this:

  • select: This allows you to select a set of columns from ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Debugging Apache Spark

Debugging Apache Spark

Holden Karau
Apache Spark 2.x for Java Developers

Apache Spark 2.x for Java Developers

Sumit Kumar, Sourav Gulati
Stream Processing with Apache Spark

Stream Processing with Apache Spark

Gerard Maas, Francois Garillot

Publisher Resources

ISBN: 9781789349108Supplemental Content