View Spark data frame like Pandas Data frame
Answered
I'm currently doing most of my development using Spark Scala now with larger datasets and was previously using Pandas
One thing that doesn't seem obvious to me; how do I view a Spark data frame the same way I view a Pandas data frame in the IDE window?
Thx
Please sign in to leave a comment.
Could you please share a screenshot how it should look like and how it looks like now?
Hi Serge,
Here are s/shots.
Here is pandas - the dataframe is nice and easy to visualise
Here is a spark dataframe. When I try to view as, there is no alternative view that lets me view the data frame like the pandas s/shot above
Thx
Feature request is welcome at https://youtrack.jetbrains.com/newIssue?project=SCL. Thanks.
created https://youtrack.jetbrains.com/issue/SCL-19672