IntelliJ Database Query NotSerializableException when Selecting Map from Parquet File with Apache Drill Driver
In order to run SQL on some Parquet files locally I added the Apache Drill JDBC driver (version 1.18.0) to IntelliJ and connected to my local Drill instance by setting the URL to: jdbc:drill:drillbit=localhost. Using the Database tools I am able to query the files and see the output rendered in IntelliJ.
This works for all fields except one, that is of type MAP meaning it has the format: { key: value, key: value, etc } (JSON object)
If this is included in the select statement, for example, where results if of SQL type MAP:
SELECT results FROM dfs.`C:\...\*.parquet`;
It gives the following error: java.io.NotSerializableException: oadd.org.apache.drill.exec.util.Text
However, it works as expected when using Drill from the command line and UI. I know Drill isn't natively supported but does anyone have any idea what could be causing this and if there is anything I can tweak to get around it?
If I use FLATTEN(results) IntelliJ will render it successfully but then the map is split out into multiple rows which is not what I'm trying to achieve. I am currently using IntelliJ IDEA Ultimate 2020.3 but also tested with 2021.2 and experienced the same issue.
Please sign in to leave a comment.
I've created bug report on our tracker, please follow and vote to get noticed on any updates.
Thank you so much, I have voted so hopefully someone is able to help