Observing a performance issue rendering query results with the database integration

Answered

We have some tables in an RDS instance where all the String columns are CLOBs. It's a byproduct of a JDBC sink connector from Kafka that is importing data from a Kafka topic to the RDS instance. The topic itself is created from a CDC (Change Data Capture) Kafka connector that is connected to the same table but the Strings are represented as VARCHAR in the source table. The query runs in milliseconds on both tables but takes over 10 minutes to render the results in IntelliJ from the table version with the CLOBs while the same query renders results almost instaneously from the source table with the VARCHAR columns. We observe the same behavior in DataGrip.

To confirm the issue was in IntelliJ we tested the same queries in SQL Developer and both versions return immediately. I postulated the issue might be some deserialization issue in IntelliJ with the CLOBs but I'm not certain. It's really the only difference between the two tables as the data is the same. We are using the CDC to capture data from multiple remote databases to a common system (Kafka) and using the JDBC sink connectors to consolidate all the data in a single schema so we can develop some cross-schema queries for use in ksqlDB.

0

Please sign in to leave a comment.