DataGrip Snowflake - Infinite Introspection

We checked the logs and we may confirm your concern. You have 90 databases in total, and it looks like you have set all your databases and schemas to be introspected (see your settings in Schemas tab of your data source properties). Now, if you trigger a full data source introspection by clicking Refresh, it will go through all the objects you have selected. However, if you modify a specific object within a schema, it should use a narrower scope as long as you have the Auto synch option turned on in the Options tab of your data source. Please check your settings and see if this changes the synching performance

I tried this by limiting number of databases and its still performing infinite loop when introspecting the last database (on auto refresh or when DDL Changes made).

However, when I manually refresh a schema or object , the introspection completes.

1. My previous employer's Snowflake instance was on AWS. My current employer's instance is also on AWS.
   In DBE-17268, he also says:
   1. “This appears to only be occurring for Azure and Snowflake. We have the same exact warehouse, db etc. set up on AWS and I don't run into any issues. Introspection works fine for the AWS connection.”

   2. “But again, I don't seem to have any issues w/ AWS. It's only w/ Azure.”

       

2. Agreed with TheMartialOne.

    

3. Also, I can confirm that if I uncheck “Auto sync”, there are no problems. Whenever I trigger a manual refresh / sync, everything works great. If I enable “Auto sync”, it introspects indefinitely on the last database.

Additional reference: https://youtrack.jetbrains.com/issue/DBE-20377

Just here to say I have the same thing occurring with Snowflake databases, the last item in the database explorer will constantly loop introspection.  

Datagrip: Build #DB-241.17890.24, built on June 7, 2024

Driver: Snowflake 3.14.4 stable

I've submitted the same thing about a year ago in another ticket. Happened for my machine and those of all my team members. Some were on M1 Macs, some M2, some Intel. Varying versions of macOS and DataGrip. Always happened. Went back and forth submitting different logs to support.

A year later, with a new employer (different Snowflake account and identity provider), and I see it's also still happening in IntelliJ IDEA Ultimate.

macOS 14.6.1 (23G93)

Snowflake 8.33.1

IntelliJ IDEA 2024.2.1 (Ultimate Edition) Build #IU-242.21829.142, built on August 28, 2024

I would upload my logs, but those contain every schema name and database name in our environment.

I've also tried various driver versions, mostly recently 3.14.4 and

ver. 3.18.0, JDBC4.2

The upload ID is 2024_09_12_KonSxhqGXfZPQu47U47usU. I also had to scrub those files. The files in there are:

continuous_introspection__introspector.0.0.log

continuous_introspection__idea.log

I looked at the SQL log file, and it's just an endless repeat of:

-- 2024-09-12 10:05:17 [<project_name_redacted>] [<data_source_name_redacted>] [introspection] [session id: 1583337601] [statement id: 1546963807] 
show <object_type_name_redacted> in schema <fully_qualified_schema_name_redacted>

Would really, really love to see this resolved! Makes it difficult to recommend DataGrip / IntelliJ (I encourage those at each company I work to use these tools), when this is constantly kind of bugging the user.

I opted not to upload the SQL Log for the same reason as Jared, out of concern for professional privacy and scrubbing effort.  The next request - “Okay, then upload your Idea log instead!” does nothing to address that concern, since the Idea log, too, contains all Schema/Database names.  It's (perhaps) understandable for the logs to contain that private data in the first place - but NOT understandable for the support request to just gloss over the privacy concern.  

That said - this issue is not a question of performance (how long it takes) it's a question of faulty operation (the fact that it never completes).  In my situation, I have 10 databases with 1 schema selected in each (out of 50 total databases with 5 schema each), and the tenth one will always loop introspection forever (hours without completing).  So I uncheck the schema in the tenth database, and then the NINTH one loops introspection forever.  And then I uncheck the schema in the ninth database and…….

As clearly indicated in the original post - “it loops on the last database I have selected”.  Doesn't matter if it's the 5th database, or the 90th - the last one will always loop forever.  

Thank you for the review of this issue.

Thanks for the input, Thomas.

We could not reproduce this issue on our end. To diagnose this behavior, we need the original logs with the database
objects specified in the file. This will help us to investigate this incident in greater detail.

I'm unable to share logs due to my companies security policy.  However, I can tell you that I'm using externalbrowser as my authenticator.  Not sure if you tried replicating using that.

Our snowflake is also on AWS

I've also been using externalbrowser as the authenticator at both employers.

Same for me - externalbrowser is the authentication method for this project.  

MattA @thomas TheMartialOne 

I created a new YouTrack ticket for this problem, DBE-22088, and asked our developers to look into it. Maybe the recent bit of information about the "externalbrowser" auth method will prove helpful in identifying the culprit.

You are welcome to follow the progress and leave additional comments there directly. Thanks.