Hi, my company has tens of thousands of tables within their TERADATA schemas, with some of these tables having hundreds and hundreds of columns. This means it's not possible to run a TOP * or similar due to user resource quota restrictions. Aside from this, due to the sheer size of the catalog it takes hours and hours to run. After those hours, it will inevitably fail when it reaches a table that has too many columns to inspect. I need to be able to restrict introspection to only the tables which I need to use but I've found nothing on this feature anywhere I have searched so far.
Things I've tried:
- Object filters. I started by trying to filter objects using connection properties->schemas->object filter. I set this to whitelist a single table as a test. Introspection ignores this and tries to run on every single table anyway. I understand that the first level will need to run (get all table names) in order to filter the object list, but once that initial catalog is built the introspection should honor the filters set by users.
- Right-click refresh. I've tried disabling auto-sync in the connection properties->options, then closing the datagrip program completely and re-opening it, then navigating down to the table, right-clicking, and refreshing. This starts an introspection on the entire connection again and doesn't honor the object choice I selected.
So far I've found no other options which seems like it may work. Unfortunately I'll need to advise my company that datagrip should not be purchased if there is no way to specify the objects which should be introspected, as it simply consumes far too much memory and time, and fails when it runs into a table with too many columns to inspect.