Hi Jetbrains team,
As a customer and heavy user of Datagrip for a few years, I've been recently rather mispleased with the roadmap of the product, so I wanted to share my feedback which I hope represents a part of industry (and user base) thas is being overlooked. Don't get me wrong, I still believe Datagrip is the best product currently available, but it's such a shame the potential is not leveraged.
Let me explain a bit of background for you to understand where I'm coming from with this feedback. I'm a Data Warehousing (DWH) expert, over the last decade involved in building analytical solutions for a number of clients across industries. Based on my experience, working in different roles, I understand the product can have following groups of users:
a) DWH Developers - In DWH world those are the people most concerned about creating data pumps (ETL/ELT), translating mapping documents from analysts into data pumps, heavily SQL focused. I think the product does well for this target group, as developers are used to work with rather complex and technical tools. However, some of the needs in this area are not respected.
b) Data modelers - Those are the designers of database objects. As the approach to building DWH tends to be model-driven architecture (MDA), modelers would use tools such as PowerDesigner to construct and iterate from general ideas of business concepts down to physical tables and columns (conceptual/logical/physical model). DDLs are then generated from the physical model and applied to database. Datagrip is only capable of a very basic visualization of the tables that provides no added value. The product doesn't help this target group in any way.
c) DWH analysts - The analysts are usually split into technical and business based on whether they have a good understanding of the technology involved or rather the business terms used, also wheter they work closely with the development team or sit in business deparments. DWH Analysts in general have the need to perform complex ad-hoc queries on multiple databases using different accounts in order to find root causes of issues, elaborate new features as well as check whether each data consumer group has proper access. While analysts close to technology can get effiecient in the product quickly, others would struggle.
e) DataOps - Those are the people that have full access to production and from time to time can make slight adjustments to the data, usually by running a script prepared by Developers and/or maintain CI/CD pipelines for quick propagation of DMLs and DDLs to higher environments.
f) Data Scientists - I will not cover as those are either using SQL similarly to analysts or Python/R or else using other products.
As Datagrip is commercial-only product, it is unlikely it will get into hands of DWH experts. We work in companies that only accept software they purchased or freeware as long as it passes security checks, since we are working with sensitive data. I've been using the product on my work computers only because I pay for the license without any contribution from the company, install it myself in a hacky way and my managers tolerate me using it. A different case is Pycharm, because it has the free version, I've seen a couple of colleagues using it - outdated version installed from company approved site. Pycharm Community is actually how I learned about other Jetbrains products and led me to using Datagrip.
Now to the actual feedback on features released in 2020:
1. Most of the new features are concerned with ancient technologies like MySQL, Postgres, DB2, etc. In DWH world the leading technologies are Teradata, Redhshift, Snowflake, BigQuery, MSSQL, etc. There is lack of support for these, some are ignored, others have buggy connections, syntax highlighting. There is lot of features aimed at improving coding of stored procedures which are rarely used in modern DWH. I can also see a lot of investment into NoSQL features and nested stuff that is rarely used for DWH.
2. Introspection is really annoying. We tend to use only a few databases, usually split by environment DEV/TEST/PROD. But we need multiple users to run analysis and checks. As there is only one user per connection, this creates tens of connections to the same physical tables. After I start Datagrip I have to do something else for about 20 minutes until all connections settle down.
3. DDLs as I explained above are rarely done by coding it. DMLs directly on database are not allowed unless you're DataOps.
4. Some of the features brought in are a bit of a joke in DWH world - "Images in database", "Export to Excel".
5. What I do appreciate are improvements in the General Dialect. Due to the lack of database support, this is the dialect I suspect is most heavily used by DWH experts. Please focus on not breaking this one as it happened multiple times.
6. An older feature "JOIN completion" came in with big announcement a time ago, so I was really excited. It never worked as expected, so we're still typing JOINs manually. I think this has something to do with the way you're testing those features on databases that have foreign keys and/or joins within same schemas. Having database constraints is anti-pattern for DWH achitecture and JOINs happen frequently across different schemas of a DWH.
Please let me know if some of the points above need more clarification and I wish you all well at JetBrains.