AI Limit reached. WTF.

Right when I got the exact same response I came googling this, if this is real. We're paying for the service and it blocks us from using it when we reach a certain amount. Don't recommend buying to ANYONE.

Hello,

We have an issue tracker for this: 

https://youtrack.jetbrains.com/issue/LLM-1587

If that doesn't work, please let me know!

That's really a joke, I'm a private person and want to learn php, buy a program instead of using a crack, buy the AI to learn scripting better and then ... bam... AI Limit... without finding any reference to it. This may be possible in your country, in Germany this is called customer deception! If I had known that, I wouldn't have paid a cent for it, an absolute impudence. I will report this to consumer protection

Since we are using different providers and models, it is not easy to present quotas. Many factors affect the usage calculation of tokens during the chat session. It depends mostly on the complexity of the interactions and the current context size. As described in the related YT issue: https://youtrack.jetbrains.com/issue/LLM-1587, sending all prompts in the same chat instance results in exponential growth of the context being sent to the LLM.

Usage limits are being reset every week. Though, there is an issue regarding incorrect timeframe ('monthly fair use') shown in the notification. The fix for the notification is implemented in the latest 2023.3.2 release. General UX is also being improved to help mitigate the issue and reduce the context size growth. 

The quota limit was created in a way that the amount is enough for regular (non-automated) usage. Reaching the quota limit is still possible in the case of very active usage. However, we do not expect it to happen often during regular use.

Verstehe ich das richtig, wenn ich eine anfrage sende lieber ein neues Chatfenster zu öffnen?

Do I understand correctly that when I send a request I prefer to open a new chat window?

I hit the limit,  in 5 hours ? i just brought it today as well, i noticed it wouldnt complete a main.cpp example of using the boost libraries,  it just refused, so i aked it about 40x to repeat ,  would it do it no it wouldnt,  chatgpt at least completes the conversation 

it has so much potential i will continue to subscribe in the hopes it improves. 

“The quota limit was created in a way that the amount is enough for regular (non-automated) usage. Reaching the quota limit is still possible in the case of very active usage. However, we do not expect it to happen often during regular use.”

I think this is way over-optimistic. If it works and people like it they will use it more. 

At the moment I have changed my habits to mix much more ChatGPT into the mix to avoid usage limits. That's not you getting things right with your estimates, that's me trying to avoid being hit with an abrupt ban.

yes im heavily dependant on it,  as an education tool as well as a helper,  but i think i will do as you do mix the two ,   use in context approaches on both and you will see the parralles in its responses,   i think i hit my limit because i asked it to repeat its self about 100x in total because it just would not complete the entire code example for  boost::filesystem boost::system chronos , regex and the other boost main libs,   in a main.cpp ,   approx 100 lines of code i imagine but only produced 20 at most, sometimes would stop after the 3rd sentance

Ajc200x 

i think i hit my limit because i asked it to repeat its self about 100x in total because it just would not complete the entire code example for  boost::filesystem boost::system chronos , regex and the other boost main libs,   in a main.cpp ,   approx 100 lines of code i imagine but only produced 20 at most, sometimes would stop after the 3rd sentance

Have you maybe tried to wait a bit longer when the freeze in the response generation happens? We had a similar issue in a previous version of the plugin: https://youtrack.jetbrains.com/issue/LLM-856, however it was resolved in the official release (2023.3). In the case of the issue, the chat stops generating the response in real time, and after some time (minute or more) it would print out the entire response at once.  

Could you maybe share a sample project or steps to reproduce the issue consistently?
Also, which version of the IDE and AI Assistant plugin are you using? Thanks!

To add to Ivan's response, I've always had it freeze on medium-long responses after about 10-20 messages back and forth in the same chat. Creating a new chat resolves it immediately, until I reach that 10-20 message threshold again.

I can also tell it “Continue” during the freeze and it will pick up where it left off. Never tried waiting a minute or so, as it didn't even say “Generating response…”

i will be more patient,  and see how it responds

Styris 
Thank you for the information.
Could you share some sample project/prompts that you used when you reproduced the issue with chat freezing?

Ivan Pajic Absolutely. Looking back it starts freezing after 40 or so messages.

I shared the logs to uploads.jetbrains.com, here's the ID:
2023_12_28_2Ao3yFdD2R2uMnq5Pz7HAQ

Issue starts at L:747 in Chat-Freeze-1.txt, and L:985 in Chat-Freeze-2.txt. The issue occurs across any project.

Styris 
Thank you very much for the data. 
We were able to reproduce the issue when the response generation stops in the middle: https://youtrack.jetbrains.com/issue/LLM-1892. 

This has to be the silliest thing I've ever purchased, I get half a day a week usage because you have made no attempt to educate users on how to use it properly, I use single chat for everything why would I randomly switch chats for every question? What other AI platform follows this rule? Fair use is not half a day a week or 2 days out of 31, you can't justify a subscription cost you may as well charge us per request, submitted for a limit increase weeks ago with no reply, awful experience avoid at all costs

Had no idea I could wait a minute and let it finish, half my requests are asking it to repeat it's self but okay I guess I hit my unfair monthly limit, imagine you got YouTube premium and you were limited to 2 days of videos a month 

I run into this a few times. Granted it was always after complex questions in on chat involving JPA, Springboot and Vaadin. 

For IntelliJ folks, I'll explain my issue, perhaps that will help you.

The IDE gave some answers, but it was obvious to me, that those were related to Vaadin < 14 and not the current one Vaadin 24. So I got tons of responses (halluzinations), that I must be wrong as something like Vaadin 24 doesn't exist. I continued asking, and then I was told that my version of Springboot is wrong, as it doesn't exist yet … Ok, meanwhile I learned that the responses are accurate when I will be using the term “Vaadin14+” … But I am encountering such fights quite often … 

Similarly I encountered more than once that I asked quite a simple question, but the AI responded with tons of information, unfortunately all wrong and useless … 

I think most of us understand that AI services cost you a lot of money to too and you have to find a “balance”, but it will definitely help if IntelliJ will understand their customers too. Support needed in my case a total of 3 days to get my paid account reactivated … and now all my former chat are gone :-(

I was surprised that during evaluation period I NEVER run into this issue. THAT was the main reason for me to pay immediately a yearly subscription, as I do know that your cost are likely relatively high. Please understand that it is then more than frustrating when your paid service is worth than the beta version was ;-). At least I wasn't informed nor prepared to such a quota. 

Ivan Pajic 
I just made another observation. IntelliJ was 2 days before the ban quite irresponsive from time to time and it was the first time that my M1 got warm. I searched for few hours for all kind of root causes, but could not find anything, Now after the license is reactived, the old chats are gone, the very same project is all the sudden responsive again … When the chats were still there, but the license no longer working IntelliJ was still slow. BTW, technical support could only help me via suggesting to in-activate the license and re-activate again.

This weekly limit thing is extremely annoying. I spent $100 on a subscription to AI in addition to my monthly All-Pack subscription…and I can't use it anymore till next week? PLEASE CHANGE THIS. I'm trying to use it for my commit messages and it's not letting me.

I completely agree. But after a little thought, I partly understand jetbrains because they are also dealing with black sheep who abuse their systems. Robots etc.

I can also confirm that I myself had unintentionally sometimes exhibited behavior as described above that increases “consumption” exponentially.

For example, AI gave me non-sense replies when I asked a question, then it was finally on the right path and I asked in the same(!) chat for a clarification but then the first answer was all the sudden picked up again and AI drifted off. When I tried to put AI back to the right spot there were quick "dialogues" in a short time considering the same topic, which might be considered as "always asking the same question" :-(

But yes, I am fine that jetbrains will get informed in such a case and that they check if we are really abusing the system. But they could probably throttle the performance first etc. Cutting the account completely out-of-the blue is VERY annoying, especially when you are losing all your chats which happend to me now 2 times ...

The experience is also frustrating for me because there were never such problems during the free phase. Then, as a paying customer, you are of course very disappointed, especially since, to my knowledge, such a quota has never been mentioned anywhere, and I don't know of any other AIs that I pay for.

I think it's a learning phase for both parties. I realize that Jetbrains doesn't want to be ripped off (that would just significantly increase the prices for all of us), but on the other hand, jetbrains should also understand that an AI that can only be used 3 days a week, is pretty meaningless. And each of us likely has phases in which we work heavily with AI and then phases in which we use it less. I see this weekly limit as very critical; I would almost rather prefer a daily limit. Or perhaps a ban for a few hours. That would help us as well to identify and report what messed up the system … 

Here's an idea JetBrains from a dev who uses ALL your IDEs daily for multiple languages:
If JetBrains is worried about abuse of the system, when reaching a limit probably it should prompt you to log into your JetBrains account and releasing the hold yourself manually. That should address any bots or other things trying to abuse the system. I had to idea there was even a limit until I was in the middle of my working day yesterday and “Sorry….no more questions till next week…”. I LOVE the new JetBrains AI a lot. I wish my new AI friend didn't get tired of me so quickly. haha

yes, it is partially a problem that their AI is too good in many cases ;-) - I would not admit that I am addicted but very close to it - haha. 

However I have one suspicion, sometimes the responses (as those of all AIs currently) are a bit outdated. They “traps” I encoutered so far are always that AI believes I'm Using Vaadin 8 (current version is 24) or Spring Boot 2 (instead of 3) etc.  This is were our new friend completely freaks out and you are getting tons of rubbish, but AIs seems to be very stubborn then and wants to convince you that are using a version, which doesn't exist yet. 

Instead of just replying “Sorry, I cannot help you on this version” it's starting to halluzinate. The responses are then still very polite but again: very, very, very long but completly non-sense.

I have the same issue. I created a ticket but could not get an answer yet. It is down for the last 24 hours. I am trying the github copilot now. There is no place where I can check quotas or limitations. The service can stop anytime without any notice. This is not the way it should be.

I started to get the notifications on Sunday night, and on Monday I still get the weekly quota reached warning. The error messages are vague too. So vague limits with vague responses is what I got up to now.

Edabaykam Sorry for the inconvenience. I checked and your request for quota extension was processed. 

Just bought the AI Assistant and got the same message on my very first chat message. I literally haven't used it one time. Then I hit a fair volume of JetBrains AI. I will be refreshed in 3 days. This really bogs down my hype with being a lifetime JetBrains Customer as I have for years already. Too big of a company to have production code errors.

That just sounds like a weird bug to me. I haven't hit a limit for a while now, it's being working perfectly.
 

That said, I should note to the developers. Due to the existence of this notional limit I have changed my usage patterns. I now throw stuff into chatgpt free quite often to mitigate the chances of hitting the limit again. I don't like changing my behaviour in this was ie feeling wary of using the tool I paid for just in case.

Reaching a point where it is unnecessary should be the goal. ie if a few developers use it more than you would like then so what, so long as the vast majority don't. Much better for keeping people onside.

Danielsoriano559 
That seems like not expected behavior. Which IDE are you using? There is a similar known issue in ReSharper. Could you please file a support request: https://intellij-support.jetbrains.com/hc/en-us/requests/new?ticket_form_id=66731 to investigate further?