Using IntelliJ with a screen reader

Answered

I've been using Jetbrains products for years now (starting ReSharper, but mostly recently mainly PyCharm, DataGrip and WebStorm, as well as a little on IntelliJ itself and Android Studio). I'm also losing my sight, so need to start preparing for when that happens.

So I'm wondering if there are any users on these forums that maybe have any experience with screen readers that can advise me.  Or even anyone from Jetbrains who can comment or point me at any resources that might help.

I mainly use a Mac and am trying to learn VoiceOver.  I posted about some problems I had using the toolbox app, and then PyCharm in the toolbox app forum (https://toolbox-support.jetbrains.com/hc/en-us/community/posts/360009750820-Using-Toolbox-App-with-OSX-VoiceOver-Catalina-) but didn't get a reply.  Since then I've read some things that suggest that VoiceOver is no good for coding.  I've also read that IntelliJ based IDEs aren't the best for screen readers.  Certainly my limited experience with the 2 together was not good, and the toolbox app appears completely unusable with VoiceOver as far as I can tell.

I'd like to keep coding on my Mac, and I'd like to keep using Jetbrains products, but if either is a no-go then I'd like to know sooner rather than later.  Any advice at all would be great.

76 comments
Comment actions Permalink

@Serge Baranov thanks for that I set

ide.mac.file.chooser.native=false and IntelliJ hasn't hanged since I restarted (Maybe just lucky so far)? but seems like that did it.  I will make a point to create some IntelliJ/voiceover issues this weekend, have been meaning to for a long time but just haven't had the time...

0
Comment actions Permalink

I've been using VoiceOver a lot more recently.  I am still using the mouse a lot but am trying to push myself further into the VO territory.

And using PyCharm is an incredibly frustrating experience.  This is with Monterey and PyCharm 2021.2 latest.  I know there are some VO fixes in 2021.3 but I find the mainline releases buggy enough without going into EAP territory but the fixes don't sound very major.

Without a doubt, the number one problem is menus.  Firstly, I can't find a way to open the context menu (VO+Shift+M doesn't work in PyCharm unlike other apps).  But if you manage it, or use one of the other popup menus like the quick fix menu etc then the experience is pretty horrible.

If the menu has a submenu, the top level menu item won't be read at all.  E.g. right click and select "Git" in text editor, and "Git" isn't read even though the submenu is.

Sometimes the first item in the menu won't be read at all.  Sometimes VO is quiet, sometimes it just seems to go berserk like it's trying over and over again to read it never getting beyond the first syllable.  (Admittedly not noticed this last one much recently.)

Some menus it won't read at all.

And using the Project Explorer is a test of patience.  It doesn't tend to crash much these days, but it's constantly Busy or Not Responding.  Performing a simple task here takes ages.  And don't dare to use VO+F to find text.

And then there are a lot of general problems - all over the place.  The new commit warnings doesn't seem to be accessible at all.  (It just says "refresh" if there's a problem).  This worries me because if new features are being added that rean't accessible, then we're never going to get to the end of it.

The dialogs are completely inconsistent.  Some work fine, some don't.  Some dialogs like Cmd+Shift+V I or Cmd+F I can type my search string in immediately.  Some like Shift+Shift I seem to have to navigate for miles to get to the text box.  Shift+Shift is one I have programmed myself to use ove the years and was a real highlight of PyCharm, but now I just find it annoying to use and end up using the mouse.

And I've written before about the number of things that just happen without telling you such as errors running a Datagrip query.  Or even just trying to navigate to the results of the query if it works.

It drives me virtually to tears every day I use it.  Even the bits that work are just so inconsistent and require so much to remember on top of everything else.

And the toolbox app still seems completely unusable with VO, or indeed the keyboard, unless I'm missing something.

Oh and there are times when it just breaks altogether and refuses to read any text in the editor..  And God help you if PyCharm decides to do a big reindex with VO enabled.  I accidentally built to a non-excluded directory and I was totally unable to use PyCharm for the best part of an hour.  (Presuming that reindex is to blame)

Don't get me wrong, PyCharm isn't the only thing that is frustrating me with VO.  Spotify, Chrome and various others are also very tricky.

I would say that on the accessibility scale... if Sublime Text is a 0/10 (doesn't work at all), and IntelliJ was a 1/10 (borderline impossible to use) before this work started, then we're probably in the range of 3/10.  It can be used but it's no fun.

I've raised a number of bugs in the issue tracker but they don't seem to get much attention.  If i raised every bug I found it would be a full time job and already my productibity is being seriousl impeded by this.  Feels a bit like dropping a pebble down a well.

Again, just to reiterate that I am still very much a newbie VO user, and PyCharm is very much not a great choice for a practice area.

I would also like to add that I am so grateful that at least problems get acknowledged and I think there is a genuine will to improve it.  But I have paid an awful lot of money over the years for Jetbrains products.  It's mostly frustrating because I don't want to abandon them - I've used them for years because for a sighted user they are the best IDEs you can get.  Just still a long-way from being a good experience for a VO user in my opinion, I'm sorry to say.

Apologies for the rant but today hasn't been a good day for VO and PyCharm and it tipped me over the edge.  It's been coming though.  :)

 

 

 

0
Comment actions Permalink

Forgot another couple of things....

If the quick-fix menu opens a 2nd menu then that other menu isn't read out.  E.g. if I have an unimported function and, open the menu and choose Add import (which is read out) and then it opens up a 2nd menu to confirm where to import from - these other options are not read.

And the Ctrl+Tab tab switcher is very hit and miss.  Sometimes it will just flip to the previous tab, other times it opens up a dialog.  As far as I can tell, this dialog isn't accessible.  I can't use arrow keys to select something, but I can use VO keys.  But I can't find a way to activate the item that's selected without using the mouse.  And I can't get rid of the dialog with ESC.  As far as I can tell, the only way to do anything with the dialog is to use the mouse or by pressing Ctrl+Tab a few more times until eventually it gets the hint.

0
Comment actions Permalink

Hi John C

Thank you for the provided feedback!

If the quick-fix menu opens a 2nd menu then that other menu isn't read out.  E.g. if I have an unimported function and, open the menu and choose Add import (which is read out) and then it opens up a 2nd menu to confirm where to import from - these other options are not read.

Thank you, we're investigating this.

And the Ctrl+Tab tab switcher is very hit and miss.  Sometimes it will just flip to the previous tab, other times it opens up a dialog.  As far as I can tell, this dialog isn't accessible.  I can't use arrow keys to select something, but I can use VO keys.  But I can't find a way to activate the item that's selected without using the mouse.  And I can't get rid of the dialog with ESC.  As far as I can tell, the only way to do anything with the dialog is to use the mouse or by pressing Ctrl+Tab a few more times until eventually it gets the hint.

WE have disabled this dialog for ctrl + tab, since in Swing this keyboard shortcut is used to bring focus out of panels that block focus.

If you'd like to provide more feedback please raise a new issue https://youtrack.jetbrains.com/issues/IDEA so that the responsible developer could take a look at it. (one issue per case)

0
Comment actions Permalink

Thanks very much @Egor Klepikov.  Sorry again for the rant, it just all got a bit much yesterday.  :)

The quick-fix menu thing - I had some instances today where it worked as I described.  And then some other times it was reading out the 2nd menu but I couldn't find a way to select the menu items, so had to use the mouse to click them.

But any improvements with menus would be very, very much appreciated.

0
Comment actions Permalink

@Egor Klepikov link. Please explain why you decided to disable the ctrl+tab switcher?  It used to work perfectly and was extremely helpful to switch to recently files?  Is there another workaround?  It definitely worked as expected earlier this year and then started behaving strange as @JohnC explained above holding ctrl and repeatidly presing tab no longer moved focus between the last used files.  I thought this was a bug in the latest EAP build as it just didn't work anymore.  Now I have to double tap shift and hope I remember the name of the class I am looking for when trying to find a recently used class...  Is it possible to re assign the switcher to option or FN+tab instead?

0
Comment actions Permalink

@Egor Klepikov link another concern:  I created at least 5 issues in the last two months and it's been completely silent/no feedback or comments at all.  So suggesting that @JonC create issues for the problems he encountered seems to be less effective than commenting here.  Pretty much the only feedback we ever get from  Jetbrains is when we comment on this thread.  Also with this switcher being removed for screen reader users is this sort of info published anywhere?

0
Comment actions Permalink
Hotkeys have been disabled in screen reader mode because it blocks navigation. For example, it is not possible to pull focus from a multi-line editor.
 
The switch can be reassigned manually, or you can use, for example, "recent files" by combining CTRL + E on windows, or Command + E on macOS.
0
Comment actions Permalink

I just upgraded to PyCharm 2021.3 and have immediately noticed 2 things.

1. VoiceOver no longer reads anything in the right click menu at all.  OK I can't open it without the mouse still, but I used to let VoiceOver read it once it was there.  Well except the bits it wouldn't read.

2. Ctrl+Tab no longer does anything at all. 

Otherwise only thing I've noticed is the lack of tooltips on mouse over (as advertised) which to be honest was the least of my worries with VoiceOver.

Now debating whether to go back to 2021.2.  

I upgraded to WebStorm 2021.3 yesterday I think and noticed the same problems.  But as I hadn't used it much with VoiceOver before I couldn't be sure that the problems hadn't already been there.  But it's definitely la step bacakwards for PyCharm.  :(

 

0
Comment actions Permalink
Thanks for your feedback!
 
As we reported earlier, the ctrl + tab combination is reserved by Swing for focus control. If we reassign action to it, Screen Reader users will lose focus control.
 
As for the context menu. Thanks for the information, we will take care of this, but for now you can assign a key combination for the context menu yourself, for this:
  1. Open action search cmd + shift + a on mac, or ctrl + shift + a on windows;
  2. Enter "show context menu" there;
  3. Press option + enter on Mac, or alt + enter and set the key combination.
 
0
Comment actions Permalink

@Egor Klepikov - sorry I completely missed the last couple of posts in this thread and didn't see your explanation.  Recent files looks useful and seemed to work with VoiceOver.  Thanks to your instructions I now have a shortcut I can use to show the context menu.  Sadly, it's completely silent when I use it so it's a bit useless but hopefully that will be fixed soon.

It would be nice if standard Mac/VO keys just worked.  One of the things that I've found most difficult with transitioning to VoiceOver is the bewildering amount of keyboard shortcuts I am having to remember - either to use VO itself or to use complicated apps like PyCharm.  There is so much for an old brain like mine to remember.  So the more things that work as expected the better.

I was reading about Fleet the other day and have signed up for the preview.  I'm hoping that with something built from the ground up that accessibility will be baked in from the start this time.

0
Comment actions Permalink

@Egor Klepikov - found another change is 2021.3.

Before if I pressed F2 I would be taken to the next problem and PyCharm would read out "text/html" and then what the problem was.  Now it reads out the text of the word that it has taken me to.  I can open the quick fix menu and scroll through to try to decipher what the problem is.  As per other menus it's a bit hit and miss whether it will read out the items in the menu or not.

Was this change intentional?  I think I preferred the old behaviour as now it's more cumbersome to figure out what the problem is.  (Even though the "text/html" bit was a bit unnecessary). I suspect it's an accidental side-effect of removing the tooltip popups.

0
Comment actions Permalink

I went back and retried it and i definitely prefer the old behaviour.  I had accidentally mistyped a function name I was calling.  F2 and it reads "equals".  The first time I tried it, it read out the name of the function.

 

 

0
Comment actions Permalink

....and another one.  I've configured notifications to read out the results of a test run so I know how many tests pass and fail.  Now I press ctrl+R to run the tests - I can see it happening but nothing gets spoken any more.  This isn't a voiceover specific thing as it is the same if I disable it.

Not all spoken notifications are broken.  The "would you like to add this file to git?" notification that appears if I externally add a file is still spoken.  I'd guess and say this was to do with the changes to the run window.

0
Comment actions Permalink

John C Thank you so much for your feedback. Please create a separate ticket for each of your problems in our youtrack.jetbrains.com . We'll check everything.

0
Comment actions Permalink

@Egor Klepikov - ok I will do when I get a chance but as has been mentioned previously usually bug reports just disappear into the ether.

I have at least noticed one improvement in 2021.3!  If I use shift+shift I can now stzrst typing straight away without having to navigate past all the tabs.  Much better!

0
Comment actions Permalink

I raised a few bugs and think they have at least been looked at.

The (non-VoiceOver) issue of not having unit test notifications read out any more can be fixed.  There is a notification "Test Runner" which works, whereas before I was using "Test Results (Run)" or something.  I'm not sure if it's possible to increase the speech raet of the announcement but would be nice.

The Ctrl+Tab problem was rejected because of the Swing issue which is annoying.  Cmd+E works well with VO but it's not the same thing.  I'm not sure if you can map Ctrl+Tab to "Next Project Window" - wasn;t sure if that might work given that it isn't a visual thing.  But I'm not sure how to use tab in a keyboard shortcut as it just changes the focus in the dialog.  (When I have more time I will try to hunt down where the config is stored and see if I can figure it out.). I know I could map it to something else but it works in pretty much every single other app on the mac.

0
Comment actions Permalink

The Toolbox App seems to have suddenly started to work (partially) in VoiceOver.  I can select it, open it up, choose an app and launch.  It doesn't seem to be possible to drilldown into an app and change options or update it or anything but it's a big improvement.  ("Update all" does work and you can turn on auto updates). Some of the buttons aren't labelled (e.g. options) but glad to see that it's no longer completely unusable.

0
Comment actions Permalink

Can anyone answer if we are ever likely to see a version of PyCharm that is genuinwely usable with VoiceOver?

There are just so many issues all over the place.  Other than most of the menus not working, the main thing winding me up at the moment is that I only need to look at the project window and PyCharm locks up.  Seriously, I think I only need to scroll a tiny bit in it and I'm back to waiting minutes for PyCharm to come back to life - if indeed it does.  I must have force quit a dozen times today.

I know it's been mentioned on here a few times so I presume somewhere deep in YouTrack is an issue gathering dust.  I'm loathe to create any more issues because they never seem to get fixed.

I'd love to see an announcement from Jetbrains that Fleet has actually taken accessibility seriously and was part of the original design but I'm not hopeful that it's going to be anything more than an afterthought.

I think I need to get my head checked that I am actually paying for this because using PyCharm right now is a totally miserable experience.

0
Comment actions Permalink

John C the issue with context menus and VoiceOver is currently in progress, we hope to get things done by April.

Seriously, I think I only need to scroll a tiny bit in it and I'm back to waiting minutes for PyCharm to come back to life - if indeed it does.  I must have force quit a dozen times today.

To help us to investigate this please provide a bit more details:

1. Does it freeze only with the VoiceOver enabled or the same happens without it? 

2. What actions exactly lead to the freeze?

3. Could you please share the complete IDE logs after you face this again? If you manage to get the PyCharm working after the freeze - or at least after the IDE restart - please capture the logs via `Help | Collect logs and diagnostic data`, upload to https://uploads.jetbrains.com/ and let us know the upload ID. Alternatively, you can capture thread dumps as per this article.

0
Comment actions Permalink

Ekaterina Valeeva - thanks for the reply.  If the context menu problem is fixed soon that would be fantastic!

Regarding the project tree, I am sure it is only happening with VoiceOver.  I think it also depends on how big the tree is.  Or possibly because the project I'm working on has a few different bits to it - different src and tests folders and different excluded pytest folders etc.  Using a much simpler project earlier and it worked OK.

When I say I only have to look at it, I'm not really exaggerating.  I don't dare use the tree with the keyboard, so typically I would use Mac soom to make it really big, and possibly touch the scroll wheel and then "Pycharm is not responding" over and over again.  I think even having the project tree open is asking for trouble when it gets like that.  Today I hid it all day long and worked around not being able to use it and it's been nowhere near as frustrating.  I maybe had to force quit once today.

I will try to find some time over the weekend to mess about with it more, and at the very least get a log file uploadeded.

Apologies for coming in here and ranting - I've just got a lot of work to get through and PyCharm was pretty much reducing me to tears yesterday.

0
Comment actions Permalink

Ekaterina Valeeva - I have now submitted logs 2022_03_12_3RrmdFc6fXYTLGdn 

What I did.... with no other applications open (other than Toolbox), I opened PyCharm.  It started my recently used project.  I waited a little then opened the project tool window and clicked the top level item.  I pressed down.  It froze for a few seconds then was OK.  I pressed down again and then it froze.  I left it a good number of minutes before force quitting.  I then opened it up and tried again with the same result.  I had to create the logs after I had reopened PyCharm because it locked up and I couldn't access the help menu.

I then disabled VoiceOver, clicked in the tree and could quite happily move up and down through the tree with no issues.  So I can confirm it only happens if VoiceOver is enabled.

0
Comment actions Permalink

@Ekaterina Valeeva I can confirm that this is a major issue in IntelliJ Idea as well.  I have both a intel and arm Mac (this issue is way worse on the Intel mac with the latest release Intel build).  It seems to be an issue with how voiceover gets updated when navigating through the file tree on the left.  When using the arrow buttons to navigate around very often IntelliJ freezes and voiceover just says "busy busy busy" I've managed to get IntelliJ Idea working again sometimes by turning off voiceover and waiting a while but mostly I have to force kill IntelliJ idea.  One more thing I noticed was that this happens very often when expanding a directory which contains multiple subdirectories.  If this issue and the menu items not being read could be resolved we might be 70-80% there in terms of having a useable/accessible IDE...  I've also reported several minor accessibility issues last year but it always goes unnoticed/gets de-prioritised it seems...  The navigation issue has gotten so bad that I now have a Windows machine next to my Mac just to navigate around in IntelliJ idea as my employer won't have much understanding if I don't deliver work...  Really hope this can be resolved soon...

0
Comment actions Permalink

I'm also finding that double-clicking a unit test in the test explorer seems to cause the same behaviour.  I'm not sure if doing so selects something in the project tree and is therefore the same problem.  But I have had to force quit a number of times today because of it.

0
Comment actions Permalink

John C Dn4141 Sorry for not getting back to you earlier.

I've checked the 2021.3.3 and 2022.1 EAP versions with different JBR versions and here are my results for the scenario when I navigate up and down the Project View using the keyboard arrows:

1. 2021.3.3 run with its default runtime (11.0.13) freezes indefinitely and cannot be recovered without killing the process.

2. 2021.3.3 run with JBR17 (17.0.2) from time to time demonstrates some lags, that is, VoiceOver says "IDEA not responding" but things get back to normal quite quickly, no deadlock.

3. 2022.1 EAP run with 11.0.14 behaves nearly the same as in pp.2.

4. 2022.1 EAP run with 17.0.2 does not freeze at all (or at least I could not reproduce any lags).

From what I learned from the responsible developer the freezes in trees with VoiceOver do not happen with JBR17 (which is going to be the default runtime for 2022.2).

If you would like to try your IDE with JBR17, you can find the detailed instructions e.g. here. JBR17 can be downloaded here.

0
Comment actions Permalink

Ekaterina Valeeva - that sounds awesome.  Thank you very much for getting back to us and trying it out. 

When you say the 2022.1 EAP do you mean IntelliJ IDEA?  I just upgraded my PyCharm to the latest EAP and it did become totally unresponsive quite quickly but maybe it doesn't have the fix in it yet.

Not tried updating the JBR runtime yet but I will give that a go when I have a little more time available.

If both this and the menu problem get fixed, it will make a huge difference.

0
Comment actions Permalink

Ekaterina Valeeva - I've got myself in a small muddle, but I think things are looking up.

I wanted to check which JBR I was using.  I totally missed your comment telling me which version was current and went to Choose Boto Java Runtime and it seemed to say my Current was 11.something which seemed a lot older than 17, so I clicked Default.  PyCharm restarted and I had a very quick play with the project tree and could move up and down it without any lock ups.

If I now check PyCharm about I am on 1.0.14.  Not 100% sure if this is different to what I had before.

I was asked to change JBR before when experiencing another issue. I thought I had switched it back but I'm not sure now.

Anyway, so far it looks good.  I'll keep an eye on it and only move to JBR17 if it starts to lock up significantly.  I'd rather stay on the default configuration so I don't get confused again!

Anyway, thanks for your help - fingers crossed but this seems a lot better so far!!

0
Comment actions Permalink

@Ekaterina Valeeva thanks!!! I'm extremely grateful for your effort, you literally saved me from losing my job as those deadlocks really started slowing me down and my managers are only concerned with Jira reports/visible output at the end of the day.... I just downloaded 2022.1 EAP on my Intel macbook pro and I haven't been able to get it to freeze at all.  (not sure if the EAP comes bundled with JBR17)?  If not I will have to give that a try as well as it's still not nearly as fast as my M1 macbook air when navigating around the file tree...

One more enhancement that would make our lives much easier is if voiceover could read the database tool table editor in IntelliJ idea.  I created an issue for that last year but it went unnoticed as far as I can tell...  I think this would be a minor change as voiceover already reads the contents of a selected cell in the editor if I cmd+tab out of the window and then cmd+tab back to the window, it just doesn't read the cell contents when I arrow around in a result set.  Do you have any idea if anyone is working on this or intending on in future?  This would be a major selling point for IntelliJ idea in terms of accessibility as I am not aware of any DB tool right now that lets you read the cell contents of a result set in the UI returned from a SQL query...

0
Comment actions Permalink

Dn4141 - interesting to know that you are (at least trying to!) use the DB features in intellij.  I used DataGrip a lot before using VoiceOver, but ended up finding it too cumbersome to use and have just given up.  Maybe it's improved, but I struggled to efficiently get between the query editor and results panes, and also found that syntax errors would appear by stealth and I'd never know about them.  I have raised a couple of youtracks on it but never got much action.  Sounds like you have got further than me anyway.  Maybe it's improved since I last tried.  My VO skills have gone from "none" to "a bit" so maybe that might help too.

Fortunately my DB needs are quite modest, so the mysql command line tool is good enough.  It might be horrible with VoiceOver, but at least it's fast to use.

0

Please sign in to leave a comment.