Scintilla 3.0.0 Broke Scrolling For Some Screenreader Software

[Davy Kager]

Hi,



Because of my visual impairment I use screenreader software on the PC. I also use Notepad++ for my text editing and coding, which in turn uses Scintilla.

After upgrading Notepad++ from version 6.4.x (Scintilla 2.7) to version 6.5.x (Scintilla 3.3) I ran into the following problem:

- Reading text line-by-line works just like it did in previous versions.

- Pressing down arrow on the last visible line shifts the display to make the newly selected line visible, but my screenreader software first briefly reports a blank line before showing the new text. It's kind of like a "blur", except it happens in text-to-speech and Braille.

- The time it takes for the newly appeared text to be read by the screenreader correlates to the cursor blink rate.



I soon found that this problem has to do with Scintilla. I compiled a few versions and found the commit responsible for the new behavior:

http://sourceforge.net/p/scintilla/code/ci/65e47ea72946778da13b01702483f1402c13a47c/

I realize that this is an old commit, my apologies. Notepad++ just hadn't updated their Scintilla library in...a while.



In particular, the change in ScintillaWin::ScrollText(int) is causing the problem. It was switched to use Redraw() which probably explains my screenreader showing a blank line until it can re-focus on the cursor. I haven't dug deep enough to fully understand what has changed, and why. However, I did restore the old function in the latest stable release and recompiled without D2D support. This solved all my issues and seems to work very well in the latest Notepad++.



Naturally, I'd be interested to know if the old behavior could be restored for non-D2D targets, or else to know what exactly is going on underneath the hood here so I can take this up with my screenreader vendor.



Many thanks,

Davy

[Neil Hodgson]

Davy Kager:

I would have expected the opposite: the ::ScrollWindow code produces a duplicate line (or fragmented lines) briefly before it is updated while the complete redraw should always draw all the lines together.

Post by Davy Kager
In particular, the change in ScintillaWin::ScrollText(int) is causing the problem. It was switched to use Redraw() which probably explains my screenreader showing a blank line until it can re-focus on the cursor. I haven't dug deep enough to fully understand what has changed, and why.

::ScrollWindow was a performance optimisation that moves a rectangular block of pixels to scroll. Then the lines scrolled into view are drawn. Like many optimisations, it added complexity and bugs which were slowly (and probably incompletely) fixed over time. Its a layering/modularity violation as it uses windowing calls instead of drawing calls for some pixels which can lead to synchronisation problems between the subsystems particularly on recent platforms where windows are buffered and managed by separate threads generally using the GPU.

This technique is much more difficult on some current platforms such as OS X and WinRT and from a maintenance perspective, I’d like to remove it completely. Unfortunately, GTK+ is still too slow at drawing - it would be worthwhile seeing if using GtkScrolledWiindow helps.

One problem with performing block moves is that pending redraw rectangles become invalid, since the target pixels have moved. When the scroll is caused by caret movement then features like the caret and caret line background need to be removed from their previous lines, and this may cause multiple outstanding redraw rectangles when previous scroll redraws have not yet been finished.

Neil

[Davy Kager]

Post by Neil Hodgson
I would have expected the opposite: the ::ScrollWindow code produces a duplicate line (or fragmented lines) briefly before it is updated while the complete redraw should always draw all the lines together.

So it would be possible that the screenreader software doesn't detect a change in the current line (because it is being duplicated), and then when the redraw is done it picks up the new line, causing no "flickering" to occur in speech and Braille? Then the "flickering" in Scintilla >= 3.0.0 could be caused by:
- The cursor not being visible right after the redraw.
- The redraw taking more time than it should (maybe 200 ms).
I tested on both Windows 7 and Windows 8.1, with the same results. So if it were specific to my system I'd guess it to be the GPU/drivers.
Would D2D rendering make any significance difference here? Also, does my argument that the cursor isn't visible make sense at all? The "flickering" delay always seems to take the same amount of time, which leads me to think that the cursor is always off just after scrolling.

Davy

[Neil Hodgson]

Post by Davy Kager
- The cursor not being visible right after the redraw.

The caret is visible straight after the redraw. Try slowing the caret down to 2 seconds with (in SciTE) caret.period=2000
The caret is immediately visible after movement down that needs a scroll.

Post by Davy Kager
- The redraw taking more time than it should (maybe 200 ms).

I see scrolling at 34ms per line with each individual line scroll visible which is the keyboard repeat rate (30Hz). Try scrolling a large file by holding the down key for a reasonable period (say 30 or 60 seconds) and see how many lines have been scrolled.

Post by Davy Kager
I tested on both Windows 7 and Windows 8.1, with the same results. So if it were specific to my system I'd guess it to be the GPU/drivers.
Would D2D rendering make any significance difference here?

Have you or Notepad++ turned on D2D? A problem here is that we don’t know what the screen reader is doing: maybe it is OCRing the window, maybe it is watching for text drawing calls and maybe it is asking Scintilla for text. If its OCRing then maybe it has to wait for the caret to disappear so that it doesn’t corrupt the text.

Post by Davy Kager
Also, does my argument that the cursor isn't visible make sense at all? The "flickering" delay always seems to take the same amount of time, which leads me to think that the cursor is always off just after scrolling.

The caret is always deliberately *on* just after scrolling since you don’t want it to disappear or blink badly during scrolling.

Neil

[Davy Kager]

Post by Neil Hodgson

Post by Davy Kager
I tested on both Windows 7 and Windows 8.1, with the same results. So if it were specific to my system I'd guess it to be the GPU/drivers.
Would D2D rendering make any significance difference here?

Have you or Notepad++ turned on D2D? A problem here is that we don’t know what the screen reader is doing: maybe it is OCRing the window, maybe it is watching for text drawing calls and maybe it is asking Scintilla for text. If its OCRing then maybe it has to wait for the caret to disappear so that it doesn’t corrupt the text.

I am compiling without D2D support, I'll have to check if Notepad++ turns on D2D if available.
As for the screenreader: it isn't OCR'ing, it gets the text either through Win32 methods or else directly from the graphics driver layer (it adds its own mirror driver to the graphics stack). I think what it does is detect the cursor by looking for something approximating the cursor shape that blinks. I'll also have to check that with the vendor, though. I actually reported this problem to them last year, but they said they couldn't replicate the issue. I thought the problem was with my system, but as I said I tested on two versions of Windows and both had the issue. So, this is to be continued...

Davy

[Davy Kager]

In addition to my previous email, I sometimes briefly see the line *below* the line that I just scrolled to (I'm now testing in SciTE 3.3.7 as well as Notepad++ 6.5.3, both the same results). What happens:
- I'm on the last visible line just above the status bar if there is one (let's say it is line number 55).
- I press the down arrow key.
- I briefly see line 57, then I see line 56, which stays focused.
- Visually, once scrolling is done, line 57 isn't visible (it shouldn't be, since I just scrolled to line 56).
This sometimes happens with a maximized window but will almost always happen with a smaller window size.

Can I make SciTE always use D2D so I can test its behavior versus the GDI mode?

Thanks for all the tips thus far,
Davy

-----Original Message-----
From: scintilla-***@googlegroups.com [mailto:scintilla-***@googlegroups.com] On Behalf Of Davy Kager
Sent: Friday, January 10, 2014 13:08
To: scintilla-***@googlegroups.com
Subject: RE: [scintilla] Scintilla 3.0.0 Broke Scrolling For Some Screenreader Software

Post by Neil Hodgson

Post by Davy Kager
I tested on both Windows 7 and Windows 8.1, with the same results. So if it were specific to my system I'd guess it to be the GPU/drivers.
Would D2D rendering make any significance difference here?

Have you or Notepad++ turned on D2D? A problem here is that we don’t know what the screen reader is doing: maybe it is OCRing the window, maybe it is watching for text drawing calls and maybe it is asking Scintilla for text. If its OCRing then maybe it has to wait for the caret to disappear so that it doesn’t corrupt the text.

I am compiling without D2D support, I'll have to check if Notepad++ turns on D2D if available.
As for the screenreader: it isn't OCR'ing, it gets the text either through Win32 methods or else directly from the graphics driver layer (it adds its own mirror driver to the graphics stack). I think what it does is detect the cursor by looking for something approximating the cursor shape that blinks. I'll also have to check that with the vendor, though. I actually reported this problem to them last year, but they said they couldn't replicate the issue. I thought the problem was with my system, but as I said I tested on two versions of Windows and both had the issue. So, this is to be continued...

Davy
--
You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scintilla-interest+***@googlegroups.com.
To post to this group, send email to scintilla-***@googlegroups.com.
Visit this group at http://groups.google.com/group/scintilla-interest.
For more options, visit https://groups.google.com/groups/opt_out.

[Neil Hodgson]

Post by Davy Kager
As for the screenreader: it isn't OCR'ing, it gets the text either through Win32 methods or else directly from the graphics driver layer (it adds its own mirror driver to the graphics stack). I think what it does is detect the cursor by looking for something approximating the cursor shape that blinks.

With the standard settings in Scintilla using GDI each line is drawn into an offscreen line bitmap and then that bitmap is copied to the screen. For the screen-reader to associate the text with the screen position, it would have to track the bitmaps and the calls that created them and their journey to the screen. It would be easier to do this in unbuffered mode (SciTE: buffered.draw=0) which draws directly to the screen but causes mild flickering.

Post by Davy Kager
- I'm on the last visible line just above the status bar if there is one (let's say it is line number 55).

With normal settings this means that line 56 and (possibly) a portion of 57 are visible. Scintilla tries to retain some context around the caret so it keeps the caret at least a full line away from the bottom when possible.

Post by Davy Kager
- I press the down arrow key.

The rectangle of the new caret position is invalidated with the current scrolling offsets and the system (invisible) caret is moved. No drawing should be performed here since the invalidation is queued.

This on-screen point is checked and its determined that it is outside the preferred area (described as in the ‘unwanted zone’ in the documentation) so a scroll is performed, commonly to move the display down one line. In the current code this updates the top line and scroll bar and invalidates the whole window. The selection is invalidated again but this has no effect as the whole window is already invalidated.

This message is complete so control is returned to Windows which will very soon ask for a repaint due to the invalidation.

The paint occurs with the caret visible as it is in its on phase.

After the blink delay, the caret is switched to its off phase, the system caret is moved, and its area invalidated.

Another paint draws the caret line with an off caret.

There are two carets here: the visible caret drawn by Scintilla and an invisible system caret which was only added for the benefit of screen readers. The system caret is updated in UpdateSystemCaret.

Post by Davy Kager
- I briefly see line 57, then I see line 56, which stays focused.
- Visually, once scrolling is done, line 57 isn't visible (it shouldn't be, since I just scrolled to line 56).

Watch out for partially visible lines since its more important that Scintilla alway show all text than it is to avoid overpaint.

Post by Davy Kager
Can I make SciTE always use D2D so I can test its behavior versus the GDI mode?

technology=1

Post by Davy Kager
Could you suggest some settings to mimic Windows' own notepad.exe as much as is possible (i.e. no fancy scrolling or other things that normally make a programmer's life easier)?

Scintilla looked to other programmers editors like Visual Studio in preference to Notepad - I’ve never examine how Notepad scrolls.

Post by Davy Kager
I'd guess there is some difference, since the screenreader works great in notepad.exe as well as in regular text edit controls (including RichEdit).

The regular text edit controls have explicit accessibility support.

Neil

[Davy Kager]

Post by Neil Hodgson
With normal settings this means that line 56 and (possibly) a portion of 57 are visible. Scintilla tries to retain some context around the caret so it keeps the caret at least a full line away from the bottom when possible.
The rectangle of the new caret position is invalidated with the current scrolling offsets and the system (invisible) caret is moved. No drawing should be performed here since the invalidation is queued.
This on-screen point is checked and its determined that it is outside the preferred area (described as in the ‘unwanted zone’ in the documentation) so a scroll is performed, commonly to move the display down one line. In the current code this updates the top line and scroll bar and invalidates the whole window. The selection is invalidated again but this has no effect as the whole window is already invalidated.
This message is complete so control is returned to Windows which will very soon ask for a repaint due to the invalidation.
The paint occurs with the caret visible as it is in its on phase.
After the blink delay, the caret is switched to its off phase, the system caret is moved, and its area invalidated.
Another paint draws the caret line with an off caret.
There are two carets here: the visible caret drawn by Scintilla and an invisible system caret which was only added for the benefit of screen readers. The system caret is updated in UpdateSystemCaret.

Would it be possible to move the system caret before the blink delay? (Apologies if that's a silly suggestion, I'm still not too familiar with the code.) I'm thinking that the screenreader is tracking the system caret (which, being system-default, is the easiest approach). This also explains why the screenreader moves to the correct line *after* the blink delay has passed. I'm not sure why reverting the ScrollText(int) method fixes that, but this seems the most logical explanation.

The screenreader showing another (incorrect) line during the initial blink delay may well have to do with the fact that it is trying to provide some kind of focus based on tracking the bitmap draw calls and/or looking for a "caret shape" somewhere in the window. This is evidently more prone to errors for this particular screenreader.

I've actually tried another screenreader, the free screenreader available from nvda-project.org, and it seems to track focus just fine. But then, it relies on API calls where my main screenreader, based on code that's been around for 15+ years, is using an off-screen model based on input from the graphics/drawing calls. It's probably (hopefully!) possible for the screenreader devs to "train" Scintilla's caret so that it can be tracked, but a solution that doesn't involve extra work on the screenreader's end would be awesome.

Thanks again for all the help, things make a lot more sense to me now. At the very least I have something substantial to tell the screenreader vendor.

Davy

[Neil Hodgson]

Post by Davy Kager
Would it be possible to move the system caret before the blink delay?

Its possible but it isn’t clear to me. Try adding a call to UpdateSystemCaret in ScrollText and see what happens. A trace (Platform::DebugDisplay) of the relevant calls - CursorUpOrDown, ScrollText, UpdateSystemCaret and Paint with time stamps would help work out how things are sequencing - its hard to work on this sort of thing in a debugger as focus changes (to the debugger and back) affect drawing and the caret.

Post by Davy Kager
... it is trying to provide some kind of focus based on tracking the bitmap draw calls and/or looking for a "caret shape" somewhere in the window.

Looking for a caret shape is more difficult for Scintilla as the caret isn’t drawn in XOR mode and can be coloured. Try caret.fore=#FF0000 (or #FFFFFF for a white caret) in SciTE and see if its still tracked. Also caret.width=3.

Neil

[Davy Kager]

Post by Neil Hodgson

Post by Davy Kager
Would it be possible to move the system caret before the blink delay?

Its possible but it isn’t clear to me. Try adding a call to UpdateSystemCaret in ScrollText and see what happens. A trace (Platform::DebugDisplay) of the relevant calls - CursorUpOrDown, ScrollText, UpdateSystemCaret and Paint with time stamps would help work out how things are sequencing - its hard to work on this sort of thing in a debugger as focus changes (to the debugger and back) affect drawing and the caret.

Spot on! I added a call to UpdateSystemCaret() in the ScrollText(int) function for the win32 branch of the code, just below the call to Redraw() (line 1323 in the 3.3.7 sources). This solved my tracking issues. It appears that the screenreader is ignoring Scintilla's caret for vertical navigation (i.e. no "bouncing" between the two carets occurs, which is how it should be). Long story short: I didn't get any side-effects from adding this extra call, although the after-blink-delay call could possibly be removed if it is added to ScrollText(int) to avoid doing the same thing twice?

To be on the safe side I'll also test horizontal navigation and test navigation with another screenreader. As I mentioned earlier the free screenreader that doesn't rely on an off-screen model never had issues in the first place, so I want to make sure moving UpdateSystemCaret() around doesn't break it. But all in all, my request would be to add this call to ScrollText(int) and possibly remove it from the post-blink code.

Davy

[Neil Hodgson]

Post by Davy Kager
Spot on! I added a call to UpdateSystemCaret() in the ScrollText(int) function for the win32 branch of the code, just below the call to Redraw() (line 1323 in the 3.3.7 sources). This solved my tracking issues.

OK, that seems reasonable.

Post by Davy Kager
although the after-blink-delay call could possibly be removed if it is added to ScrollText(int) to avoid doing the same thing twice?

The after-blink-delay call is in InvalidateCaret that invalidates the rectangle containing the caret so it can blink. Removing the UpdateSystemCaret() here will stop the screen reader seeing most caret moves.

Minimizing changes to the system caret may help avoid repeated updates to screen readers but its difficult to work out all the ‘interesting’ changes. Its likely screen magnifiers redraw on every update which is expensive but they probably want to show each caret blink.

Neil

[Davy Kager]

Post by Neil Hodgson

Post by Davy Kager
Spot on! I added a call to UpdateSystemCaret() in the ScrollText(int) function for the win32 branch of the code, just below the call to Redraw() (line 1323 in the 3.3.7 sources). This solved my tracking issues.

OK, that seems reasonable.

I tested with my own screenreader for a few hours (of real day-to-day use of Notepad++) and encountered no issues with the suggested fix applied. Other screenreaders also don't show any negative effects from this patch. It should be safe for inclusion.

One thing I did wonder about was if UpdateSystemCaret() shouldn't be in the response to the repainting message Windows will send after a Redraw() (assuming Redraw() is asynchronous). That said, having a call to UpdateSystemCaret() in ScrollText(int) has no negative impact on (screenreader) performance. It appears to do what it is meant to do.

Davy

[Neil Hodgson]

Post by Davy Kager
I tested with my own screenreader for a few hours (of real day-to-day use of Notepad++) and encountered no issues with the suggested fix applied. Other screenreaders also don't show any negative effects from this patch. It should be safe for inclusion.

Committed.
http://sourceforge.net/p/scintilla/code/ci/1458b939549f908dc06867d9fc9cbc800c7db05f/

Post by Davy Kager
One thing I did wonder about was if UpdateSystemCaret() shouldn't be in the response to the repainting message Windows will send after a Redraw() (assuming Redraw() is asynchronous).

Not all painting is for the caret. For example, a background spell check or similar may draw indicators without affecting the caret.

Neil

[Davy Kager]

Post by Neil Hodgson

Post by Davy Kager
I tested with my own screenreader for a few hours (of real day-to-day use of Notepad++) and encountered no issues with the suggested fix applied. Other screenreaders also don't show any negative effects from this patch. It should be safe for inclusion.

Committed.

Thanks much for all the help with this issue! I'll contact the Notepad++ folks to include it once the next stable release is out.

Post by Neil Hodgson

Post by Davy Kager
One thing I did wonder about was if UpdateSystemCaret() shouldn't be in the response to the repainting message Windows will send after a Redraw() (assuming Redraw() is asynchronous).

Not all painting is for the caret. For example, a background spell check or similar may draw indicators without affecting the caret.

Ah, I only mentioned it in case the system caret would have issues updating/moving since the actual paint to scroll text hadn't occurred yet, so the system caret would briefly be on text that is not visible yet. Either way, it works great for me, so there shouldn't be a need to complicate things anyway.

Davy

[Davy Kager]

Post by Neil Hodgson

Post by Davy Kager
- The cursor not being visible right after the redraw.

The caret is visible straight after the redraw. Try slowing the caret down to 2 seconds with (in SciTE) caret.period=2000
The caret is immediately visible after movement down that needs a scroll.

Setting caret.period=2000 versus caret.period=25 does not seem to make a difference. What I did notice is that if I scroll onto a blank line, the line below it also becomes visible (according to the screenreader), but when scrolling to a line of text, that line will be the bottom line (again according to the screenreader). This may or may not be relevant to the problem, I'll have to test further. The fact that caret blink rate doesn't affect performance in SciTE makes me think that in Notepad++ the quickly blinking caret may help to put the screenreader back on track, but that the problem itself is caused by something else. I'll try and confirm this with the screenreader devs later.

Davy

[Davy Kager]

Post by Davy Kager

Post by Neil Hodgson

Post by Davy Kager
- The cursor not being visible right after the redraw.

The caret is visible straight after the redraw. Try slowing the caret down to 2 seconds with (in SciTE) caret.period=2000
The caret is immediately visible after movement down that needs a scroll.

Setting caret.period=2000 versus caret.period=25 does not seem to make a difference. What I did notice is that if I scroll onto a blank line, the line below it also becomes visible (according to the screenreader), but when scrolling to a line of text, that line will be the bottom line (again according to the screenreader). This may or may not be relevant to the problem, I'll have to test further. The fact that caret blink rate doesn't affect performance in SciTE makes me think that in Notepad++ the quickly blinking caret may help to put the screenreader back on track, but that the problem itself is caused by something else. I'll try and confirm this with the screenreader devs later.

Bit of a mistake on my end -- apologies for the multiple messages -- I'd forgotten about SciTEGlobal.properties and only set the caret period locally. That said, a global caret period of 2000 actually has two effects:
- It increases the "flickering" time in the screenreader to 2 seconds, indicating the caret *is* responsible for the issue.
- It very clearly shows that the screenreader first puts focus on the line below the one that I'm scrolling to. I'm wondering if this has anything to do with font size and/or screen resolution.

Setting caret.policy.yjumps=1 seems to improve things somewhat, although it is clear that the screenreader has some serious issues tracking the cursor. The odd thing is that it doesn't seem to have these problems when navigating the already-visible text. Could you suggest some settings to mimic Windows' own notepad.exe as much as is possible (i.e. no fancy scrolling or other things that normally make a programmer's life easier)? That is to say, I'm not sure how closely SciTE resembles the classic Notepad behavior by default. I'd guess there is some difference, since the screenreader works great in notepad.exe as well as in regular text edit controls (including RichEdit).

Davy