Multi-phase drawing

[Neil Hodgson]

Currently Scintilla draws line by line with each line completely owing ints strip of pixels. This leads to cutting off extreme ascenders and descenders although this depends on characters used and font. For Western text, its the Scandinavian “Å” that often sees its ring above disappear or have only half appear. Some Asian languages use diacritics for vowels or other reasons and these can stack like “
àŒ€àœ€àœŽâ€ (Tibetan) leading to character portions both above and below the line. Emoji like “🌵" are also often large and this was seen in the Swift experiment. Sometimes, text drawing is not clipped and the result can be ascenders that appear and disappear as the caret moves up and down. Here is an example (on Cocoa with a retina display so double normal size) with some missing ascenders and descenders

[Mike Lischke]

This can be alleviated in Scintilla by setting the extra ascent and descent values but that leads to fewer lines per page and its often not needed: its commonly fine for "Å" to share space with the previous line since descenders will interfere only rarely.

Is this really a good approach to add quite complex drawing just to solve too small line heights? If the user wants to use such characters why not simply requiring him to increase the line height? That would be the solution with the least impact. Assuming that large descenders would only rarely conflict with large ascenders in the next line is only a pious hope. Soon you might have to find another workaround for this.

Mike

[Neil Hodgson]

Post by Mike Lischke
Is this really a good approach to add quite complex drawing just to solve too small line heights? If the user wants to use such characters why not simply requiring him to increase the line height?

While someone who writes in Tibetan should take the time to optimise their environment, the recipients of files should see a reasonably faithful representation of the text. Frequent use of problematic characters may be isolated to some language communities, but many more will see occasional text in these languages as isolated names in HTML, data dumps or similar.

It is unlikely we will see emoji used much for identifiers in source code, but I do expect to start seeing it used in comments.

Post by Mike Lischke
That would be the solution with the least impact. Assuming that large descenders would only rarely conflict with large ascenders in the next line is only a pious hope.

Its producing a better outcome in most cases. Using English language letter frequencies, the 5 lower-case letters with descenders, gjpqy, make up a total of around 6.2% of letters on average. The descenders use a fraction (say 1/4) of the full character width so we’re looking at a 1-2% chance of a descender pixel being used. Source code will be more punctuation heavy so will generally have a lower chance unless the font is using braces that descend below the base line. So, the proposed change should have a clash occur for about 1 in 50 extremely tall characters. There’s about a 1 in 4 chance of a lower-case letter ascending to full height, and all upper-case do, so extremely deep descenders like “
ཀུ” are much more likely to clash.

http://en.wikipedia.org/wiki/Letter_frequency

I was constrained by time when writing the previous mail so the examples aren’t completely fair. Helvetica in the before example is pretty much the worst case you will see with accents on “Ŵ” and “Å” completely missing. Here is the multi-phase drawing using Helvetica showing some overlaps between emoji and the line above.

Loading Image...

Neil

[Neil Hodgson]

The Editor class is currently huge, with Editor.cxx taking up 314K, so I am planning on refactoring it somewhat after the next release. The basic plan is to move drawing, printing and measurement of the text and margin areas into separate classes, EditView and MarginView. A rough version of EditView has been implemented and moves 20% of the code out of Editor. Ideally, the user interaction code would be moved into a controller class but I can't see a simple effective way to do that currently.

EditView needs read-only access to the document as well as to some other Editor state, such as the selection and folding. The portion of Editor needed for EditView is being moved into a new EditModel class. Editor becomes a subclass of EditModel so that all this state can be accessed without a large rewrite.

If anyone is interested in examining or helping with the refactoring, a patch containing the current work is at
http://scintilla.org/EditView.patch

Some of the simplest changes have already been integrated.This is mostly using const for parameters that should not be modified by drawing and measuring: the document, visual styles, selection, etc. This is implemented in the 4 change sets starting with
https://sourceforge.net/p/scintilla/code/ci/16c0dea4601b46e5a2c456a2da9b6dbe5eb6e66a/

The multi-phase drawing code may evolve into allowing additional phases to be added by downstream projects. For example, it may be useful to show variable values inline in a debugger, or colour blocks in a HTML/CSS editor. Its likely that this level of customisation would require adding code inside Scintilla instead of using a published API, at least initially.

Neil

[Neil Hodgson]

The refactored code has been committed and there are now EditModel, MarginView and EditView classes with corresponding files.

Platform maintainers may need to make some changes. The GetClientRectangle and GetVisibleOriginInMain virtual methods have been made const. If your platform-specific subclass implements either of these methods, it too should declare them const so that they continue to be called. There is no compiler warning message to indicate a potential problem.

This change set added the const declarations and shows some adaptation for Cocoa and GTK+:
https://sourceforge.net/p/scintilla/code/ci/05ac19058666d404b33c577fc3395dab1c82d3a6/

Some fields have been moved to the MarginView and EditView classes so platform code may need to prefix those fields with marginView. or view.

The rest of the refactored code was committed in these change sets
https://sourceforge.net/p/scintilla/code/ci/c3515e39e0c94282b5ce4e5f770b0d8b847190c6/
https://sourceforge.net/p/scintilla/code/ci/e81c48157e9091c8904be0c8c07e1215bec7b931/
https://sourceforge.net/p/scintilla/code/ci/79d5565f76ff1de1371a20f98b4654e17fc85726/
https://sourceforge.net/p/scintilla/code/ci/f84b3fbe9464f0d9b92f9e9d6cefd0c66e0bf1bf/

Neil

[Neil Hodgson]

The multi-phase drawing feature has now been committed. There is a new API SCI_SETPHASESDRAW which accepts 3 values, SC_PHASES_ONE(0), SC_PHASES_TWO(1), or SC_PHASES_MULTIPLE(2).

SC_PHASES_ONE is the original Scintilla behaviour where each run of text was drawn along with its background colour independently in its own distinct rectangle, and any text outside this rectangle may be cut off. Truncation was was common when the style included italics or the text was kerned. SC_PHASES_TWO is the two phase drawing scheme introduced to allow text to overlap into the next/previous run which ensured that portions of characters were not cut off horizontally.

SC_PHASES_MULTIPLE extends this to allowing a small amount of overlap vertically so that extreme ascenders and descenders have a good chance of not being cut off. While extreme ascenders and descenders may occur in any language depending on the font chosen, they are common for emoji and for some languages like Tibetan.

SC_PHASES_MULTIPLE may be slower than the other settings due to layout being calculated more times in the painting code. However this should be reduced to almost zero if layout caching is set to SC_CACHE_PAGE or higher.

There is a corresponding SCI_GETPHASESDRAW to retrieve this state. The previous SCI_SETTWOPHASEDRAW API has been superseded by SCI_SETPHASESDRAW although SCI_SETTWOPHASEDRAW still works for now. It may later be formally deprecated.

The emphasis in the API name is on the implementation mechanism and may be better expressed as the result of the setting: either text that may be cut off, may share space horizontally or may share space both horizontally and vertically.

This change perturbed quite a bit of code so there is a chance of new bugs.

SciTE adds a new phases.draw property that calls SCI_SETPHASESDRAW with its value.

Available from the Mercurial repositories:
hg clone http://hg.code.sf.net/p/scintilla/code scintilla
hg clone http://hg.code.sf.net/p/scintilla/scite

Neil

[Colomban Wendling]

Post by Neil Hodgson
The multi-phase drawing feature has now been committed. There is a
new API SCI_SETPHASESDRAW which accepts 3 values, SC_PHASES_ONE(0),
SC_PHASES_TWO(1), or SC_PHASES_MULTIPLE(2).
[...]

Nice, MULTIPLE should also allow to workaround the fractional line
height problems I had with printing, which resulted in lower descents
like "_" to be cropped. Unfortunately it didn't seem to do anything
when applied when printing.

Anyway, I just tested it in SciTE and Geany with GTK 3.12 and I guess
there is a problem: Loading Image...
I didn't try to debug this yet, but tell me if you need me to test stuff.

Regards,
Colomban

[Colomban Wendling]

Post by Colomban Wendling
[...]
Anyway, I just tested it in SciTE and Geany with GTK 3.12 and I guess
there is a problem: http://ban.netlib.re/misc/scite-multidraw.png
I didn't try to debug this yet, but tell me if you need me to test stuff.

Hum, somebody pointed out to me that I actually forgot to mention that
the problem appears with phases.draw=2 (MULTIPLE).

[Neil Hodgson]

Post by Colomban Wendling
Nice, MULTIPLE should also allow to workaround the fractional line
height problems I had with printing, which resulted in lower descents
like "_" to be cropped. Unfortunately it didn't seem to do anything
when applied when printing.

The FormatRange method isn't looping over the page for each phase like the PaintText method does now. That could be changed to loop multiple times over the page, possibly just twice - once for the backgrounds with drawBack | drawIndicatorsBack and then once for the foreground with drawText | drawIndentationGuides | drawIndicatorsFore | drawLineTranslucent.

Post by Colomban Wendling
Anyway, I just tested it in SciTE and Geany with GTK 3.12 and I guess
there is a problem: http://ban.netlib.re/misc/scite-multidraw.png
I didn't try to debug this yet, but tell me if you need me to test stuff.

OK, there is a fix in Hg. The problem is that GTK+ defaults to buffered drawing and buffered drawing works by first drawing into a single line pixmap and then copying the line to the screen. So when setting phases.draw=2, buffered.draw=0 should also be set. Unfortunately without buffering, Scintilla on GTK+ 2 flickers and sometimes paints aren't completed. Just tried GTK+ 3.x and it appears OK now. Really, Scintilla should now be relying on the double buffering provided by GTK+ instead of using its single line buffer.

The fix in Hg treats SC_PHASES_MULTIPLE as SC_PHASES_TWO when buffered drawing is on.

Neil