how does scintilla trigger syntax highlighting on less than the whole text?

[Cosmin Apreutesei]

Hi,

I'm trying to integrate the scintilua lexers into a text editor of my own
for the purpose of syntax highlighting. I have found that lexing the whole
text on each key stroke is too slow. But scintilla doesn't have this
problem so I assume that scintilla calls the lexer only on parts of the
text and not on the entire text every time. What is the logic by which
scintilla decides which part of the text needs re-lexing when the text
changes? Any hints/pointers on where to look in the code appreciated.

Regards,
Cosmin.

[Colomban Wendling]

Post by Cosmin Apreutesei
Hi,

Hi,

Post by Cosmin Apreutesei
I'm trying to integrate the scintilua lexers into a text editor of my
own for the purpose of syntax highlighting. I have found that lexing the
whole text on each key stroke is too slow. But scintilla doesn't have
this problem so I assume that scintilla calls the lexer only on parts of
the text and not on the entire text every time. What is the logic by
which scintilla decides which part of the text needs re-lexing when the
text changes? Any hints/pointers on where to look in the code appreciated.

I'm not a Scintilla developer, but AFAIK the logic is quite simple:

1) only the text after the modification position may be affected by it,
so only it needs re-lexing;

2) only the part actually visible on the view needs highlighting.

so basically, when the buffer is modified only invalidate the style for
everything *after* the modification position, and only actually trigger
highlighting until the last visible position on the view (or rather I
think Scinitlla actually does until the end of the last visible line,
but that's not much difference). When the view is scrolled and more
line appears, trigger highlighting for those.

Regards,
Colomban

[Gur Stavi]

Post by Colomban Wendling
1) only the text after the modification position may be affected by it,
so only it needs re-lexing;

This is not entirely correct an edit may change the highlighting before it
but it is up to the specific lexer to go back and not the editor. The
editor only need to make sure that the entire document is accessible and
not only the visible part.

Another issue is that lexing should be sequential. If you change the 1st
char in a document all lexing become invalid and you should only lex the
visible page. But if you now jump to the end of the document you should not
just lex the last visible page, you should re-lex the entire document
sequentially until the last page. Lexers need to trust the editor to do
that or they will have hard time on each lex request determining how much
they need to go back to find an 'idle' position to start from.

Gur

[Cosmin Apreutesei]

Thanks for the help, things are clear now.