C++ lexer handling of macros

[Neil Hodgson]

Feature request #1051 http://sourceforge.net/p/scintilla/feature-requests/1051/ was to allow
#if defined SYMBOL
to be treated the same as
#if defined(SYMBOL)

I didn't like the implementation so made a more ambitious change that redid much of the preprocessor evaluation code so it can now handle macros with arguments. It also implements the variant of defined from the feature request.

The main desire for arguments to macros is for version checking macros such as GTK_CHECK_VERSION used in GTK+ where it may be used to make code work on different versions of the GTK+ API. An example from SciTE:

#if GTK_CHECK_VERSION(2,20,0)
#define IS_WIDGET_FOCUSSED(w) (gtk_widget_has_focus(GTK_WIDGET(w)))
#else
#define IS_WIDGET_FOCUSSED(w) (GTK_WIDGET_HAS_FOCUS(w))
#endif

This macro and the related version numbers are defined in gtkversion.h like so (simplified):

#define GTK_MAJOR_VERSION (2)
#define GTK_MINOR_VERSION (0)
#define GTK_MICRO_VERSION (6)
/* check whether a Gtk+ version equal to or greater than
* major.minor.micro is present.
*/
#define GTK_CHECK_VERSION(major,minor,micro) \
(GTK_MAJOR_VERSION > (major) || \
(GTK_MAJOR_VERSION == (major) && GTK_MINOR_VERSION > (minor)) || \
(GTK_MAJOR_VERSION == (major) && GTK_MINOR_VERSION == (minor) && \
GTK_MICRO_VERSION >= (micro)))

The lexer is unable to pick up multi-line definitions but, since this is defined in a header, it needs to be defined to the lexer in keyword list 4 (keywords5 from SciTE's point of view). In this format:

keywords5.$(file.patterns.cpp)=\
GTK_MAJOR_VERSION=2 \
GTK_MINOR_VERSION=0 \
GTK_MICRO_VERSION=6 \
GTK_CHECK_VERSION(major,minor,micro)=(GTK_MAJOR_VERSION>(major)||(GTK_MAJOR_VERSION==(major)&&GTK_MINOR_VERSION>(minor))||(GTK_MAJOR_VERSION==(major)&&GTK_MINOR_VERSION==(minor)&&GTK_MICRO_VERSION>=(micro))) \

The line defining GTK_CHECK_VERSION will be wrapped in this mail but shouldn't be in the .properties file. Further, since macros are separated by spaces in the keyword list, all the spaces must be removed from the macro.

An IDE may want to control the version numbers itself instead of using settings files. Then it could implement a control, such as a menu, for choosing the API level that will be used.

Instead of reformatting important macros like these, it may be better to allow setting a header document as a string property to the C++ lexer. The lexer would then lex the header document, building a symbol table of macros, before lexing the user's document.

This was a large change so there is a good chance of bugs. Nested macros or other complex arguments are unlikely to work.

Available from the Mercurial repository:
hg clone http://hg.code.sf.net/p/scintilla/scite
and from
http://www.scintilla.org/scite.zip Source
http://www.scintilla.org/wscite.zip Windows executable

Change set:
http://sourceforge.net/p/scintilla/code/ci/efde54e0aa3fa529e3fdc96cb247f63e0c8a3e62/

Neil