Using Scintilla from Swift

[Neil Hodgson]

Swift is a new language from Apple that will be available with Xcode 6. Looks to me like its a decent current language if a little unambitious. While I don't think it is worthwhile using Swift in Scintilla, it should be possible to use Scintilla from Swift.

Swift can use Objective C and C code fairly easily but, as the interop documentation says: "You cannot import C++ code directly into Swift. Instead, create an Objective-C or C wrapper for C++ code.". Unfortunately, Scintilla exposes some C++ features in the main ScintillaView.h header which causes difficulty for the bridging importer. Most problematically, Scintilla on Cocoa has always wrapped everything in namespace "Scintilla" and this errors out.

Lesser issues include problems with bare type names - it seems to want typedefs like old style C. InfoBarCommunicator.h can be made compliant with the 'modern' form of enumeration using NS_ENUM which should be OK to change now.

diff ~/merc/scintilla/cocoa/ScintillaView.h ScintillaView.h

26d25
< namespace Scintilla {
40,41c39,40
< class ScintillaCocoa;
< }
---

typedef struct SCNotification SCNotification;
typedef struct ScintillaCocoa ScintillaCocoa;

48c47
< - (void)notification: (Scintilla::SCNotification*)notification;
---

- (void)notification: (SCNotification*)notification;

100c99
< Scintilla::ScintillaCocoa* mBackend;
---

ScintillaCocoa* mBackend;

117c116
< @property (nonatomic, readonly) Scintilla::ScintillaCocoa* backend;
---

@property (nonatomic, readonly) ScintillaCocoa* backend;

131c130
< - (void) notification: (Scintilla::SCNotification*) notification;
---

- (void) notification: (SCNotification*) notification;

182c181
< - (void) registerNotifyCallback: (intptr_t) windowid value: (Scintilla::SciNotifyFunc) callback;
---

- (void) registerNotifyCallback: (intptr_t) windowid value: (SciNotifyFunc) callback;
diff ~/merc/scintilla/cocoa/InfoBarCommunicator.h InfoBarCommunicator.h

24c24
< enum NotificationType {
---

typedef NS_ENUM(NSInteger, NotificationType) {

https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/BuildingCocoaApps/index.html#//apple_ref/doc/uid/TP40014216-CH2-XID_0

https://developer.apple.com/library/ios/releasenotes/ObjectiveC/ModernizationObjC/AdoptingModernObjective-C/AdoptingModernObjective-C.html

Neil

[Mike Lischke]

Post by Neil Hodgson
Swift is a new language from Apple that will be available with Xcode 6. Looks to me like its a decent current language if a little unambitious. While I don't think it is worthwhile using Swift in Scintilla, it should be possible to use Scintilla from Swift.
Swift can use Objective C and C code fairly easily but, as the interop documentation says: "You cannot import C++ code directly into Swift. Instead, create an Objective-C or C wrapper for C++ code.". Unfortunately, Scintilla exposes some C++ features in the main ScintillaView.h header which causes difficulty for the bridging importer. Most problematically, Scintilla on Cocoa has always wrapped everything in namespace "Scintilla" and this errors out.

Hmm, how is that supposed to work with other C++ code? You cannot remove all namespaces just to make it Swift compatible.

ScintillaView doesn't use a namespace (it's Objective-C code after all). ScintillaCocoa does and this is exactly the scenario that I'd expect to be needed for Swift. ScintillaView is the Obj-c wrapper that can be used in Swift, but the wrapper must still be able to use a namespace. Otherwise it will mostly be impossible to wrap C++ code, no?

Post by Neil Hodgson
Lesser issues include problems with bare type names - it seems to want typedefs like old style C. InfoBarCommunicator.h can be made compliant with the 'modern' form of enumeration using NS_ENUM which should be OK to change now.

No problem with that.
If that's only about making the notifcation callback non-namespace'd then fine, but removing the namespace altogether would be a shock.

Mike

[Neil Hodgson]

Post by Mike Lischke
Hmm, how is that supposed to work with other C++ code? You cannot remove all namespaces just to make it Swift compatible.

We only really need namespaces for a couple of types that use the same name as old MacOS types: Point and Style. Style is easy to change (perhaps to FontStyle or VisualStyle) since it is only used internally. Point is more difficult to change as its part of the platform interface so will break all non-core platform interfaces. Having a Scintilla namespace also helps avoid clashes with client code and future platform API additions.

Post by Mike Lischke
ScintillaView doesn't use a namespace (it's Objective-C code after all). ScintillaCocoa does and this is exactly the scenario that I'd expect to be needed for Swift.

"ScintillaCocoa" is already reasonably unique as it starts with the "Scintilla" prefix. If others have a problem with that there could be another preprocessor symbol to decide between namespaced and non-namespaced. ScintillaCocoa is also a class which is C++ and I'm unsure about the rules regarding declaring it as both a struct and class in different places. All definitions could be changed to struct since the first line defines "private:" visibility.

Post by Mike Lischke
ScintillaView is the Obj-c wrapper that can be used in Swift, but the wrapper must still be able to use a namespace. Otherwise it will mostly be impossible to wrap C++ code, no?

The wrapper implementation files do need to use namespaces. However, the header ScintillaView.h may not.

There are a multiple approaches. One is to hide the definitions in the header and use void* parameters which are cast to the right types in the implementation. Another is to move a few things out of namespaces when they are sufficiently unique.

The only one changed in the above diffs, apart from ScintillaCocoa is SCNotification which is unlikely to clash without a namespace. It needs Sci_NotifyHeader which has a Sci_ prefix and so matches the Objective C convention of using prefixes for uniqueness. Other client code may need other Sci_* structs and they could reasonably be allowed out of the namespace.

Post by Mike Lischke
If that's only about making the notifcation callback non-namespace'd then fine, but removing the namespace altogether would be a shock.

The diffs were just rough work to see what could be done. My current preference is to make ScintillaCocoa, SCNotification, and the Sci_* structs available without a namespace even when SCI_NAMESPACE is set. I'm unsure about whether to do this unconditionally or to choose with another preprocessor symbol (SCI_EXPOSE_PREFIXED_NAMES?).

To avoid disrupting existing client code that may be using explicit namespace qualifiers, the identifiers could also be made made available inside the Scintilla namespace although I haven't researched the syntax for this yet. So Objective C and C++ client code could switch to SCNotification or continue using Scintilla::SCNotification.

After some ugly hacking this Swift code is now running:

import Cocoa
import Scintilla

class AppDelegate: NSObject, NSApplicationDelegate {
@IBOutlet var window: NSWindow
func applicationDidFinishLaunching(aNotification: NSNotification?) {
let x = Scintilla.SCI_SETSELEOLFILLED;
let y = ScintillaView();
}
func applicationWillTerminate(aNotification: NSNotification?) {
}
}

Neil

[Neil Hodgson]

Xcode 6 is still pre-release and the tools used for combining ObjectiveC/C++ with Swift may change and are currently difficult to use. I'll hold off committing most changes until Xcode 6 is released. The use of NS_ENUM inside InfoBarCommunicator.h looks to be a good change anyway so will be committed soon.

There are a couple of mechanisms that can be used to expose the Objective C/C++ definitions to Swift. A bridging header was first tried but this had some problems. Then a module map was used and this worked well although I don't really understand modules. The current module.modulemap for Scintilla is:

framework module Scintilla {
umbrella header "ScintillaView.h"
export *
module * { export * }
}

This works since ScintillaView.h includes the Scintilla.h, SciLexer.h, and InfoBarCommunicator.h so exports everything that might be wanted by client code. It also does not require a "Scintilla." prefix before every feature as other variants did but I don't understand why. Here is the main file AppDelegate.swift for a working Swift application that looks like Loading Image...

// AppDelegate.swift in Swiftee
// Copyright 2014 Neil Hodgson.
// This code is public domain.

import Cocoa
import Scintilla

class AppDelegate: NSObject, NSApplicationDelegate {

@IBOutlet var window: NSWindow
var sci: ScintillaView?

// sciMsg is just to avoid a lot of casts since Swift is picky and
// a CInt can't be passed as a CLong without a cast.
func sciMsg(m:CInt, wp:CInt=0, lp:CInt=0) {
sci?.message(CUnsignedInt(m),wParam: CUnsignedLong(wp),lParam:CLong(lp))
}

func applicationDidFinishLaunching(aNotification: NSNotification?) {
sci = ScintillaView(frame: window.contentView.bounds)
if let ssci = sci {
ssci.autoresizingMask = NSAutoresizingMaskOptions.ViewWidthSizable |
NSAutoresizingMaskOptions.ViewHeightSizable
}
window.contentView.addSubview(sci)
sciMsg(SCI_STYLECLEARALL)
sciMsg(SCI_SETLEXER, wp:SCLEX_CPP)
sciMsg(SCI_STYLESETBOLD, wp:SCE_C_OPERATOR, lp:1)
sciMsg(SCI_STYLESETFORE, wp:SCE_C_COMMENTLINE, lp:0x008000)
sciMsg(SCI_STYLESETFORE, wp:SCE_C_WORD, lp:0x800000)
sciMsg(SCI_STYLESETFORE, wp:SCE_C_WORD2, lp:0xA00060)
sciMsg(SCI_STYLESETFORE, wp:SCE_C_STRING, lp:0x800080)
sciMsg(SCI_STYLESETITALIC, wp:SCE_C_STRING, lp:1)
sci?.setStringProperty(SCI_SETKEYWORDS, parameter: 0,
value: "class func import let return var @IBOutlet")
sci?.setStringProperty(SCI_SETKEYWORDS, parameter: 1,
value: "CInt CLong CUnsignedInt CUnsignedLong")
sci?.insertText("// Multiply by 5\nfunc mult5(a: CInt) -> CInt {\n\treturn a * 5;\n}\n")
}

func applicationWillTerminate(aNotification: NSNotification?) {
// Insert code here to tear down your application
}
}

The module map shown above should also work for Objective C client code but I was unable to make it build. Modules fix the use of textual inclusion in C / Objective C / C++ as they allow the compiler to parse each header once to produce a module then reuse the parsed form whenever the module is imported. This should speed up compilation as each header line is often parsed many times: for Scintilla+SciTE on Windows, according to Header Hero, the compiler parses 37 million header lines so each line is parsed around 91 times in a build.

http://clang.llvm.org/docs/Modules.html

Neil

[Neil Hodgson]

Post by Neil Hodgson
The use of NS_ENUM inside InfoBarCommunicator.h looks to be a good change anyway so will be committed soon.

This is committed as
https://sourceforge.net/p/scintilla/code/ci/0c2dd7388665aef6630f331fbfebf2c6ba67c822/
The IBDisplay enumeration should also use a platform macro which would be NS_OPTIONS since it is a set of bits. Haven't done this yet as IBDisplay wasn't used in ScintillaView.h as it doesn't import InfoBar.h.

Some other changes were made to ease later enhancements.

ScintillaCocoa no longer accesses ScintillaView.backend. Its not needed and it meant that ScintillaView.backend had to be defined in the header. I'd like to hide ScintillaView.backend in a future revision as client code should not be accessing the ScintillaCocoa C++ class. The hiding could be performed by moving its declaration to the ScintillaView @implementation but, since that breaks on old versions of OS X and 32-bit builds, mBackend can be changed to void* and the backend property be exposed with a class extension inside the ScintillaView.mm file:

@interface ScintillaView ()
- (struct ScintillaCocoa*) backend;
@end

This also means that direct access to mBackend inside ScintillaView changes to self.backend as self.backend is of the correct (ScintillaCocoa*) type.

Change set to avoid ScintillaView.backend in ScintillaCocoa:
https://sourceforge.net/p/scintilla/code/ci/059d0bbd6314f562b744c64fbf250c202489c5fc/

8 months ago, the delegate mechanism in ScintillaView was enhanced to handle notifications in a way more consistent with Cocoa. At the time, a comment was added saying the previous method registerNotifyCallback would be deprecated. It is now marked with the deprecated attribute so client code that uses it will produce a warning. A future release will remove the method.

I'd like to move other internal features out of the ScintillaView.h header. The definition of the SCIMarginView class does not need to be published so should be moved into ScintillaView.mm, leaving behind an @class stub so ScintillaView can still define a field of this type. The methods positionSubViews, setMarginWidth, scrollerAction, and updateMarginCursors should not be used by clients so should be hidden inside ScintillaView.mm or, for setMarginWidth which is used only from ScintillaCocoa, in a category on ScintillaView only visible to ScintillaView and ScintillaCocoa.

The ScintillaTest program was able to use a Scintilla module. This required a different module map to Swift - there are problems with defining where the needed headers are. Hopefully the module feature will be more solid with the release Xcode 6.

Neil

[Neil Hodgson]

Post by Neil Hodgson

Post by Neil Hodgson
The use of NS_ENUM inside InfoBarCommunicator.h looks to be a good change anyway so will be committed soon.

This is committed as
https://sourceforge.net/p/scintilla/code/ci/0c2dd7388665aef6630f331fbfebf2c6ba67c822/

This won't build in Xcode 4.2 which is the most recent release available on OS X 10.6.8 Snow Leopard. It does work in Xcode 4.6.3 on OS X 10.7.5.

It builds for OS X 10.6 from 10.7 or above so I don't know if this will be a problem for anyone. OS X developers generally seem to be on the current release or one behind although users are often stuck on earlier releases.

Neil

[Neil Hodgson]

Some more clean-up changes committed for Cocoa. The changes are the safest parts of interoperating with Swift and allowing use of modules in Objective C. Modules are really only supported in the beta version of Xcode so no module map is included at this time. Client code should not have problems unless it is using internal Scintilla headers.

NS_OPTIONS used for IBDisplay enumeration similar to earlier change to NotificationType.
https://sourceforge.net/p/scintilla/code/ci/369eb056f5849739fdb46499f2c2ca2960efd904/

SCI_NAMESPACE is only active in Scintilla.h when the source is being compiled as C++. A namespace directive causes compile failures for C, Objective C, and Swift. This helps when a C/Objective C/Swift source file is being compiled along with C++/Objective C++ and SCI_NAMESPACE is set in Xcode globally.
https://sourceforge.net/p/scintilla/code/ci/b7bf992bbf847b7d75b34c21347cf4709e7a5eb0/

Frameworks are supposed to publish their API in their Headers sub-directory. Scintilla previously included some private headers in this directory and these files are no longer copied there. This makes it easier for clients to see which APIs should be used and can allow some tools to automatically include the full set of public headers (Scintilla.h, SciLexer.h, InfoBarCommunicator.h, InfoBar.h, ScintillaView.h).
https://sourceforge.net/p/scintilla/code/ci/d1dc91faddf1599f6f6f82ab717e6e72c9b320a9/

ScintillaTest now uses the headers from the Scintilla framework instead of using relative paths. This is to check that the framework is exposing the necessary headers.
https://sourceforge.net/p/scintilla/code/ci/9592277093cde207d19e62c60147cb647c9d20d9/

Some private methods on ScintillaView are no longer exposed in the header and an unused method was removed from ScintillaCocoa.

I currently have two different potential module map files. The first uses ScintillaView.h as an 'umbrella header' (header that holds the whole API) for the framework but does not expose InfoBar since it is not included in ScintillaView. The second lists every header explicitly. BTW, trying to write module maps is frustrating as the documentation isn't great, the warning messages are opaque, and their compiled form is cached somewhere strange and you have to manually clear the cache after some changes.

http://clang.llvm.org/docs/Modules.html

framework module Scintilla {
umbrella header "ScintillaView.h"

export *
module * { export * }
}

------------------------------------------------------------

framework module Scintilla {
header "Scintilla.h"
header "SciLexer.h"
header "InfoBarCommunicator.h"
header "InfoBar.h"
header "ScintillaView.h"

export *
}


Neil

[Mike Lischke]

Hey Neil,

Post by Neil Hodgson
I currently have two different potential module map files. The first uses ScintillaView.h as an 'umbrella header' (header that holds the whole API) for the framework but does not expose InfoBar since it is not included in ScintillaView. The second lists every header explicitly.

Can't you use a mixed mode? The docs say:

"Any headers not included by the umbrella header should have explicit header declarations. Use the -Wincomplete-umbrella warning option to ask Clang to complain about headers not covered by the umbrella header or the module map."

So, you could use the umbrella header and add explicitely only those headers that are not included by the umbrella header.

Post by Neil Hodgson
BTW, trying to write module maps is frustrating as the documentation isn't great, the warning messages are opaque, and their compiled form is cached somewhere strange and you have to manually clear the cache after some changes.
http://clang.llvm.org/docs/Modules.html

Tbh, I never heard of that before. I guess it's used for Objective-C as it uses #import normally. But the module maps are not exposed, at least I have never seen one.

Mike

[Neil Hodgson]

Post by Mike Lischke
"Any headers not included by the umbrella header should have explicit header declarations. Use the -Wincomplete-umbrella warning option to ask Clang to complain about headers not covered by the umbrella header or the module map."
So, you could use the umbrella header and add explicitely only those headers that are not included by the umbrella header.

I had tried that and received various error messages but tried again with this one and it worked.

framework module Scintilla {
umbrella header "ScintillaView.h"
module InfoBar {
header "InfoBar.h"
}
export *
module * { export * }
}

Post by Mike Lischke
Tbh, I never heard of that before. I guess it's used for Objective-C as it uses #import normally. But the module maps are not exposed, at least I have never seen one.

Yes, the use of modules has been a mostly hidden optimisation for system headers for the past year.
http://stoneofarc.wordpress.com/2013/06/25/introduction-to-objective-c-modules/

Modules are also likely to appear in C++17. Here's the current proposal
http://isocpp.org/files/papers/N4047.pdf

Neil

[Philippe Lhoste]

OK, next step is to convert Emoji Unicode symbols to corresponding color icons*, as seen
in many screenshots** about Swift... :-D

Or not. Beside the first wow effect, it is a very useless feature! ;-)

* http://apps.timwhitlock.info/emoji/tables/unicode
**
https://twitter.com/futurepaul/status/473902211463118848
https://plus.google.com/106534048061626372350/posts/U3bontk4bkg
--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[Neil Hodgson]

OK, next step is to convert Emoji Unicode symbols to corresponding color icons*, as seen in many screenshots** about Swift... :-D

The colour icons come for free on Cocoa. There was a bug in Scintilla that caused the width of the emoji to be incorrect but that is now fixed with
https://sourceforge.net/p/scintilla/code/ci/ea62af90133c2a59fb47dc45bb146b9c9a1a6606/

The emoji are not really in the font Scintilla asks for so are taller than expected. A little extra line spacing was added here so they wouldn't be cut off:
Loading Image...

Neil

[Philippe Lhoste]

Post by Neil Hodgson
The colour icons come for free on Cocoa. There was a bug in Scintilla that caused the
width of the emoji to be incorrect but that is now fixed with
https://sourceforge.net/p/scintilla/code/ci/ea62af90133c2a59fb47dc45bb146b9c9a1a6606/
The emoji are not really in the font Scintilla asks for so are taller than expected. A
http://scintilla.org/SwiftEmoji.png

Are these icons or fonts? I have never seen fonts with various colors, so I suppose it is
a kind of substitution of glyphs with icons.

When I wrote "useless", I meant "in code", such icons are more legitimate in comments,
although a bit noisy, IMO. Well, I could use them when I add a comment about an apparently
useless line of code added only to please FindBugs... :-)
--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[Neil Hodgson]

Are these icons or fonts? I have never seen fonts with various colors, so I suppose it is a kind of substitution of glyphs with icons.

Its a font containing images. Everyone is using different techniques for these fonts.
http://opentype.info/blog/2013/07/03/color-emoji-in-windows-8-1-the-future-of-color-fonts/

Neil

[Philippe Lhoste]

Post by Neil Hodgson

Are these icons or fonts? I have never seen fonts with various colors, so I suppose it is a kind of substitution of glyphs with icons.

Its a font containing images. Everyone is using different techniques for these fonts.
http://opentype.info/blog/2013/07/03/color-emoji-in-windows-8-1-the-future-of-color-fonts/

Thanks for the very interesting link. Indeed, it is a mess. And Apple is true to its
reputation of preferring proprietary, closed solutions. Even Microsoft became more open
than them...

I appreciate the solution of Microsoft as it is fully scalable, but the PNG solutions are
richer (gradients...).
--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --