How to configure devel environment to work on Gecko for SailfishOS    Posted:


It turned out that newcomers have difficulties with setting up their development environment to work on Gecko integration with Nemo, SailfishOS or any other Mer-based Linux. Hence this post.

Configure development environment

Gecko engine development for embedded devices is quite resource consuming. If you want to try it you'd better have a mighty computer running Linux and avoid virtualization. It's better not to use the standard SailfishOS SDK, but to resort to Mer chroot-based SDK.

First of all you'll need to install Mer platform SDK and enter into it. Look at this wiki page for details and follow the instructions till the part "Basic tasks".

Then you'll need to create and install a rootfs. More detailed instructions can be found here, but below is an extarct from there:

MerSDK$ sudo mkdir -p /parentroot/srv/mer/targets
MerSDK$ cd /tmp

Now in the current directory (which is /tmp) create a file mer-target-armv7hl.ks with the following content:

# -*-mic2-options-*- --arch=armv7hl -*-mic2-options-*-

lang en_US.UTF-8
keyboard us
timezone --utc UTC
part / --size 500 --ondisk sda --fstype=ext4
rootpw rootme

user --name mer  --groups audio,video --password rootme

repo --name=sailfish --baseurl=http://releases.sailfishos.org/sdk/latest/jolla/armv7hl --save --debuginfo

%packages
glibc-devel
mesa-llvmpipe-libEGL-devel
mesa-llvmpipe-libGLESv2-devel
shadow-utils
rpm-build
meego-rpm-config
zypper
%end

%post
## rpm-rebuilddb.post from mer-kickstarter-configs package
# Rebuild db using target's rpm
echo -n "Rebuilding db using target rpm.."
rm -f /var/lib/rpm/__db*
rpm --rebuilddb
echo "done"
## end rpm-rebuilddb.post

## arch-armv7hl.post from mer-kickstarter-configs package
# Without this line the rpm don't get the architecture right.
echo -n 'armv7hl-meego-linux' > /etc/rpm/platform

# Also libzypp has problems in autodetecting the architecture so we force tha as well.
# https://bugs.meego.com/show_bug.cgi?id=11484
echo 'arch = armv7hl' >> /etc/zypp/zypp.conf
## end arch-armv7hl.post

%end

Then create a target:

MerSDK$ sudo mic create fs mer-target-armv7hl.ks -o /parentroot/srv/mer/targets --pkgmgr=zypp --arch=armv7hl --tokenmap=MER_RELEASE:latest
MerSDK$ sudo chown -R $USER /parentroot/srv/mer/targets/mer-target-armv7hl/*

Now tweak the target to recognize you:

MerSDK$ cd /parentroot/srv/mer/targets/mer-target-armv7hl/
MerSDK$ grep :$(id -u): /etc/passwd >> etc/passwd
MerSDK$ grep :$(id -g): /etc/group >> etc/group

Configure it to work with Scratchbox2 together:

MerSDK$ cd /parentroot/srv/mer/targets/mer-target-armv7hl/
MerSDK$ sb2-init -d -L "--sysroot=/" -C "--sysroot=/" -c /usr/bin/qemu-arm-dynamic -m sdk-build -n -N -t / mer-target-armv7hl /opt/cross/bin/armv7hl-meego-linux-gnueabi-gcc
MerSDK$ sb2 -t mer-target-armv7hl -m sdk-install -R zypper ref --force

Clone web engine git repos to your local file system (e.g. into the local directory /home/rozhkov/tmp/mozilla-temp/):

$ mkdir -p /home/rozhkov/tmp/mozilla-temp
$ cd /home/rozhkov/tmp/mozilla-temp
$ git clone git@github.com:tmeshkova/xulrunner-package.git
$ cd xulrunner-package
$ ./pull.all.sh

After that install build requirements to the rootfs:

MerSDK$ cd /home/rozhkov/tmp/mozilla-temp/xulrunner-package
MerSDK$ grep --color=never BuildRequires mozilla-central/rpm/xulrunner-qt5.spec | sed -e '/^#.*$/d' | gawk -F: '{ print $2 }' | tr ',' ' '| xargs sb2 -t mer-target-armv7hl -m sdk-install -R zypper in
MerSDK$ grep --color=never BuildRequires qtmozembed/rpm/qtmozembed-qt5.spec | sed -e '/^#.*$/d' | gawk -F: '{ print $2 }' | tr ',' ' '|xargs sb2 -t mer-target-armv7hl -m sdk-install -R zypper in
MerSDK$ grep --color=never BuildRequires embedlite-components/rpm/embedlite-components-qt5.spec | sed -e '/^#.*$/d' | gawk -F: '{ print $2 }' | tr ',' ' '|xargs sb2 -t mer-target-armv7hl -m sdk-install -R zypper in
MerSDK$ grep --color=never BuildRequires sailfish-browser/rpm/sailfish-browser.spec | sed -e '/^#.*$/d' | gawk -F: '{ print $2 }' | tr ',' ' '|xargs sb2 -t mer-target-armv7hl -m sdk-install -R zypper in

Remove the packages that are not really needed for engine development:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-install -R zypper rm qtmozembed-qt5 qtmozembed-qt5-devel xulrunner-qt5 xulrunner-qt5-devel

And finally build the stuff with the build.sh script:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build ./build.sh -j -t mer

The option -j here makes build.sh build sailfish-browser binaries as well.

When everything is successfully built the script will show instructions on how to run the newly built browser:

...
make[1]: Leaving directory `/home/rozhkov/tmp/mozilla-temp/xulrunner-package/sailfish-browser/src'
make: Leaving directory `/home/rozhkov/tmp/mozilla-temp/xulrunner-package/sailfish-browser'

prepare run-time environment:
export LD_LIBRARY_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/src
export QML_IMPORT_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/qmlplugin5
export QML2_IMPORT_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/qmlplugin5

run unit-tests:
export QTTESTSROOT=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/tests
export QTTESTSLOCATION=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/tests/auto/mer-qt5
export QTMOZEMBEDOBJDIR=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer
/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/tests/auto/run-tests.sh

run test example:
/home/rozhkov/tmp/mozilla-temp/xulrunner-package/objdir-mer/dist/bin/qmlMozEmbedTestQt5  -fullscreen  -url about:license
/home/rozhkov/tmp/mozilla-temp/xulrunner-package/objdir-mer/dist/bin/sailfish-browser about:license

Note

Due to a bug in gecko build scripts you might encounter an error message about missing config.status file after the build configuration phase. In this case just copy the file objdir-mer/config.status to your working directory and run build.sh again:

MerSDK$ cp objdir-mer/config.status .
MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build ./build.sh -j -t mer

The best way to test the build is to mount the working directory into the device's file system so that the path to the built binaries on the device is the same as in the host filesystem. For this you'll need to have the package sshfs installed on the device:

[nemo@localhost-001 ~]$ mkdir tmp
[nemo@localhost-001 ~]$ sshfs rozhkov@192.168.2.14:/home/rozhkov/tmp tmp
[nemo@localhost-001 ~]$ devel-su
[root@localhost-001 nemo]$ cd /home
[root@localhost-001 home]$ ln -s nemo rozhkov
[root@localhost-001 home]$ exit
[nemo@localhost-001 ~]$ cd tmp/mozilla-temp/xulrunner-package
[nemo@localhost-001 xulrunner-package]$ export LD_LIBRARY_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/src
[nemo@localhost-001 xulrunner-package]$ export QML_IMPORT_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/qmlplugin5
[nemo@localhost-001 xulrunner-package]$ export QML2_IMPORT_PATH=/home/rozhkov/tmp/mozilla-temp/xulrunner-package/qtmozembed/objdir-mer/qmlplugin5
[nemo@localhost-001 xulrunner-package]$ /home/rozhkov/tmp/mozilla-temp/xulrunner-package/objdir-mer/dist/bin/sailfish-browser about:license

By now you should have working development environment. If you change code under mozilla-central/embedding/embedlite, qtmozembed, embedlite-components or sailfish-browser just run the build.sh script again:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build ./build.sh -j -t mer

If you change something inside other gecko components, e.g. under mozilla-central/dom/events or mozilla-central/gfx, then you'll need to rebuild the outdated object files too with the option -o:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build ./build.sh -j -t mer -o dom/events,gfx

This way there is no need to rebuild all other object files. And if you're working on the engine only then you might want to use the option -e of build.sh that makes the script to rebuild only the engine:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build ./build.sh -e -t mer

Useful info

If you're working on JS components don't forget to reset the start up cache before testing your work:

[nemo@localhost-001 xulrunner-package]$ rm -fr ~/.mozilla/mozembed/startupCache/

If you need to switch on logging in the engine then define NSPR_LOG_MODULES environment variable:

[nemo@localhost-001 xulrunner-package]$ export NSPR_LOG_MODULES=TabChildHelper:5,EmbedLiteTrace:5,EmbedContentController:5

Warning

In order to see logging from components other than EmbedLite you'd need to have a so called debug build of xulrunner. Use the option -d of the build.sh script to create it.

Unfortunately the gecko build system is based on python which runs under qemu inside Scratchbox2 by default (unless you do x86 build). It is possible to accelerate python though. To achieve this you need to use a special Scratchbox2 mode sdk-build+pp and to add the option -r to build.sh:

MerSDK$ sb2 -t mer-target-armv7hl -m sdk-build+pp ./build.sh -r -j -t mer

Happy hacking!

Comments

EmbedLite Initialization    Posted:


I guess some people may find useful the couple of words below on how EmbedLite embedding is initialized. I've written them mostly because I'm trying to wrap my head around the topic myself. So please let me know if you see non-sense.

Initialization procedure

First of all a toolkit specific embedding (e.g. qtmozembed) must pre-configure embedlite with the function LoadEmbedLite(). Then we should instantiate EmbedLiteApp class. This is done with the function XRE_GetEmbedLite(). Only one instance of EmbedLiteApp class can be created. Let's call this singleton "embedLiteApp".

Next step is to set up callbacks to the embedding into "embedLiteApp". Currently the callbacks should be implemented as a class inheriting to EmbedLiteAppListener. The callbacks are called by "embedLiteApp" in order either to notify our embedding about application-wide events or to configure native threads. Only one instance of this callback collection makes sense.

Then the embedding (qtmozembed) should register all needed manifests of XPCOM components with EmbedLiteApp::AddComponentManifest().

At this stage a separate thread with embedlite can be started with either EmbedLiteApp::StartWithCustomPump() or EmbedLiteApp::Start() methods. The former method does an asynchronous call and returns immediately, the latter one returns only after embedlite has stopped. After that the web engine starts its initialization procedures.

Internally "embedLiteApp" schedules a call to its EmbedLiteApp::StartChild() which is supposed to create a thread for embedlite either itself or with help of a toolkit specific embedding (through the EmbedLiteAppListener::ExecuteChildThread() callback) and inside the thread it calls EmbedLiteApp::StartChildThread(). This function loads manifests of XPCOM components, loads libxul.so, initializes the gecko webengine, creates a new message loop for the thread and schedules creation of "App Thread" actors or communication end points in other words. One end point (an instance of EmbeLiteAppThreadParent) is a parent actor used to deliver messages from the just created thread to the parent thread where "embedLiteApp" lives. The other end point (an instance of EmbedLiteAppThreadChild) is a child actor used to communicate with objects living in the child thread. At the moment of creation the EmbedLiteAppThreadChild instance opens a communication channel to EmbedLiteAppThreadParent. The communication protocol is defined in the file PEmbedLiteApp.ipdl. Direct method calls from one thread to another must be avoided since the objects can be placed into different processes actually, not threads.

Basically the instance of EmbedLiteApp represents a chrome (or main UI) thread. It communicates with a toolkit specific embedding through installed callbacks (see EmbedLiteAppListner) and with Gecko webengine through the actor EmbedLiteAppThreadParent.

Also the initialization procedures includes creation of EmbedLiteAppService implementing the interface nsIEmbedAppService. This service keeps track of created web views and is used by XPCOM components to communicate with the veiws.

After the implementation of nsIEmbedAppService is up and running the web engine is considered to be fully functional. This event gets propagated to the main UI thread (the parent actor receives the async Initialized() message). Also the "embedliteInitialized" message is broadcasted with nsIObserverService. From now on we can create actual web views.

WebView creation

For every native widget representing a web view there should exist a corresponding instance of EmbedLiteView class. This class exposes public API for web views to the toolkit specific embedding (i.e. qtmozembed). Just like in the case of EmbedLiteApp the embedding is supposed to register its callbacks to the instance of EmbedLiteView. The callbacks are organized as virtual methods of a class inheriting to EmbedLiteViewListener. A native widget is supposed to instantiate the class and to register it with EmbedLiteView::SetListener().

The instance of EmbedLiteView can be created by a native widget with the method EmbedLiteApp::CreateView(). Internally in the method "embedliteApp" generates an unique ID, then instantiates EmbedLiteView identified by the ID and puts it into a local hash of views. After that it sends a message async CreateView(viewId, parentViewId) to the gecko thread/process and returns the just created EmbedLiteView instance to the native widget. The native widget installs its callbacks and that's it. At this moment the EmbedLiteView instance still cannot be used to communicate with the actual web view because it doesn't exist yet.

When the gecko thread receives the message async CreateView() (via EmbedLiteAppThreadChild::RecvCreateView()) it creates a pair of subprotocol actors EmbedLiteViewThreadParent and EmbedLiteViewThreadChild. The former lives in the same thread as EmbedLiteAppThreadParent does (the main UI thread). And the latter lives in the gecko thread together with EmbedLiteAppThreadChild. The parent end point for the view serves as a communication channel to the corresponding web view which gets actually created by the child end point. The act of web view creation happens in the method EmbedLiteViewThreadChild::InitGeckoWindow(). Instances of EmbedLiteViewThreadChild keep handles to the created "browser windows". When a new "browser window" is created and properly initialized the child end point sends a async Initialized() message to the corresponding parent. The parent end point directly calls the callback ViewInitialized() registered by the toolkit specific embedding. Now the native widget is notified that its web view is fully functional.

Web view initialization

So, what actually happens inside EmbedLIteViewThreadChild::InitGeckoWindow()? First of all we create an object representing web browser, that is the object must implemenent the interface nsIWebBrowser. The reference to this object is kept in the private member EmbedLiteViewThreadChild::mWebBrowser.

Then we create an interface instance for the nsIBaseWindow interface out of the web browser object.

Note

Remember that interface instances of different types can refer to the same physical object implementing more than one interfaces.

Also we create a fake browser widget EmbedLitePuppetWidget inheriting to PuppetWidget and implementing the interface nsIWidget. This is how this abstraction is described in Gecko code:

This "puppet widget" isn't really a platform widget.  It's intended
to be used in widgetless rendering contexts, such as sandboxed
content processes.  If any "real" widgetry is needed, the request
is forwarded to and/or data received from elsewhere.

Then we initialize the base window with the widget:

rv = baseWindow->InitWindow(0, mWidget, 0, 0, mViewSize.width, mViewSize.height);
if (NS_FAILED(rv)) {
  return;
}

The important part is that we initialize the window which hasn't been created yet, because as said in the documentation for the property nsIWebBrowser.containerWindow:

The embedder must create one chrome object for each browser object that is
instantiated. The embedder must associate the two by setting this property
to point to the chrome object before creating the browser window via the
browser's nsIBaseWindow interface.

After that we

  1. create and initialize a chrome object (nsIWebBrowserChrome),
  2. associate it with the web browser object,
  3. finally create the base window (nsIBaseWindow),
  4. request an interface object for nsIDOMWindow from the web browser object,
  5. register the view ID in nsIEmbedAppService,
  6. broadcast the event "embedliteviewcreated" on behalf of the nsIDOMWindow instance to interested observers,
  7. instantiate an interface object for nsIWebNavigation out of the base window,
  8. associate the web browser with the chrome object (so now they know each other),
  9. mark the base window visible,
  10. create a TabChildHelper instance,
  11. send async Initialized() message to the main UI thread.

TabChildHelper is a private object handling various tasks for EmbedLiteViewThreadChild such as viewport calculations and handling scroll events originating from content.

Note

The current goal is to make TabChildHelper share functionality with upstream dom/ipc/TabChild class, in order to reduce maintenance burden.

On compositing

The code of EmbedLitePuppetWidget basically is a copy-paste from mozilla's PuppetWidget class. Would be nice to refactor it to avoid code duplication. Mainly the code differs in how compositor objects are created. In fact the base class PuppetWidget doesn't create any compositor objects since it's a responsibility of a native widget, but embedlite does create a compositor inside this fake widget by calling the static method gfxPlatform::GetPlatform() (see EmbedLitePuppetWidget::CreateCompositor). This method

  1. initializes a font rasterizer library,

  2. initializes Qt's graphic platform (looks like there is no much Qt specific stuff left there),

  3. creates a Cairo surface,

  4. creates the compositor thread and the global compositor map if they haven't been created before (see static void CompositorParent::StartUp()).

    Note

    Only one compositor thread per gecko proccess is created.

  5. creates the image bridge thread connected to the compositor thread via the pair of actors ImageBridgeParent and ImageBridgeChild.

    Note

    The PImageBridge protocol is used to allow isolated threads or processes to push frames directly to the compositor thread/process (from the content thread) without relying on the main thread which might be too busy dealing with content script. Again only one image bridge thread per gecko process can be created.

In addition to that the fake widget creates

  1. an instance of LayerManager class;
  2. an instance of EmbedLiteCompositorParent class which is a subclass of the CompositorParent actor class and a CompositorChild end point. This CompositorChild instance serves as a communication channel to the compositor thread for the LayerManager object;
  3. a shadow manager (a child end point of the LayerTransaction subprotocol);
  4. a shadow forwarder connected to the shadow manager

and registers the shadow manager in the image bridge. Images drawn in the content thread by the layer manager get forwarded through the image bridge to the compositor thread which is supposed to render the images into a GL context. See this page for a better explanation of compositing.

Warning

Currently the EmbedLiteCompositorParent class implements methods that are called from the main UI thread. But the object is supposed to live in the compositor thread. This may become a problem if UI and gecko get moved to separate processes.

Comments

What's behind Sailfish browser    Posted:


In this post I'd like to shade some light on what technology is used in the browser application for Sailfish OS.

By now it's a widely known fact that the browser is based on the Gecko engine which is developed by Mozilla corp. and is used in their Firefox browser and Firefox OS. For some reason it's not that known that the Sailfish browser is built upon the EmbedLite embedding API (also known as IPCLiteAPI) for Gecko.

This embedding API started as a research project in Nokia by Oleg Romashin and Andrey Petrov at the times when Nokia was still developing the Maemo platform. Currently the project is maintained by Tatiana Meshkova.

It would help us a lot if the API made its way to the main Gecko repository and became a part of the engine. Unfortunately this hasn't happened yet and the current status of such integration you can see in these two bugs:

  1. https://bugzilla.mozilla.org/show_bug.cgi?id=746800
  2. https://bugzilla.mozilla.org/show_bug.cgi?id=713681

Later on Oleg has implemented a Qt embedding library (qtmozembed) that uses this API. And this library has enabled the development of a very lightweight Qt-based alternative to the Fennec browser for the Nokia N9 mobile phone. Also this browser has been ported to the Nemo project and packaged as Cutefox. Thanks to Andrey Kozhevnikov (aka CODERus), Stephan Beyerle (aka Morpog), Michael Demetriou (aka qwazix) and Ivaylo Dimitrov (aka freemangordon). The Cutefox browser became a starting point for the Sailfish browser.

Now let's see how all these components work together.

This page breifly describes the architecture of EmbedLite API. The key points are:

  • Native UI and Gecko engine live in different threads. Theoretically they can live in different processes thanks to Mozilla's IPDL.
  • Rendering is based on off main thread compositing which provides multi-threaded responsive rendering. The same as in Firefox OS and Firefox for Android.
  • XUL is not used.

Basically EmbedLite provides API to:

  • start/terminate the Gecko engine in a separate thread/process;
  • create/destroy a web view;
  • install a toolkit specific listeners to receive events generated inside Gecko in the native UI;
  • send messages/events from the native UI to Gecko;
  • get/set Gecko preferences;
  • load XPCOM components needed for features implemented in the native UI.

This functionality is exposed with two main classes: EmbedLiteApp and EmbedLiteView. EmbedLiteApp represents the Gecko engine. Only one instance of this class is allowed to be created. EmbedLiteView represents a web view naturally. It's possible to create many instances of this class.

Also EmbedLite defines two interfaces for toolkit specific listeners: EmbedLiteAppListener and EmbedLiteViewListener. An implementor of a toolkit specific embedding is supposed to implement listeners inhereting to those interfaces. Particularly the Qt embedding I mentioned above implements these two listeners in the classes QMozContextPrivate (derived from EmbedLiteAppListener) and QGraphicsMozViewprivate (derived from EmbedLiteViewListener).

The Sailfish browser doesn't know anything about all these details. They are hidden inside qtmozbed's interface classes:

  1. QMozContext which encapsulates the web engine (via QMozContextPrivate);
  2. QuickMozView which encapsulets a web view (via QGraphicsMozViewPrivate) and provides declarative QtQuick interface. In other words it can be used as a QML component in your Qt application.

Here's a bit outdated static model:

/images/qtembed.png

In the next post I'll describe in more details what happens when EmbedLite is initialized and a web view gets created.

Comments