A C++ Hello World And The Rose Gold Walled Garden Of Doom

This is Part 3 on my series about cross-compilation. You can check out Part 1 and Part 2 first !

A C++ Hello World And A Glass Of Wine, Oh My !_Wherein we try to use the Microsoft Visual C++ Compiler on Linux_hackernoon.com

A C++ Hello World And the Cute Heartless Rainbow_Wherein we cross-compile Qt Aplications for Windows with a couple of alien tool-chains and the power of rainbow._hackernoon.com

© https://linuxnewbieguide.org

You cannot caters to the needs of Windows and Linux users while ignoring the third major, well, second actually, desktop operating system.

The Operating System I’m talking about is of course developed and commercialized by a company best known as the one who gave Clang to the world, is mostly responsible for maintaining WebKit (after most of the industry moved to Chromium), and created some other amazing open sources softwares such as CUPS.

And for that we should be grateful.

You would think that a company who went to the trouble of starting a completely new compiler to offer a better user experience would make it easy to cross-compile to their platform.

However.

That company is Apple.

Like on Linux and Windows, we will need to acquire and setup 3 pieces. A compiler, some system headers and libraries like libc++ and a Sdk for desktop integration purposes.

If you have done OS X development before, you know that all off that can be found in XCode, a 5GB package of bundled tools most of which we won’t need.

XCode is free software. As in beer. Which we will need a lot of. XCode is however proprietary, which is fine.

The issue is that if you read the Terms and condition attached to XCode, you will find the following clause.

2.7 Restrictions; No Other Permitted Uses The grants set forth in this Agreement do not permit You to, and You agree not to, install, use or run the Apple Software or Apple Services on any non-Apple-branded computer or device, or to enable others to do so.

I’m not a lawyer. However it seems to me that Apple is actively forbidding cross-compilation using their libraries, regardless of the technical do-ability.

Therefore, some of the ideas discussed in the rest of the article could, if applied, void any agreement you have with Apple.

You need an Apple ID, and so you need to create an account on apple.com. I don’t recall having a more terrible account creation experience in a while. The biggest offense is certainly their antiquated security policies.

How not to do account creation and security, courtesy of Apple.

They will then send you an email for verification, which is great. But instead of having a link in the email you will get a code that you cannot even paste and have to type in manually.

You will then search for XCode. Fortunately, some good Samaritans maintain valid download links on Stackoverflow since 2012.

This turned into a “All software is terrible” rant again. I’m sorry.

On a more positive note, someone already set up a nice collection of scripts to get you started with building an OsX toolchain on Unix. Works with Cygwin too !

You will need to clone it.

cor3ntin/osxcross_osxcross - OS X cross toolchain for Linux, *BSD and Windows (Cygwin)_github.com

It’s a fork from Thomas Pöchtrager’s work which I had to patch up.

XCode 7.3 is shipped as a DMG and while this is an osx-specific file format, osxcross come with a script to extract it, making good use of Darling. More on that later.

osxcross/tools/gen_sdk_package.sh Xcode_xxx.dmg

Unfortunately, the open source community is still waiting on apple to release a ld64 linker with support for TBD files v2 that are used in later version of osx to not have to ship the .dylib in the sdk.

TBD files are pretty cool, they are a YAML representation of the symbols included in a dynamic library, alleviating the need to ship the actual library. They are pretty similar in concept to the .lib files that are generated by MSVC when building a DLL. I think TBD files could be used across platforms but for now LLVM can’t handle them (yet ?) and the open-source ld64 can’t handle the new version.

So we will have to stick to a 10.11 SDK. It’s reasonable ! I went to the trouble of supporting xip files that are used to package the later versions of XCode. A format that is inspired by babushka dolls, but with compressed archives instead. Unfortunately, we can’t use anything more recent than XCode 7.3. I hope it will change soon !

You can then run move the generatedMacOSX10.11.sdk.tar.xz to osxcross/tarballs then launch SDK_VERSION=10.11 ./osxcross/build.sh

You will also need to run osxcross/build_llvm_dsymutil.sh

And in no time, you will have a complete toolchain for OSX, for both i386 and x86_64 ( even if you have absolutely 0 reason to build anything in 32 bits when targeting OSX ).

It evens builds my personal favorites : otool and install_name_tool. If you ever built something on OSX, you know how terrible these tool ares. Or rather how terrible the OSX loader is.

I’m properly impressed by the work that went into osxcross.

Configuring QBS is rather straight forward, though there are some things to take care of.

In osxcross/target/bin , run:

ln -s x86_64-apple-darwin15-ld ldcp osxcross-llvm-dsymutil x86_64-apple-darwin15-dsymutil

This will later help clang finding the proper tools. If you want to support multiple tool-chains, put ldin a subfolder

Here is my profile configuration than you can adapt

The -prefix options tells clang where to find the proper ld(ld64) since the system linker is not suitable to link Mach-O application.

The rest is just giving qbs the proper search paths.

Unfortunately, support for .plist in qbs is not portable, so you will encountered an error

ERROR: TypeError: Result of expression 'PropertyList' [[object Object]] is not a constructor.at JavaScriptCommand.sourceCodeat Rule.prepare in /opt/qtcreator/share/qbs/modules/cpp/DarwinGCC.qbs:262:18

Comment out the rule in DarwinGCC.qbs to fix the issue.

Of course, not being able to create info.plist files will be very limitating and it would be great if QBS could handle those files in a platform-agnostic manner.

For now, in all our .qbs project files, we will put the following to disable bundling and therefore Info.plist generation

Depends {name: "bundle"}bundle.isBundle: false

At that point, we are able to build the simple console Hello World seen in Part one

# file helloworldMach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE>

But can it run ?

Oh, darling !

To run our windows application we used wine. There is a rather recent effort — It started in 2012 while WINE started in 1993; Windows 3.1 was just released !— to offer a translation layer, called darling. The project is far from being as mature and it doesn’t seems to have any kind of financial support. I hope it will catch on.

Darling | macOS translation layer for Linux_Darling - macOS translation layer for Linux_darlinghq.org

You can clone and build darling, Follow the instructions on github. On my machine, it took a bit under one hour for the whole thing. Once installed, it’s about 800MB. which is unsurprising as it is a complete system which comes with all the usual tools including g++, ruby, python, Perl, git, bash, swift, ssh…

But, the build complete with no error, and amazingly, it works, and seems very reactive. Being more modern that wine, it is containerized !

Oh, Hi Mac

Adding magic with binfmt

So now we can run a mac command, but what if we could just hide the magic away ?

Using a kernel facility and a systemd service , I was able to create a file /etc/binfmt.d/darling.conf so that the kernel can handle launching Mach-O files.

:Mach-O 64b:M::\xcf\xfa\xed\xfe::/usr/bin/darling_interpreter::Mach-O 32b:M::\xce\xfa\xed\xfe::/usr/bin/darling_interpreter::Mach-O FAT:M::\xca\xfe\xba\xbe::/usr/bin/darling_interpreter:

/usr/bin/darling_interpreter is a script that launches darling shell — it should be executable.

#!/bin/bash/usr/local/bin/darling shell "$1"

The darling documentation suggests that darling <binary> should work, but it does not.

After restarting the binfmt service ( systemctl restart systemd-binfmt ), we are able to launch OSX application transparently.

Which mean we can do this

Any sufficiently advanced technology is indistinguishable from magic

Oh, and by the way, you can do the same thing for windows executables and WINE. Some distributions do that out of the box.

In part 2, I attempted to install the win32 version of the Qt Framework on Linux without using windows. I failed.

Can we get Qt for mac without a mac ?

I did download the installer. It’s a .dmg. It would be an issue, but with the power of darling we can mount DMGs on Linux. No problem at all. It’s the kind of things we do here.

But mounting the Qt installer dmg reveals that it contain a binary and a .dat file rather than a simple folder or something manageable.

Presumably, the binary is the installer. Maybe it runs on darling ? No. Hard dependency on the OpenGL framework.

Great software bundled in crappy unusable packages seems to be a recurring theme.

Is all hope lost again ? Not this time.

We can build Qt for mac, like we attempted for Windows. But it will work. Mac has make . It knows about clang and gcc, it’s very much like Linux in a lot of aspects. There is an UNIX under there after all ( But I always had the feeling that OSX internals are terrible, hidden under a nice user interface. For starter, a large number of tools haven’t been maintained after their upstream version moved to GPLv3, 10 years ago ).

Alas, it means dealing with Qt’s complex build system. It took some time to hack the qmake build files. See, Qt makes the terrible assumption that all toolchain on osx involve xcode. We don’t have xcode.

But once you bypass all the automatic probes and assumptions about what’s installed on the system….

….you can get it to work !

#configure -release -opensource -confirm-license -xplatform macx-cross-clang -skip qtwebengine -nomake examples -nomake tests -prefix /home/cor3ntin/dev/cross-compilers/osx/qt5_10

It was not quite the end of the road. Qt Widgets failed to build because of missing dependencies. QtLocation failed to build because the libc++ headers were too old or broken ( I fixed that by copying the latest libc++ version within the OSX SDK. It worked). Then QtLocation complained because std::auto_ptr was not defined so I hacked a few headers.

I tried to get qwebengine(chromium ) to build, but it uses another build system still, and I had done enough build system hacking for one night.

But in the end most of Qt did build.

It took me a few hours but I got Qt !

And what we have then, is something quite interesting. The binaries are natives Linux ELF, while the frameworks and libraries are Macho-O. That will be handy in a minute.

Hello, I’m a Mac — And I’m a PC. And I am a Mac too. — What ?

Qt is a big piece of software making full use of the underlying system capabilities, in term of OS integration. If we can build that, we can build almost anything.

I initially called my mkspec file darling-clang. I was a bad name. it also prevented qbs to understand that it was a mac build. Rather than renaming the mkspec and rebuilding Qt, I modified qbs. the code of qbs-setup-qt is actually parsing the mkspec’s .conffiles with regular expressions. Somehow, it works. Just don’t breathe on it.

Eventually, once I gave qbs what it expected to understand that we were dealing with mac, I was able to call

qbs-setup-qt osx/qt5_10/bin/qmake osx-x64-qt510

Which created the right profiles and modules. I cleaned up my profiles manually to merge clang-osx and osx-x64-qt510

And then we can compile or magnificent Hello World app !

Compiling an OSX application based on Qt on Linux, using qbs : check

What now ?

Well, we have a complete toolchain, maybe we can check some things ?

Using otool -L , the osx equivalent of ldd we can assert that we indeed link to some libraries

We can use other tools. such as nm or strings or the dreaded install_name_tool

Unfortunately, Darling can’t handle anything remotely graphical yet, So we need a Mac to actually launch our app.

A real mac; It is illegal to visualize Mac OSX, unless it’s on a mac. Some words comes to mind. Imagine my French.

Let’s talk about the mac. The Mac. You probably know the one. Most companies have it.

This Is A True Story.

It used to belong to Bob. It’s Bob’s mac. But then, 5 years ago, Bob died, so T_he Mac_ was offered to Alice. Alice left the company soon after - Pure coincidence, probably.

Yet The Mac was always The Mac. It has no master. It has no puppet either. You can connect to it at ssh [email protected]. The password is simply pass. It’s not in the same LAN as the other servers running on virtual machines somewhere. It’s not administered in anyway, presenting a convenient back door in an otherwise tightly secured network.

When it’s running. Sometime it’s just down for days at a time. It’s not just like people don’t care, they also don’t know where it’s physically located. A mac mini is easy to loose. Under someone desk, wedging an old chair, used as a coffee table.

Last time it crashed, you had to track it down for three days straight, like chasing after a big cat in the mountains. You even tried to call Bob’s widow.

You finally found the mac sandwiched between Carol’s screen and the Oxford Dictionary. “It was the prefect height!”, objected Carol when you took back her 800$ monitor stand. You traded the mac for an old IKEA catalog that Carol found to be as practical, if not more, than a Mac-Mini.

You plugged the mac back in and diligently updated it it to the last version of OSX, “Cougar” (or something, it’s hard to keep track).

And when, a few days later, your coworker got a new car and you lost your home, I was left wondering : Is it appropriate to require a credit card to apply free security fixes ?

But truth is, the mac is doing an important job. Running all the tasks that can only be run on the mac. Signing installers, doing mac packages, iOS stuff, running the New York stock exchange… you are not quite sure, it’s Bob’s after all.

Maybe life would have been different if we could virtualize OSX.

I do happen to have a 2011 mac-mini, gift of a former employer. Its life was a a bit different than the mac’s. It was never loved and spent the last two years in a box. It only gets to see the life of the day for the needs of this article. It is also the reason why I’m 4 days late in my publication schedule. I tried to install High Sierra — Grey screen; I had to reformat, install Lion, then install El Captain. So El captain is what we will be using. It features a whooping 2GB of memory, the system using 3/4 of that.

You should enable VNC, remote share and SSH in the systems parameters of your mac.

This article is starting to be a bit long. So here is a quick visual summary of the work achieved so far:

Definitively improving our productivity.

We should stop fooling around.

• Copy you OSX build of Qt on the OSX Machine. You can use scp -r , rsync or a shared folder ( through samba )
• Copy your helloworld-gui on the machine, again, using scp -r .
• Our cross-compiled build of Qt does not contain macdeployqt. You could get it by installing the official version of Qt on the mac directly. To avoid doing that and to not have to deal with the hairy mess of [install_name_tool](http://doc.qt.io/qt-5/osx-deployment.html), we can set up DYLD_FALLBACK_FRAMEWORK_PATH to point to the folder containing all the Qt*.framework . DYLD_FALLBACK_FRAMEWORK_PATH is somewhat sane but may not work and has some associated security risks. Please don’t use it in shipped applications.

export DYLD_FALLBACK_FRAMEWORK_PATH=/Users/cor3ntin/dev/cross-compilation/qt5_10/lib

• Like on windows, we need to provide a qt.conf file next to our helloworld-gui, to tell Qt where to load its plugins from ( the application will simply not run otherwise ). Mine looks like that

[Paths]Prefix=/Users/cor3ntin/dev/cross-compilation/qt5_10/Plugins=plugins

Now, connected to the mac through sshyou can execute your application and it will appear on the mac screen and on the remote display / VNC session

It feels like we climbed a mountain !

Can we make all that legal ?

Clang, LLVM, ld64 and the associated tools are open source projects. That does not mean the open source versions match the version Apple is using.

In fact, Apple’s Clang is a modified version of Clang proper and they lag a few versions behind upstream. Which is ironic given they started the project.

LD64 And the “cctools” are released under the “Apple Open Source licence” And the versions of that tools used by XCode are 2 years ahead of what the open source community has access too.

The frameworks themselves are not open source, and, as I mention at the beginning, not redistributable.

And the open source alternative cocotron which is now only maintain by the Darling people is insufficient.

There are a few issues with it

• They do not have a build script to actually build an SDK and only install the .dylib . this could probably be fixed rather easily.
• They have a limited set of frameworks and that set is insufficient to build Qt. Qt uses the following framework, those prefixed by !!!! are missing in Darling

* AppKit* ApplicationServices!!!! AssetsLibrary* AudioToolbox!!!! AudioUnit!!!! AVFoundation!!!! Carbon* Cocoa!!!! CoreAudio!!!! CoreBluetooth* CoreFoundation* CoreGraphics!!!! CoreLocation!!!! CoreMedia!!!! CoreMotion* CoreServices* CoreText* CoreVideo!!!! fftreal* Foundation* ImageIO!!!! IOBluetooth* IOKit!!!! OpenCL* QuartzCore* Security!!!! SystemConfiguration

• You could possibly compile the Qt Framework on OSX and then copy them to your Linux machine, it would probably work in most cases.
• Utilizing an SDK that is not the official SDK kinda defeats the purpose of cross compiling to test your software works on mac. you are just testing it works on Darling. There is no guarantee that Darling SDK headers match the official one. Mingw suffers from that issue too.

So it’s technically possible to cross compile complex applications ( including Qt and Qt based one) for Mac on Linux. It’s even possible to run non-graphical applications and unit tests directly and seamlessly.

But it’s illegal to use the SDK making the whole exercise a bit pointless depending your legislation. Open sources alternative exist but may not be sufficient and sufficiently reliable.

And while we can be hopeful, it’s doubtful Apple will ever have more developer friendly politics.

And on that terrible disappointment, it’s time to end !