I’m a heavy user and also a code contributor to Apache Superset. Running Superset on my MacBook is the only reason to have a Docker(still a VM inside?) installed which I think is too heavy. Superset puts most heavy work onto the database side, I was thinking is there may be some possibility to have a to make it easier to use Superset on my MacBook. Superset.app My technical stack is mainly backend, some keywords like: Python, Golang, C, Backend I have absolutely no experience with Objective-C or Swift and don’t even understand how a macOS App and a UNIX executable are related. But it still deserves a try. Packaging Apache Superset is a Python + React project, to avoiding introduce a new stack, I think using Python to develop the UI part might be easy. For the backend, everyone using Python knows that the packaging is a really complex problem. Especially with Python packages including dynamically linked shared libraries(*.dylib or *.so). Some people may say what about using to generate a and install them on the first launch. Well, there may be mainly 2 problems: pip freeze requirement.txt It will break the convention of “Dragging into Trash means Uninstall”. The user’s Python environment may also be polluted and cause various problems. After some Googling, I found there are mainly 3 tools I can use to do the packaging: PyInstaller (9.2k stars, Windows/Linux/macOS) cx_Freeze (905 stars, Windows/Linux/macOS) Py2app (189 stars, macOS only but actively developing) PyInstaller is a pretty old project, a lot of Python-based Windows programs are built with it, and not to mention, it has 9.2k stars. Typically the most “stared” tool is the best choice, but not this time. As far as I know, there is no silver bullet to solve these packaging problems. Most of the packaging tools have to keep a bunch of recipes to solve the tricky libs. Superset introduces a lot of database drivers which are hard to deal with. With years of experience solving the problems, I need a relatively simple tool in case of hacking is needed. So my decision is to give py2app a try. *.dylib *.so Dynamic Libs I suffered from the Linux dynamic lineage problems. Linux kernel ABI is quite stable, and Glibc handles the ABI very carefully. Even libs like OpenSSL tried their best to keep a stable ABI. But the upper-level libs shipped with different Linux distributions and pkg management systems constructed a matrix. When it comes to macOS, most libs shipped with the system are quite stable and things in Linux like , , , are merged into . libc libc++ libpthread libm libSystem.B.dylib Although, the command in macOS is not GNU version, you can still see the big difference of dependent libs of between CentOS 8.1 macOS 12.3. ps /bin/ps and Solving the packaging of dynamic libs is quite nasty. But I think the main principles are just two: Keep necessary lib at its lowest version Trim lib to minimal from the upper level The first principle is easy to understand, lower version lib usually means less API required. But AFAIK, macOS doesn’t provide any cross-compiling way to do that. All I have to do is compile SuperChart on a lower version of macOS. To do that I brought a second-hand Mac Mini released in 2009 and installed macOS 10.15 with . macOS Catalina Patcher For the “Trim lib” part, here is an example of how I tried to solve the dependence tree. To upload SuperChart to MAS (Mac App Store), I have to get rid of the dependence of which introduced by . pyarrow Security.framework pyarrow First, I tried to remove pyarrow from SuperChart. Later I found that it may cost a lot because it is not only involved in the parquet reading part but also in the superset serializing part. After some digging, I realized that the seems to be some system-level OpenSSL in macOS. As we usually use pyarrow for serializing/deserializing or parquet reading, I don't think the crypto stuff is not really necessary for pyarrow. Security.framework Easy to know pyarrow is just a Python binding of . So all I need to do is trim libArrow out of the . Thanks to the well-implemented Arrow compiling system, after some flags are set. I got a very clean . Apache Arrow Security.framework libarrow.500.dylib Frontend For the frontend, there are 2 kinds of choices: GUI Frameworks: PyQT, Tkinter, wxPython, Kivy, PySide (So much old and new frameworks)… Web-Based Frameworks: Electron Here are some useful comparisons or lists for “Python GUI Framework” Tkinter/PyQT/PySide/Kivy Comparison with Helloworld Some old but “official” list Yet another “Awesome XXX”, but with 128k stars It seems there are much more options that can cause confusion. So I have to sort my specific needs instead of just Googling “Python GUI Framework” and choosing the most popular one. Here are my needs: Superset is a really big project, I don’t want to rewrite the whole React stuff I just wanna a macOS app, not a Linux, Windows, or any mobile platform The smaller the better Electron.js is really really popular, after simply: ps aux | grep '(Renderer).app' | grep -v "Google Chrome.app" I came across 3 Apps built with Electron.js that are currently running on my MacBook. But as I know, nearly every OS is shipping with an inside to make it easy for the App to open a web page. Why don't I just use that instead of packaging a Chromium which makes my package approximately 100MB larger? XXWebView So I got which let me open and control to emulate an App UI. And also, thanks to my "the smaller the better" choice made it possible to put to MAS. As Chromium uses some deprecated API of macOS which is not allowed for MAS. pywebview WKWebview SuperChart Maybe the title could be "How I put a Python-based App onto Mac App Store". SuperChart in MAS Also published here.
