Shenanigans in getting a Java swing GUI app to run on Android
In this post I’ll be talking about something a little different.
Let me start off by saying that I quite like Android as an operating system. My first Android device was the Nook Simple Touch which ran Android 2.1, and learning how to root it, install a launcher, and play Angry Birds was an enlightnening experience. From then on, I rooted pretty much every android device I owned, played with custom roms, kernels, and mods, with mostly with no real idea what I was doing or what the various pieces meant or did. I just knew that I could use cool apps that didn’t work unless you rooted, like Xposed modules for youtube background play and global adblock.
Finally, the rooting golden days came to an end as the new phones simply allowed you to do most of the customizations without root. For me the last time I rooted a device was a Sony Xperia XZ2, which was a great phone but had DRM that broke a lot of important functions if you rooted. After that, I got a Samsung S10e, which I never even bothered trying to root.
At some point along the line I stopped being a teenager messing with phones and started working in the Database and Devops fields. I learned Linux, what “rooting” actually meant, and what all the steps I used to do as a hobby actually meant, and also, I learned that Android actually has a Linux kernel running underneath, which I thought was pretty cool, but ultimately - just trivia. I was a strong Linux proponent and advocate at this point, but it wasn’t like that translated into Android in any way.
Then, one evening, I accidentally discovered one of Samsung’s greatest features, Dex. I won’t go into Dex in great detail here, but the main thing is that when you plug the device into an external screen via HDMI, it launches a magical desktop mode on the screen, with full mouse and keyboard support. It still only runs the Android apps you happen to have installed, but in windowed or full screen mode, just like a regular computer.
I don’t quite recall how I discovered it, but there’s an app called Termux, which is an interesting concept - it gives you a full blown Linux environment, interacting with the Linux kernel inside Android. It has its own dedicated package manager and repositories, and you can install many linux programs, like Vim or OpenSSH.
With Dex, you can use it just like you would use the terminal in any linux distro, but naturally there are many limitations. The biggest limitation is there’s no UI. Also, there’s a relatively small selection of packages available.
Enter PRoot.
In modern versions of Android, the kernel is new enough to support containers. PRoot basically allows you to specify a directory and say “launch a container with this arbitrary directory as the / directory”. So, if I were to download an Ubuntu filesystem (using an app called Andronix) into a folder on my phone, I could then use proot to launch a container running ubuntu… with EVERY app that ubuntu has avaialable for the ARM architecture used in phones. This definitely takes care of the app availability in Termux problem, but we still need a way to run GUI apps. So, the generally recommended solution is to run a VNC server in the container, and then use any VNC viewer app from the Play store to connect via localhost. You open the VNC client full screen in Dex, with a physical mouse and keyboard installed, and you now have a full blown desktop Linux environment… running on a phone!
This will allow you to run any Linux app compiled for ARM on your external screen, including Chrome or Firefox (with desktop extensions, of course), VS Code, Libreoffice, Gimp, Blender, or whatever else catches your fancy. The only drawback, which can be a big one for some workloads, is lack of hardware acceleration.
This works really well, and I was able to eliminate the need for a laptop in my work life entirely. I use Outlook and Office for Android in native Dex, and for a browser I use Firefox in a proot. At some point I upgraded my phone to the absolute beast of a Samsung S21 Ultra, with its whopping 12GB of RAM, even more powerful than my old laptop, specifically because it would now also fill the role of daily driver PC. This was finally a true continuum, a device where my Android phone is also may main development device for writing code, and SSHing into servers.
Until Android 12 came along, that is. In Android 12, google silently released a major breaking change, (see also) silently meaning it does not appear in any changelog, which kills background processes using too much CPU or spawning too many background processes, with absolutely no way to disable it.
This effectively prevents any usage of a PRooted distro with a GUI, since the VNC server is killed right away. After reading and finding out that the fix was already added to a pull request against Android, but would only make it into A13 (or maybe A12L) , I started looking for other was to run my apps. Andronix’s documentation explained that there is a method where you start an X server as an app on the Android host, and then by seeting the DISPLAY variable, GUI apps in termux or a proot can open in that X server app.
After some trial and error, I was able to get Ardour running on my phone. I used Ardour as a test app because it is complex and can get CPU intensive, and requiresd using sound as well. I quickly came up with a script for fast launching of GUI apps into my X app, and I was back in business. (You can see examples of my scripts and how to invoke them toward the end of this post.)
Finally, with all that out of the way, I can bring up how I was able to run the JavaFX swing app.
At this point, you probably realize that I’m using a PRoot with java installed. But if you simply tried, it might just not work, as happened to me. The Java program in question is Maptool, a virtual tabletop for RPG games such as Dungeons and Dragons. They do have officially supported installers for linux as well as other OSes, but none for ARM linux. So, I downloaded the JAR file from the github releases, copied it into my PRoot, and tried to run it. At first I tried an apt install openjdk and then a java -jar Maptool.jar , but that failed:
At this point I started debugging the various errors. The official Maptool releases come bundled with JavaFX, but openjdk doesn’t. This particular PRoot was using Fedora and I couldn’t find it in the repos, so I downloaded it manually and copied it into the proot. It probably would have worked, but I didn’t know how to tell java to use JavaFX, so I assumed i had to install a version from the repos. I went to an Ubuntu PRoot, and there i found the openjfx package. Of course simply installing it didn’t work either, and I scoured the internet. It was rather difficult since very few people have tried running java apps on Termux with a GUI and also posted about it online. I found various posts with different parts of the info I needed, and I slowly cobbled together a start script, which used the some of the same stuff from my existing program start scripts and a Java invocation command to properly link JavaFX to the JVM and fix various “doesn’t export” errors that kept getting thrown. It’s worth mentioning that I don’t know Java very well.
Finally, I was able to get Maptool to launch, but the windows were blank and with warnings I couldn’t read:
I thought this was a JavaFX error, and eventually realized it was that the app was trying to use Hardware acceleration. (remember I said earlier that that doesn’t work in PRoot?) Then, finally, I added a Dprism Java flag to my script and it ran!
At this point I couldn’t think of anything. According to the logs, there were no errors, but the program obviously wasn’t working right.
Eventually, I found someone who posted that they were having a similar issue when using VNC. Then I thought I should try running a VNC server like I had done before A12 broke it. I did that, and:
Success!!
̶N̶o̶w̶,̶ ̶I̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶w̶a̶i̶t̶ ̶f̶o̶r̶ ̶a̶ ̶n̶e̶w̶ ̶A̶n̶d̶r̶o̶i̶d̶ ̶u̶p̶d̶a̶t̶e̶ ̶t̶o̶ ̶f̶i̶x̶ ̶t̶h̶e̶ ̶V̶N̶C̶ ̶p̶r̶o̶b̶l̶e̶m̶.̶ ̶I̶’̶l̶l̶ ̶o̶f̶ ̶c̶o̶u̶r̶s̶e̶ ̶k̶e̶e̶p̶ ̶t̶r̶y̶i̶n̶g̶ ̶t̶o̶ ̶g̶e̶t̶ ̶i̶t̶ ̶t̶o̶ ̶r̶u̶n̶ ̶i̶n̶ ̶a̶n̶ ̶X̶ ̶w̶i̶n̶d̶o̶w̶ ̶b̶u̶t̶ ̶f̶o̶r̶ ̶n̶o̶w̶ ̶t̶h̶a̶t̶ ̶e̶n̶d̶ ̶i̶s̶ ̶s̶t̶u̶c̶k̶.̶
EDIT SEPTEMBER 2022:
Android 12L has hit Samsung’s tablets! There is one simple ADB command which must be run, either from another android, a PC, or even the device itself, using some workarounds and wireless ADB. This command worked on A12, but would not persist reboots. Now on the new version of Android, it fixes the problem completely, meaning that I can easily run my desktop apps on my tablet, or my phone, when that gets the next version of Android, be they java or whatever.
The ADB command is:
adb shell settings put global settings_enable_monitor_phantom_procs false
Start scripts for A12 XSDL:
In my PRoot home directory I create a directory called “launchers”. This is where my various start scripts live.
Before launching any start script, I first have to open XSDL and wait 5 seconds for it to be ready.
/home/kobi/launchers/ardour:
#!/bin/bash
export DISPLAY=127.0.0.1:0
export PULSE_SERVER=127.0.0.1:4713
xfwm4 --replace &
/opt/Ardour-6.9.0/bin/ardour6 &I can then run it from inside Termux like so:
./start-fedora.sh "bash /home/kobi/launchers/ardour &"
Or, if I’m already in the proot:
./start-fedora.sh
launchers/ardourFirefox:
#!/bin/bash
export DISPLAY=127.0.0.1:0
export PULSE_SERVER=127.0.0.1:4713
xfwm4 --replace &
firefox &Start script for Maptool with VNC:
And finally, the whole point of this ramble, I proudly present, the start script to launch Maptool (in VNC, so I don’t need to launch the window manager which the xstartup will handle for me):
#!/bin/bash
java --module-path /usr/share/openjfx/lib/ \
--add-modules=javafx.controls,javafx.graphics,javafx.fxml,javafx.web,javafx.swing \
--add-opens=java.base/sun.security.util=ALL-UNNAMED \
--add-opens=java.base/java.util=ALL-UNNAMED \
--add-opens=java.desktop/java.awt=ALL-UNNAMED \
--add-exports=java.desktop/java.awt=ALL-UNNAMED \
--add-opens=javafx.graphics/javafx.scene=ALL-UNNAMED \
--add-exports=javafx.base/com.sun.javafx=ALL-UNNAMED \
--add-exports javafx.base/com.sun.javafx.collections=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.css=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.util=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.stage=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.cursor=ALL-UNNAMED \
--add-exports=javafx.base/com.sun.javafx.reflect=ALL-UNNAMED \
--add-exports=javafx.base/com.sun.javafx.beans=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.scene=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.sg.prism=ALL-UNNAMED \
--add-exports=javafx.base/com.sun.javafx.logging=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.prism=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.glass.ui=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.geom.transform=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.tk=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.glass.utils=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.font=ALL-UNNAMED \
--add-exports=javafx.controls/com.sun.javafx.scene.control=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.scene.input=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.application=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.geom=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.iio=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.prism.paint=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.scenario.effect=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.scenario.effect.impl=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.scenario.effect.impl.prism=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.text=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.scene=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.scene.text=ALL-UNNAMED \
--add-exports=javafx.graphics/javafx.scene.paint=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.embed=ALL-UNNAMED \
--add-exports=javafx.controls/com.sun.javafx.scene.control.behavior=ALL-UNNAMED \
--add-exports=javafx.controls/com.sun.javafx.scene.control.inputmap=ALL-UNNAMED \
--add-exports=javafx.graphics/com.sun.javafx.scene.traversal=ALL-UNNAMED \
-Dprism.order=sw -jar ~/MapTool-1.11.5.jar &Termux / Andronix scripts and shortcuts can be found in my Gitlab snippets repo.
If you made it this far, congratulations. I hope you enjoyed it! And hopefully, if you own an Android and ever wanted to run Java on it, you can now.
Lastly, as a side note, you don’t strictly need Dex. Tablets can use their own screen size to connct with VNC. Dex is only to fully emulate a PC experience, and Motorola’s Ready For is a great alternative to Dex.
Happy Hacking!