A Python Runtime on Android · Konubinix' Site

see,
- Android,
- java classes from python,
- python,

I want to get access to the android java classes from python and in particular connect android and a board with a single microcontroller, hence being able to use something like https://github.com/jacklinquan/usbserial4a. Although there are several libraries that provide access to java classes from python, this one depends on pyjnius. Creating a compatibility layer should not be that hard, but I would rather avoid that, so I will prefer solution encompassing pyjnius.

My final objective is to be able to quickly prototype DIY funny useless electronic stuffs using old phones. To have a fast feedback loop, I would remotely control the phone (using rpyc¹) from the comfort of my PC. The phone would provide the battery, processing power and its classical hardware (GNSS, light…) while the microcontroller would provide analog control to deal with motors, sensors, etc. The phone would provide power to the MCU over USB while communicating using serial interface. Because ESP8266 and ESP32 provide wifi and bluetooth, this requirement might be put into question, but it still makes sense when I want to use some disconnected boards like an uno r3.

I will likely need to install the phone application several times, so this should be as easy as dropping a file in the sdcard and that’s all.

I don’t need a graphical interface, only a mean to control it remotely².

I have access to a powerful arm mac or I can use my cheap amd64 pc. I’d rather use the former for it is muuuuch faster.

trying using buildozer

I have already played a lot in the past with kivy and buildozer. Also, this is were pyjnius was born. Therefore, this is the obvious first candidate.

I used to use the kivy/buildozer image in the past, but it seems abandoned now. So let’s follow the documentation using earthly to get the best of both worlds: an evolving setup with the reproducibility of earthly.

I did a lot of trials and errors to find a configuration that works. The variables where:

arm mac or amd64 pc
ubuntu or debian
- if ubuntu, 20.04 or 22.04
opendjdk X for X in [17..21]
openjdk manually installed or via the package manager
the version of p4a, dev, release or default

Here are all my trials. It took ages because the cache is very poorly handled in buildozer/p4a, and I often needed to clean the cache folders to actually understand what happened. Of course, cleaning the cache means very long builds.

VERSION 0.8

IMPORT github.com/Konubinix/Earthfile AS e

generate:
    FROM e+debian-python-user-venv --packages="buildozer" --extra_packages="sed"
    WORKDIR /app
    ARG name=proofofconcept
    RUN buildozer init
    RUN sed -i -r \
        -e "s/^(title = .+)$/title = ${name}/" \
        -e "s/^(package.name = .+)$/package.name = ${name}/" \
        -e 's/^(requirements = .+)$/\1,android,pil,rpyc,plyer,pyjnius   /' \
        -e 's/^(#android.permissions = .+)$/android.permissions = CAMERA,VIBRATE,INTERNET,READ_EXTERNAL_STORAGE,MANAGE_EXTERNAL_STORAGE/' \
        buildozer.spec
    # RUN echo "p4a.branch = v2024.01.21" >> buildozer.spec
    SAVE ARTIFACT buildozer.spec AS LOCAL buildozer.spec

apk:
    # https://buildozer.readthedocs.io/en/latest/installation.html
    # on ubuntu 22.04, macos --platform amd64, jdk 17 => sdkmanager platform-tools -> Warning: IO exception while downloading manifest
    #    replay with --verbose gives -> java.security.NoSuchAlgorithmException: in sdkmanager
    # same with 20.04
    # on ubuntu 22.04, macos --platform amd64, jdk 18 => sdkmanager "platforms-tools" -> too many errors (no more info in --verbose)
    # on ubuntu 22.04, macos --platform amd64, jdk 21 => sdkmanager "platforms;android-31" -> too many errors, Core dumped (no more info in --verbose)
    # on ubuntu 22.04, macos --platform amd64, jdk 17 manually installed like briefcase => sdkmanager "platforms-tools" -> too many errors (no more info in --verbose)
    # on debian, macos --platform amd64, jdk 17 manually installed like briefcase =>
    # on debian, macos --platform arm64, jdk 17 manually installed like briefcase => Aidl cannot be executed You might have missed to install 32bits libs
    # on debian, linux amd64, jdk17 -> ok
    #FROM ubuntu:22.04
    # hack to avoid 20.04 from asking for the timezone -> https://github.com/microsoft/AirSim/issues/3877
        # ARG DEBIAN_FRONTEND=noninteractive
        # ENV TZ=Europe/Paris
        # RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
    # end of hack
    # RUN apt update && apt install -y git zip unzip openjdk-17-jdk python3-pip autoconf libtool pkg-config zlib1g-dev libncurses5-dev libncursesw5-dev libtinfo5 cmake libffi-dev libssl-dev
    # RUN apt install -y libffi-dev zlib1g-dev
    # RUN pip3 install --upgrade buildozer Cython==0.29.33 # virtualenv  # the --user should be removed if you do this in a venv
    FROM e+debian-python-user-venv --packages "buildozer Cython==0.29.33" --extra_packages="build-essential libssl-dev automake curl patch libffi-dev zlib1g-dev  openjdk-17-jdk git zip unzip python3-pip autoconf libtool pkg-config zlib1g-dev libncurses5-dev libncursesw5-dev libtinfo5 cmake libffi-dev libssl-dev"
    USER root
    ENV VIRTUAL_ENV=y # to avoid ths --user
    # install jdk manually, cloning the bebavior of briefcase
        # RUN apt purge -y openjdk-17-jdk
        # RUN apt update && apt install -y curl

        # ENV JDK_MAJOR_VER=17
        # ENV JDK_RELEASE=17.0.12
        # ENV JDK_BUILD=7
        # ENV JAVA_HOME=/opt/openjdk
        # ENV PATH="${JAVA_HOME}/bin:${PATH}"

        # RUN curl --fail --silent --show-error --location -o /tmp/openjdk.tar.gz \
        #         https://github.com/adoptium/temurin${JDK_MAJOR_VER}-binaries/releases/download/jdk-${JDK_RELEASE}+${JDK_BUILD}/OpenJDK${JDK_MAJOR_VER}U-jdk_x64_linux_hotspot_${JDK_RELEASE}_${JDK_BUILD}.tar.gz && \
        #     mkdir -p ${JAVA_HOME} && \
        #     tar -xzf /tmp/openjdk.tar.gz -C ${JAVA_HOME} --strip-components=1 && \
        #     rm /tmp/openjdk.tar.gz
    # end of install

    WORKDIR /app
    COPY --dir main.py /app
    ARG gen=yes
    IF test "${gen}" = "yes"
        COPY --dir +generate/buildozer.spec /app/
    ELSE
        COPY --dir ./buildozer.spec /app/
    END
    CACHE /root/.buildozer
    CACHE /app/.buildozer
    ARG cleanapp=no
    IF test "${cleanapp}" = "yes"
       RUN --no-cache rm -rf /app/.buildozer/*
    END
    ARG cleanroot=no
    IF test "${cleanroot}" = "yes"
       RUN --no-cache rm -rf /root/.buildozer/*
    END
    RUN apt update
    RUN apt install -y libltdl-dev
    RUN yes y | buildozer android debug | grep -vi download
    RUN mv bin/*apk app.apk
    SAVE ARTIFACT app.apk AS LOCAL app.apk

I somehow managed to get a build from some of those setups, but barely that I could reproduce easily after cleaning everything.

In the end, the only setup that kinda worked without much hassle was the officially documented one: amd64, ubuntu 22.04, and the recommended list of packages. Too bad for the use of mac…

There are several great features that come with this setup:

I can quickly prototype graphical interface if need be by simply putting python code in the sdcard
I don’t actually need to bother with one for now,
it comes with kivy that is very nice to use (if need be),
it does not expect some boilerplate code to run, only the spec file and the code,

I can even run the app as a service, making it run in the background.

As a conclusion, it works and should continue working as long as kivy gets some momentum. I had to play a lot with it and dig into several github issues to find some solutions, so it was definitely not easy. But, it appears to provide enough flexibility to adapt to my edge case pretty well.

trying with with briefcase

This is the new kid in the block. It uses chaquopy, that feels more curated and less hacky than python-for-android, but younger and much less featureful.

Unfortunately, adding a built dependency such as pyjnius appears to be hard. Trying using it will raise: I get "Chaquopy cannot compile native code". They have no incentive making it available for they already provide an alternative. See how jpype won’t make it in there. On the other hand, it might be an hint that they want to curate a small subset of working libraries, so that might be a good sign after all.

Also, it needs some boilerplate code and therefore might lead to a difficult maintenance.

In addition, it comes with the graphical interface and it is not clear whether I can avoid using it or not.

VERSION 0.8

IMPORT github.com/Konubinix/Earthfile AS e

generate:
    FROM e+debian-python-user-venv --packages "briefcase" --extra_packages="git"
    WORKDIR /app
    RUN briefcase new --no-input
    RUN sed -i -r 's/^requires = \[/requires = \["rpyc",/' helloworld/pyproject.toml
    SAVE ARTIFACT helloworld AS LOCAL helloworld

apk:
    FROM --platform linux/amd64 e+debian-python-user-venv --packages "briefcase" --extra_packages="python3 git build-essential pkg-config python3-dev python3-venv"
    WORKDIR /app
    # CACHE /app/build # is not updated automatically ?
    CACHE /home/sam/.cache
    CACHE /root/.gradle
    USER root # to get access to the cache
    ARG cleanapp=no
    IF apk "${cleanapp}" = "yes"
       RUN --no-cache rm -rf /home/sam/.cache/*
    END
    ARG cleanroot=no
    IF apk "${cleanroot}" = "yes"
       RUN --no-cache rm -rf /root/.gradle/*
    END
    ARG gen=yes
    IF test "${gen}" = "yes"
        COPY --dir +generate/helloworld /app/
    ELSE
        COPY --dir ./helloworld /app/
    END
    COPY --dir ./app.py /app/helloworld/src/helloworld/app.py
    WORKDIR /app/helloworld
    RUN yes y | briefcase create android --no-input
    RUN briefcase build android
    RUN mv /app/helloworld/build/helloworld/android/gradle/app/build/outputs/apk/debug/app-debug.apk app.apk
    SAVE ARTIFACT app.apk AS LOCAL app.apk

It will work in mac, with the amd64 emulation, so that’s a plus. With arm64 it will complain that “Briefcase cannot install Android SDK on this machine.”

earthly --no-cache +apk --cleanroot=yes --cleanapp=yes

Takes 13 minutes.

Let’s replace the code of the app by the call to rpyc.

<<rpyc.org:serve-ex()>>

As the time of writing this, there is no way to run the application as a service (https://github.com/beeware/briefcase/discussions/1553).

As a conclusion, this technology is more opinionated about using the full beeware stack and in particular running a toga application. It does this job pretty well. But for edge cases like mine, it feels like it might not be a good candidate.

do it for real

As per my previous experiments, I decided to use buildozer.

the build

To use usbserial4a, the documentation says that I need to add a content filter. I actually found out that removing it did not change the fact I could discuss with the board.

This is the content that was asked to add.

<intent-filter>
  <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
</intent-filter>

<meta-data android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
           android:resource="@xml/device_filter" />

Now, let’s create an application, adding a bunch of dependencies that I deem useful.

VERSION 0.8

IMPORT github.com/Konubinix/Earthfile AS e

generate:
    FROM e+debian-python-user-venv --packages="buildozer" --extra_packages="sed"
    WORKDIR /app
    ARG title=proofofconcept
    ARG name=konixpoc
    ARG domain=eu.konubinix
    RUN buildozer init
    # permissions taken from https://github.com/kivy/kivy-launcher/blob/main/buildozer.spec
    # plumbum is required by rpyc
    # I added several dependencies that I wish I add in my previous PoC, so that I will be able to use them when my current phone will become old see below my experiment with the wiko cink peax
    # some packages that I wanted but cannot build
    # - scipy needs lapack that needs fortran that needs a legacy ndk
    # - zmq fails https://github.com/kivy/python-for-android/issues/2706
    ARG extra=",able_recipe,dbus-next,usbserial4a,usb4a,Pillow,av,av_codecs,cryptography,ffpyplayer,ffpyplayer_codecs,matplotlib,pandas,pycryptodome,sqlite3,requests,numpy,opencv,opencv_extras"
    RUN sed -i -r \
        -e "s/^(title = .+)$/title = ${title}/" \
        -e "s/^(package.name = .+)$/package.name = ${name}/" \
        -e "s/^(package.domain = .+)$/package.domain = ${domain}/" \
        -e "s/^(requirements = .+)$/\1,android,plumbum,rpyc,plyer,pyjnius,oscpy,requests${extra}/" \
        -e "s/# android.accept_sdk_license = False/android.accept_sdk_license = True/" \
        -e 's/^(#android.permissions = .+)$/android.permissions = SEND_SMS,READ_SMS,RECEIVE_SMS,WRITE_SMS,READ_PHONE_STATE,RECORD_AUDIO,CAMERA,WRITE_SETTINGS,VIBRATE,MANAGE_EXTERNAL_STORAGE,READ_EXTERNAL_STORAGE,WRITE_EXTERNAL_STORAGE,ACCESS_LOCATION_EXTRA_COMMANDS,ACCESS_NETWORK_STATE,ACCESS_NOTIFICATION_POLICY,ACCESS_WIFI_STATE,BLUETOOTH,BLUETOOTH_ADMIN,BROADCAST_STICKY,CHANGE_NETWORK_STATE,CHANGE_WIFI_MULTICAST_STATE,CHANGE_WIFI_STATE,DISABLE_KEYGUARD,EXPAND_STATUS_BAR,FOREGROUND_SERVICE,GET_PACKAGE_SIZE,INSTALL_SHORTCUT,INTERNET,KILL_BACKGROUND_PROCESSES,MANAGE_OWN_CALLS,MODIFY_AUDIO_SETTINGS,NFC,READ_SYNC_SETTINGS,READ_SYNC_STATS,RECEIVE_BOOT_COMPLETED,REORDER_TASKS,REQUEST_COMPANION_RUN_IN_BACKGROUND,REQUEST_COMPANION_USE_DATA_IN_BACKGROUND,REQUEST_DELETE_PACKAGES,REQUEST_IGNORE_BATTERY_OPTIMIZATIONS,SET_ALARM,SET_WALLPAPER,SET_WALLPAPER_HINTS,TRANSMIT_IR,USE_FINGERPRINT,VIBRATE,WAKE_LOCK,WRITE_SYNC_SETTINGS,BLUETOOTH, BLUETOOTH_ADMIN, BLUETOOTH_SCAN, BLUETOOTH_CONNECT, BLUETOOTH_ADVERTISE, ACCESS_FINE_LOCATION/' \
        -e "s|^(#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY)$|services = ${name}:./service.py:foreground:sticky|" \
        buildozer.spec

    # -e "s/^(fullscreen = .+)$/fullscreen = 1/" \ to get the app in fullscreen mode
    # RUN echo "p4a.branch = v2024.01.21" >> buildozer.spec
    # appears not to be required -> -e "s/^#android.manifest.intent_filters =.+$/android.manifest.intent_filters = intent-filter.xml/" \

    SAVE ARTIFACT buildozer.spec AS LOCAL buildozer.spec

apk:
    # https://buildozer.readthedocs.io/en/latest/installation.html
    FROM ubuntu:22.04
    ARG extra_deps="ccache"
    ARG lzma_deps="autopoint"
    RUN apt update && apt install -y ${lzma_deps} ${extra_deps} git zip unzip openjdk-17-jdk python3-pip autoconf libtool pkg-config zlib1g-dev libncurses5-dev libncursesw5-dev libtinfo5 cmake libffi-dev libssl-dev
    RUN pip3 install --upgrade buildozer Cython==0.29.33 # virtualenv  # the --user should be removed if you do this in a venv
    WORKDIR /app
    CACHE --id root-buildozer /root/.buildozer
    CACHE /app/.buildozer
    CACHE /root/.gradle
    ARG cleanp4a=no
    IF test "${cleanp4a}" = "yes"
       RUN --no-cache rm -rf /app/.buildozer/android/platform/python-for-android/
    END
    ARG cleanapp=no
    IF test "${cleanapp}" = "yes"
       RUN --no-cache rm -rf /app/.buildozer/*
    END
    ARG cleanroot=no
    IF test "${cleanroot}" = "yes"
       RUN --no-cache rm -rf /root/.buildozer/* /root/.gradle/*
    END
    ARG gen=yes
    IF test "${gen}" = "yes"
        COPY --dir +generate/buildozer.spec /app/
    ELSE
        COPY --dir ./buildozer.spec /app/
    END
    COPY --dir service.py main.py intent-filter.xml /app
    # skip the download lines to avoid killing my pc with all the output
    RUN yes y | buildozer android debug | grep -vi download && mv bin/*apk app.apk
    SAVE ARTIFACT app.apk AS LOCAL app.apk

Now, let’s build it

Takes 35 minutes with no cache and 1 minute when the cache is warm and I only edited a python file. 25 minutes with a warn /root. It generates a 40M apk.

And with all the dependencies: 1h30m with a cleanapp=yes but a fresh cache for the rest. It generates a 150M apk.

the entrypoints

I want the application to load whatever code I decided to put in the sdcard.

Therefore, I need to split the code in two: the part that will be run inside the apk, and the part inside the sdcard. The former is responsible to find and run the later.

inside the apk

The main application will try to find /sdcard/poc/poc.py:main()

import sys
from jnius import autoclass
from pathlib import Path

from android.permissions import request_permissions, Permission
request_permissions([Permission.WRITE_EXTERNAL_STORAGE,Permission.READ_EXTERNAL_STORAGE])

print("STARTED MAIN")
Environment = autoclass('android.os.Environment')
sys.path.append(str(Path(Environment.getExternalStorageDirectory().absolutePath) / "poc"))

from poc import main
main()

I did not put any wait on the permissions, therefore, it will likely crash the first time will you press the accept button and work the second time.

Also, if you are running android 10+, you will have to accept that the application has access to the sdcard.

clk android adb shell appops set --uid eu.konubinix.konixpoc MANAGE_EXTERNAL_STORAGE allow

For more information, see the note about accessing the files.

The service will try to run /sdcard/poc/poc.py:service()

import sys
from jnius import autoclass
from pathlib import Path

print(f"STARTED SERVICE -> {os.environ['PYTHON_SERVICE_ARGUMENT']}")
Environment = autoclass('android.os.Environment')
sys.path.append(str(Path(Environment.getExternalStorageDirectory().absolutePath) / "poc"))

from poc import service
service()

inside the sdcard

Now, you can put whatever you want in the main() and service() functions in /sdcard/poc/poc.py

I personally decided to split the main and service code in separate modules.

Therefore, I created a directory sdcard/poc/app in which I put the service and the main code.

My entry point (/sdcard/poc/poc.py) looks like this

def main(*args, **kwargs):
    from app import main
    main.run()


def service(*args, **kwargs):
    from app import service
    service.run()

Then, I created a main that simply runs the service and exits.

def run():
    from jnius import autoclass
    service = autoclass('eu.konubinix.konixpoc.ServiceKonixpoc')
    mActivity = autoclass('org.kivy.android.PythonActivity').mActivity
    argument = "FOOBAR"
    service.start(mActivity, 'presplash', 'title', 'content', argument)

To make my android a remotely controlled device, I will simply run rpyc.

Note that I need to import a few stuff for rpyc to correctly provide them afterwards. I learn along the way and add them when I realize I need them.

I like being notified when the service is ready, hence the vibration at the beginning.

About PythonService.mService.setAutoRestartService(True), your mileage may vary. I decided that I wanted a long running service and not bother restarting it. This is also why I defined it as foreground and sticky in the buildozer definition.

import os
import sys
from jnius import autoclass
from pathlib import Path
# I need to start using at least one function of plyer from here, otherwise,
# when controlling from rpyc, it will raise notimplementederror
import plyer
# Same, trigger the load from here so that rpyc is ok
# from usbserial4a import serial4a
# from able import BluetoothDispatcher, GATT_SUCCESS

def run():
    plyer.vibrator.vibrate()
    print(f"STARTED SERVICE POC -> {os.environ['PYTHON_SERVICE_ARGUMENT']}")
    # see https://python-for-android.readthedocs.io/en/develop/services.html
    Environment = autoclass('android.os.Environment')
    PythonService = autoclass('org.kivy.android.PythonService')
    PythonService.mService.setAutoRestartService(True)
    rpyc()

def rpyc():
    nil

make it shine!

As described in the note about accessing the files, from android 10, you need to explicitly give the consent for the application to access the files.

This can be done from the command line.

clk android adb shell appops set --uid eu.konubinix.konixpoc MANAGE_EXTERNAL_STORAGE allow

Or from the dedicated graphical interface that you will find in the settings.

I personally start it with

clk android adb shell am start -a android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION

Then, you can run the application.

clk android adb shell am start -n eu.konubinix.konixpoc/org.kivy.android.PythonActivity

Starting: Intent { cmp=eu.konubinix.konixpoc/org.kivy.android.PythonActivity }

And to stop it

clk android adb shell am force-stop eu.konubinix.konixpoc

You should feel the phone vibrating and the

to run a graphical user interface

Pick whatever example from https://github.com/kivy/kivy/tree/master/examples and put it in /sdcard/poc/app/myexample

As long as the app loads this example, you should be good to go.

While trying this, I found several limitations though.

the audio example makes the application crash after the first sound is played, (looks linked to kivy on android),
the demo/showcase fails because kivy.extras.highlight.KivyLexer is not part of the build
the demo/multistroke fails because either kivy does not find the .kv files, or it does not manage to import the historymanager. The later is easily fixed with appending the path to sys.path, but I don’t know yet how to fix the former. It might not be a big deal, but not something I want to tackle for now.

As long as the application is quite simple, I suppose this should work.

For instance, take the demo/pictures one, drop it in /sdcard/poc/app/pictures and make the main.py:run() code load this example.

def run():
    from .pictures.main import PicturesApp
    PicturesApp().run()

It works without issue on my bullhead

note about accessing the files

Before android 10, an application could read and write the sdcard, provided the application asked for it.

Introduced in 10 and enforced in 11, now applications have to use the scoped storage, in which the application won’t directly access the files, but asks the user for files using dedicated intents.

Fortunately, there is still a way to circumvent this, using MANAGE_EXTERNAL_STORAGE that provides file access to the application. It was meant for applications that needs to get them, because it does not make sense otherwise, like antivirus. In my mind, this is one of those use cases.

Once installed, one needs to find the setting to “allow access to all files” in android to allow the application.

clk android adb shell am start -a android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION

Starting: Intent { act=android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION }

You can do this automatically

clk android adb shell appops set --uid eu.konubinix.konixpoc MANAGE_EXTERNAL_STORAGE allow

Doing this, you can continue loading a python module from the sdcard and get a very fast feedback loop.

note about permissions

Apart from the more specific case of MANAGE_EXTERNAL_STORAGE, you can grant permission from the command line using commands such as

clk android adb shell pm grant eu.konubinix.konixpoc android.permission.CAMERA

trying with an arduino board

Using an UNO R3 and the code from with arduino.

Now, let’s try to play with it from the comfort of my computer, while plugged on my phone.

import rpyc

c = rpyc.classic.connect(ip, int(port))

usb = c.modules.usb4a.usb
serial4a = c.modules.usbserial4a.serial4a
from pprint import pprint

usb_device_list = usb.get_usb_device_list()
usb_device_name_list = [device.getDeviceName() for device in usb_device_list]
usb_device_dict = {
    device.getDeviceName():[            # Device name
        device.getVendorId(),           # Vendor ID
        device.getManufacturerName(),   # Manufacturer name
        device.getProductId(),          # Product ID
        device.getProductName()         # Product name
    ] for device in usb_device_list
}
pprint(usb_device_dict)

portstr = usb_device_list[0].getDeviceName()
device = usb.get_usb_device(portstr)

if not usb.has_usb_permission(device):
    usb.request_usb_permission(device)
    exit(0) # needs to wait for the user input before going further

serial_port = serial4a.get_serial_port(
    portstr,
    9600,   # Baudrate
    8,      # Number of data bits(5, 6, 7 or 8)
    'N',    # Parity('N', 'E', 'O', 'M' or 'S')
    1,      # Number of stop bits(1, 1.5 or 2)
    timeout=1,
)
if not serial_port.is_open:
    print("Serial port not open, that's strange")
    exit(1)

serial_port.write(b'Hello world!')
print([line.decode().strip() for line in serial_port.readlines()])
serial_port.close()

{'/dev/bus/usb/001/002': [9025, 'Arduino (www.arduino.cc)', 67, None]}
['Received: H', 'Received: e', 'Received: l', 'Received: l', 'Received: o', 'Received:', 'Received: w', 'Received: o', 'Received: r', 'Received: l', 'Received: d', 'Received: !']

That rocks !

note about building on old phones

To build for android 4, chatgpt tells me that I need to target the API 19 (from 14 to 19) and ndk 19, the sdkmanager teels me that

+apk | [WARNING]: Target API 19 < 30 +apk | [WARNING]: Target APIs lower than 30 are no longer supported on Google Play, and are not recommended.

Not being able to easily recreated an old apk incites me to create a very generic application with as many useful dependencies as needed and allowing to drop some files to the sdcard and archive it. That way, I can get back to an old phone when I want instead of throwing them away.

Even so, the differences of versions of python/rpyc etc will most likely make this difficult, but at least I will be able to try.

archiving for later

Because building on old phones might be difficult, let’s try to build one and keep it for later.

with sdk 31, ndk 23b

without extras

earthly --config /dev/null +apk --extra="" && ipfa app.apk && du -sh app.apk

[[https://konubinix.eu/ipfs/bafybeia3mruhulszc67l5nygmggkrqkpe3ju6r4ntkhby2y2kdz7esdthy?filename=app.apk][app.apk]]
35M	app.apk

earthly --config /dev/null +generate --extra="" && ipfa buildozer.spec

[[https://konubinix.eu/ipfs/bafkreic6jvpktix3z2nkn76kbwfgxg5e2ju7rfxzr2xhnxptuouc44cnfm?filename=buildozer.spec][buildozer.spec]]

with extras

earthly --config /dev/null +apk && ipfa app.apk && du -sh app.apk

[[https://konubinix.eu/ipfs/bafybeifabte2qwcoodwqty55qla2ieiynrqp73e3obtugdxvib7vwdju3q?filename=app.apk][app.apk]]
146M	app.apk

earthly --config /dev/null +generate && ipfa buildozer.spec

[[https://konubinix.eu/ipfs/bafkreifiys54ouqja3rff5ox7dzdo77y72awzfb5tqdgmc6xbpgvqt5224?filename=buildozer.spec][buildozer.spec]]

I tried on my zte blade s6 running android 5, I could communicate with the arduino board easily.

a legacy from another lifetime

It is actually not the first time I wanted to do this. Here is an apk that should work on older phones (android 4.1.2 and less).

poc.apk

Note that while playing with it (see from the python android runtime to a custom app), I made some improvements, so I publish here the last version. For instance, I applied the fix in

https://github.com/kivy/python-for-android/pull/382/commits/f02e6ea956908bc08f42cf631c656a7ef4b301c6.

I made it contain the same code as the more recent app. That way, I can put the same code in and old and a new device, provided I write a code compatible with python 2 AND 3 (without f-string for instance).

I also implemented a way to access dispatchKeyEvent, as per access hardware volume keys in kivy.

It will work-ish with rpyc, but not in a satisfying way (see trying on my old wiko cink peax). A lot of EOFError: connection closed by peer and plyer needs several workaround to work that way.

wakelock off, see enable wakelock
orientation portrait, see change the orientation
no fullscreen, see change fullscreen settings
service ran using the new (in 2017) service API (it will crash on the AndroidService or start_service one https://github.com/kivy/python-for-android/issues/1049) In /sdcard/poc/main.py
```
def run():
    from jnius import autoclass
    from android import mActivity
    service = autoclass('org.poc.poc.ServicePoc')
    argument = ''
    service.start(mActivity, argument)
```
Then, it will run /sdcard/poc/service/main.py:run(). It will show you in the notification the status of the service. In case of Exception, it will put the stack trace in /sdcard/serviceerror.txt
package=“org.poc.poc”
platformBuildVersionCode=“19”
platformBuildVersionName=“4.4.2-1456859”
python 2.7.2
kivy 1.10.0
rpyc 4.1.5
recently added requests, oscpy

All of those are now available with new major version (hence incompatible ones).

Plus, python 2 reached its end of life in 2020, so using it might be like shooting oneself in the foot.

trying on my old wiko cink peax

Because I have an old phone (android 4.1.2), I decided to give it a try.

clk android -d cinkpeax adb install poc.apk
clk android -d cinkpeax adb shell am start -n org.poc.poc/org.kivy.android.PythonActivity

I tried putting the following code in the /sdcard/poc/main.py file

import os

os.a = (locals(), globals())
os.environ["NO_COLOR"] = "1"

from rpyc.core import SlaveService
from rpyc.utils.server import ForkingServer
import rpyc
import sys

print("VERSIONS: " + str(rpyc.__version__) + sys.version)


def run():
    ForkingServer(
        SlaveService,
        hostname="0.0.0.0",
        port=9999,
        reuse_addr=True,
    ).start()

Fortunately, python 2 is still available in nix, although considered “INSECURE”.

Note: installing kivy 1.10.0 was is involved, because it needs cython and some compilation steps. I did not try this. I would only need this to be able to test a graphical interface before uploading it to the phone, so that’s not a blocker.

I pushed the code of usb4a and usbserial4a in the phone.

I could play with it, but several functions did not work, at least not using rpyc.

>>> usb.get_usb_device_list()
[<android.hardware.usb.UsbDevice at 0x53777de0 jclass=android/hardware/usb/UsbDevice jself=<LocalRef obj=0x1de0086e at 0x52f60550>>]

In [34]: d.getDeviceClass()                                                                                                                                                                                    (6 results) 11:52:01 [514/2375]
Out[34]: 2

In [35]: d.getDeviceName()
Out[35]: '/dev/bus/usb/001/006

In [37]: d.getDeviceProtocol()
Out[37]: 0

In [39]: device.getManufacturerName()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-39-321147138486> in <module>()
----> 1 device.getManufacturerName()

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/netref.pyc in __getattr__(self, name)                                                                                                                                              151             return syncreq(self, consts.HANDLE_GETATTR, name)
    152     def __getattr__(self, name):
--> 153         return syncreq(self, consts.HANDLE_GETATTR, name)
    154     def __delattr__(self, name):
    155         if name in _local_netref_attrs:

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/netref.pyc in syncreq(proxy, handler, *args)
     70         raise ReferenceError('weakly-referenced object no longer exists')
     71     oid = object.__getattribute__(proxy, "____oid__")
---> 72     return conn.sync_request(handler, oid, *args)
     73
     74 def asyncreq(proxy, handler, *args):

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/protocol.pyc in sync_request(self, handler, *args)
    521         isexc, obj = self._sync_replies.pop(seq)
    522         if isexc:
--> 523             raise obj
    524         else:
    525             return obj

AttributeError: 'android.hardware.usb.UsbDevice' object has no attribute 'getManufacturerName

========= Remote Traceback (1) =========
Traceback (most recent call last):
  File "/home/test/project/.buildozer/android/platform/build/dists/poc/private/lib/python2.7/site-packages/rpyc/core/protocol.py", line 347, in _dispatch_request
  File "/home/test/project/.buildozer/android/platform/build/dists/poc/private/lib/python2.7/site-packages/rpyc/core/protocol.py", line 630, in _handle_getattr
  File "/home/test/project/.buildozer/android/platform/build/dists/poc/private/lib/python2.7/site-packages/rpyc/core/protocol.py", line 596, in _access_attr
AttributeError: 'android.hardware.usb.UsbDevice' object has no attribute 'getManufacturerName

In [48]: usb.has_usb_permission(device)
---------------------------------------------------------------------------
jnius.jnius.JavaException                 Traceback (most recent call last)
<ipython-input-48-e5961d8985eb> in <module>()
----> 1 usb.has_usb_permission(device)

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/netref.pyc in __call__(_self, *args, **kwargs)
    197         def __call__(_self, *args, **kwargs):
    198             kwargs = tuple(kwargs.items())
--> 199             return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
    200         __call__.__doc__ = doc
    201         return __call__

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/netref.pyc in syncreq(proxy, handler, *args)
     70         raise ReferenceError('weakly-referenced object no longer exists')
     71     oid = object.__getattribute__(proxy, "____oid__")
---> 72     return conn.sync_request(handler, oid, *args)
     73
     74 def asyncreq(proxy, handler, *args):

/home/sam/test/kivy/venv/lib/python2.7/site-packages/rpyc/core/protocol.pyc in sync_request(self, handler, *args)
    521         isexc, obj = self._sync_replies.pop(seq)
    522         if isexc:
--> 523             raise obj
    524         else:
    525             return obj

jnius.jnius.JavaException: JVM exception occurred: Unknown exception code: 1401250256 msg null

========= Remote Traceback (1) =========
Traceback (most recent call last):
  File "/home/test/project/.buildozer/android/platform/build/dists/poc/private/lib/python2.7/site-packages/rpyc/core/protocol.py", line 347, in _dispatch_request
  File "/home/test/project/.buildozer/android/platform/build/dists/poc/private/lib/python2.7/site-packages/rpyc/core/protocol.py", line 624, in _handle_call
  File "/sdcard/poc/usb4a/usb.py", line 112, in has_usb_permission
    return usb_manager.hasPermission(usb_device)
  File "jnius/jnius_export_class.pxi", line 906, in jnius.jnius.JavaMultipleMethod.__call__ (jnius/jnius.c:29136)
  File "jnius/jnius_export_class.pxi", line 638, in jnius.jnius.JavaMethod.__call__ (jnius/jnius.c:25547)
  File "jnius/jnius_export_class.pxi", line 732, in jnius.jnius.JavaMethod.call_method (jnius/jnius.c:26739)
  File "jnius/jnius_utils.pxi", line 93, in jnius.jnius.check_exception (jnius/jnius.c:4317)
JavaException: JVM exception occurred: Unknown exception code: 1401250256 msg null

But, putting this code inside the main.py, I could manage to get some communication with the arduino board.

import os
from usbserial4a import serial4a
from usb4a import usb

get_serial_port = serial4a.get_serial_port

l = usb.get_usb_device_list()
d = l[0]
portstr = d.getDeviceName()
device = usb.get_usb_device(portstr)

usb.request_usb_permission(device)

serial_port = serial4a.get_serial_port(
    portstr,
    9600,  # Baudrate
    8,  # Number of data bits(5, 6, 7 or 8)
    'N',  # Parity('N', 'E', 'O', 'M' or 'S')
    1,  # Number of stop bits(1, 1.5 or 2)
    timeout=1,
)

serial_port.write(b'Hello world!')
message = [line.decode().strip() for line in serial_port.readlines()]
#serial_port.close()

os.a = (locals(), globals())
os.environ["NO_COLOR"] = "1"

from rpyc.core import SlaveService
from rpyc.utils.server import ForkingServer
import rpyc
import sys

print("VERSIONS: " + str(rpyc.__version__) + sys.version)


def run():
    ForkingServer(
        SlaveService,
        hostname="0.0.0.0",
        port=9999,
        reuse_addr=True,
    ).start()

Now, I can see some data flowing between the devices, although some of it gets lost.

In [120]: serial_port.write("truc")
Out[120]: 4

In [121]: serial_port.readline()
Out[121]: 'Reeceived: r\r\n

In [122]: serial_port.readline()
Out[122]: 'Received: u\r\n

In [123]: serial_port.readline()
Out[123]: 'Received: c\r\n

In [124]: serial_port.readline()
Out[124]:

As a conclusion, I proved that I could make something out of an old phone and play with some board, but it lacked a bit of the fast feedback loop aspect that makes python enjoyable in the first place.

Also, I was lucky that I could still run ipython with python2 and rpyc3.

There are so many abandoned old phones out there with which I would have less troubles. Thus, I don’t think this use case is worth it.

Notes linking here

4 (braindump)
buildozer (braindump)
change the orientation (braindump)
connect to a wiimote (braindump)
discover where my chicken get out of the fence (braindump)
enable wakelock (braindump)
from the python android runtime to a custom app (braindump)
pomodo timer on kivy (braindump)
simple python web server in android (braindump)
some note if using with kivy (braindump)
use https on old android phones (braindump)
use the camera with kivy on android
wiko cink peax 2 (braindump)
zte blade s6 (p839f30) (braindump)

Permalink

It would have been awesome to use a jupyter kernel instead, but last time I tried, building libzmq was veeery difficult. ↩︎
And even if I wanted a, interface, I would likely create a progressive web app using alpine.js, and discussing with the app on 127.0.0.1, and not use whatever the app framework provides to have a fast feedback loop.
↩︎