Add QRP Labs QMX via the hamlib backend

docs/qmx-hamlib-status.md

@@ -0,0 +1,88 @@
+# QRP Labs QMX via the hamlib backend: status (PR #925)
+
+Running status for the work on branch `feat/qmx-hamlib-rig` (fork
+`laniol8926/FT8AF`), adding a "Qrp Labs QMX (Hamlib)" rig entry backed by
+hamlib's real `qrplabs_qmx_caps` instead of hand-rolled Kenwood emulation.
+Full narrative, with logs, is in the PR #925 comment thread; this is the
+condensed current state.
+
+## Confirmed working (real hardware, 2026-10-21)
+
+- `rigaddress.txt` entry (`Qrp Labs QMX (Hamlib),809,9600,26`) resolves to
+  hamlib's real `RIG_MODEL_QRPLABS_QMX` (`0x809` = 2057) and opens correctly.
+- CAT works end-to-end: frequency set from the app matches the radio's own
+  displayed frequency after a live change (tested at 7.074 MHz and
+  14.074 MHz).
+- Mode stays in Digital through a frequency change (previously it reverted to
+  USB every time).
+
+## Bugs found and fixed, in order
+
+1. **Wrong mode string produced an invalid CAT command** (`9b5e8787`, after a
+   wrong first attempt in `e046e0a7`/`37897125` that was reverted). hamlib's
+   QMX backend has no `mode_table` override, so `kenwood_set_mode()` falls
+   back to the generic Kenwood table, where `RIG_MODE_PKTUSB` sits at index
+   13 — encoded as the letter `'D'` (`"MDD;"`), which the QMX's own CAT
+   manual doesn't recognize (only single-digit `MD` values are valid; 6 =
+   FSK, the QMX's real digital mode). Fixed by requesting `"RTTY"` instead of
+   `"PKTUSB"` — `RIG_MODE_RTTY` sits at table index 6, producing the valid
+   `"MD6;"`.
+   - The reverted first attempt sent a vendor command believed to be
+     `"Q41;"` ("stay in digital mode") after every mode/frequency change.
+     Checked against QRP Labs' own official CAT manual
+     (`qrp-labs.com/images/qmx/manuals/cat_1_04_003.pdf`): there is no `Q41`
+     command. `Q4` is "Get/Set TX Rise Threshold" — `"Q41;"` was silently
+     setting it to 1%, an unrelated, actively harmful side effect.
+
+2. **Hamlib model ID silently truncated to a byte on every app restart**
+   (`47c8fc4e`). `GeneralVariables.civAddress` was reused for both a genuine
+   single-byte ICOM CI-V address and the hamlib model ID (routinely over a
+   byte — QMX's `2057`). Its persistence (`CivAddressConfig.encode()`) masks
+   with `& 0xFF`, so `2057` round-tripped through disk as `9` — opening an
+   unrelated hamlib backend (model 9, SDR#) instead of the QMX on every
+   subsequent launch. Fixed with a separate, unmasked `hamlibModelId`
+   field/config key.
+
+3. **That fix's migration gap crashed the app** (`4a76f2af`). An install that
+   had already selected a Hamlib rig before the new config key existed loaded
+   `hamlibModelId = 0`; passing model `0` to hamlib's `rig_init()` aborts the
+   whole process (native SIGABRT) about half a second later, not a graceful
+   failure. Fixed by refusing to open any `modelId <= 0` in
+   `HamlibRig.ensureOpen()`, plus a self-healing migration that derives the
+   value from the currently-selected rig on first connect.
+
+4. **The real structural bug: handshake replies were silently dropped**
+   (`d78aebe9`). `HamlibRig.onReceiveData()` only forwards radio bytes into
+   the native bridge once `handle != 0` — but the original single
+   `nativeOpen()` call only set `handle` *after* it returned, and it ran
+   hamlib's own `rig_open()` (including its synchronous CAT handshake)
+   *internally*, before returning. Every byte the radio sent back during that
+   handshake was dropped, so hamlib always timed out waiting for an ack that
+   could never arrive — confirmed on real hardware, where `serial.recv`
+   showed genuine traffic arriving in the exact window hamlib's own trace
+   reported "0 chars". Fixed by splitting the native call into
+   `nativeCreate()` (builds the bridge, returns a handle) and
+   `nativeRigOpen(handle)` (runs the handshake), with `HamlibRig.ensureOpen()`
+   publishing the handle in between.
+
+## Known issue, not yet root-caused
+
+TX: the rig keys/transmits, but audio is underdriven (low), with CAT and mode
+both confirmed working correctly in the same test. Not yet investigated —
+some earlier audio findings in the PR thread (the libusb I/O write failure,
+the "OS never registers a sound card" finding, the `BOTH`-vs-`LEFT` channel
+routing default) were drawn while CAT was *also* silently broken by bug #4
+above, so they may not all still apply as previously understood and are
+worth re-checking fresh now that CAT and mode are both reliable.
+
+## Known test gaps
+
+- No local build/unit-test execution has been possible on the Linux ARM
+  board this work was primarily developed on (`aapt2`/`adb`/`zipalign` are
+  all x86_64-only there). Real building and on-device testing for this status
+  update were done via a Windows laptop with Android Studio, connected over
+  Claude Code's Remote Control feature.
+- The Hamlib-backed code paths in `HamlibRigTest.java` are gated behind
+  `ensureOpen()`, which always returns false in a JVM unit test (no native
+  lib) — consistent with that test file's own documented precedent, hamlib
+  behavior itself is verified on real hardware, not in CI unit tests.

ft8af/app/src/main/assets/rigaddress.txt

@@ -75,3 +75,4 @@ UA3REO Wolf SDR(USB),00,4800,16
 (tr)uSDX (TS-480),00,9600,7
 Yaesu FT-891 (Hamlib),40C,38400,26
 ICOM IC-7300MK2,B6,115200,0
+Qrp Labs QMX (Hamlib),809,9600,26

ft8af/app/src/main/cpp/ft8af_glue/hamlib_jni.cpp

@@ -64,6 +64,7 @@ struct HamlibBridge {
     std::atomic<bool> running{false};
     jobject sink = nullptr;   // global ref to HamlibNative.HamlibSink
     jmethodID onBytesToRig = nullptr;
+    int model_id = 0;   // for logging only; set in nativeCreate, read in nativeRigOpen
 };
 
 // ---- pump thread: hamlib's outbound CAT bytes -> Java sink ------------------
@@ -78,6 +79,16 @@ static void *pump_main(void *arg) {
     while (b->running) {
         ssize_t n = read(b->srv_fd, buf, sizeof(buf));
         if (n > 0) {
+            // DIAGNOSTIC (temporary): confirm exactly what the pump thread
+            // reads from hamlib's loopback socket, and that it reads it at
+            // all — suspected gap between hamlib's successful write_block()
+            // and CAT bytes (e.g. "FA;") reaching the real serial port.
+            char hex[512 * 3 + 1];
+            int hoff = 0;
+            for (ssize_t i = 0; i < n && hoff < (int) sizeof(hex) - 4; ++i) {
+                hoff += snprintf(hex + hoff, sizeof(hex) - hoff, "%02X ", buf[i]);
+            }
+            LOGI("pump: read %zd bytes from srv_fd: %s", n, hex);
             jbyteArray arr = env->NewByteArray((jsize) n);
             if (arr == nullptr) { break; }
             env->SetByteArrayRegion(arr, 0, (jsize) n, reinterpret_cast<jbyte *>(buf));
@@ -118,12 +129,23 @@ static HamlibBridge *handle_to_bridge(jlong h) {
 extern "C" {
 
 /*
- * nativeOpen: build the bridge, init+open the rig on the loopback socket.
- * Returns an opaque handle (>0) or 0 on failure.
+ * nativeCreate: build the bridge and start the pump thread, but do NOT call
+ * rig_open() yet. Returns an opaque handle (>0) or 0 on failure.
+ *
+ * Split from the open call (see nativeRigOpen below) so Java can publish the
+ * handle to HamlibRig.handle BEFORE rig_open()'s own synchronous handshake
+ * runs. HamlibRig.onReceiveData() only forwards radio bytes into the bridge
+ * when handle != 0 — with the original single nativeOpen(), handle was only
+ * set after it returned, so every byte the radio sent during rig_open()'s own
+ * handshake (its ID;/FA; probes) was silently dropped on the Java side, and
+ * hamlib always saw 0 chars and timed out, even against a radio that was
+ * actually replying. Confirmed on real hardware: serial.recv showed real
+ * traffic arriving while hamlib's own trace reported "Timed out ... after 0
+ * chars" for the exact same window, every single time.
  */
 JNIEXPORT jlong JNICALL
-Java_com_k1af_ft8af_wave_HamlibNative_nativeOpen(JNIEnv *env, jclass,
-                                                 jint modelId, jobject sink) {
+Java_com_k1af_ft8af_wave_HamlibNative_nativeCreate(JNIEnv *env, jclass,
+                                                    jint modelId, jobject sink) {
     ensure_vm(env);
     auto *b = new HamlibBridge();
 
@@ -173,8 +195,29 @@ Java_com_k1af_ft8af_wave_HamlibNative_nativeOpen(JNIEnv *env, jclass,
         close(b->listen_fd); delete b; return 0;
     }
 
-    // 5. open (connects to our listener; initial CAT handshake flows over the
-    //    bridge to the real radio via Java)
+    b->model_id = modelId;
+    LOGI("hamlib bridge for model %d created on 127.0.0.1:%d (not yet opened)",
+         modelId, port);
+    return reinterpret_cast<jlong>(b);
+}
+
+/*
+ * nativeRigOpen: run rig_open() on an already-created bridge (see
+ * nativeCreate above for why this is split out). Java must publish the
+ * handle to HamlibRig.handle before calling this, so nativeFeedFromRig can
+ * deliver bytes the radio sends during rig_open()'s own handshake.
+ *
+ * Returns RIG_OK (0) on success, or a (negative) hamlib error code. On
+ * failure the bridge is fully torn down and freed here, same as the
+ * original single-call nativeOpen's failure path — the handle must not be
+ * used again by the caller after a non-zero return.
+ */
+JNIEXPORT jint JNICALL
+Java_com_k1af_ft8af_wave_HamlibNative_nativeRigOpen(JNIEnv *env, jclass,
+                                                     jlong handle) {
+    auto *b = handle_to_bridge(handle);
+    if (b == nullptr) return -1;
+
     int rc = rig_open(b->rig);
     if (rc != RIG_OK) {
         LOGE("rig_open failed: %d (%s)", rc, rigerror(rc));
@@ -193,10 +236,10 @@ Java_com_k1af_ft8af_wave_HamlibNative_nativeOpen(JNIEnv *env, jclass,
         sfd = b->srv_fd.load();
         if (sfd >= 0) close(sfd);
         delete b;
-        return 0;
+        return rc;
     }
-    LOGI("hamlib rig %d opened via 127.0.0.1:%d", modelId, port);
-    return reinterpret_cast<jlong>(b);
+    LOGI("hamlib rig %d opened", b->model_id);
+    return RIG_OK;
 }
 
 /* Feed bytes received from the real radio into hamlib's read side. */
@@ -273,7 +316,7 @@ Java_com_k1af_ft8af_wave_HamlibNative_nativeClose(JNIEnv *env, jclass, jlong han
     b->running = false;
     // Unblock the pump whether it is parked in accept() (never connected) or
     // read(): close the listener first, then shut down the accepted socket.
-    // Joining before this could deadlock. See nativeOpen's failure path.
+    // Joining before this could deadlock. See nativeRigOpen's failure path.
     int lfd = b->listen_fd.exchange(-1);
     if (lfd >= 0) { shutdown(lfd, SHUT_RDWR); close(lfd); }
     int sfd = b->srv_fd.load();

ft8af/app/src/main/java/com/k1af/ft8af/GeneralVariables.java

@@ -545,6 +545,16 @@ public static Context getMainContext() {
     public static int iaruRegion = 2;//Operator's IARU region (1/2/3), used to gate Message-Creator QSY frequency options to legal band edges. Default 2 (the Americas). See com.k1af.ft8af.message.SpecialMessage.
     public static boolean autoCQAfterQSO = false;//Auto-CQ: keep calling CQ after each completed QSO (chain without re-tapping). Refreshes the TX watchdog per QSO and forces pure CQ (ignores Hunt).
     public static int civAddress = 0xa4;//CI-V address
+    /**
+     * hamlib model ID for Hamlib-backed rigs (instructionSet == HAMLIB), e.g.
+     * 2057 for Qrp Labs QMX. Deliberately separate from {@link #civAddress}:
+     * that field (and its {@code CivAddressConfig} persistence) is a genuine
+     * single-byte ICOM CI-V address (0x00-0xFF); a hamlib model ID routinely
+     * exceeds a byte (2057 = 0x809), and persisting it through civAddress's
+     * byte-masked encode/decode silently truncated it (2057 -> 9, opening the
+     * wrong hamlib backend entirely on the next app launch).
+     */
+    public static int hamlibModelId = 0;//hamlib model ID, see rigaddress.txt's address column for HAMLIB rigs
     public static String civAddressStored = null;//Raw "civ" config text as hydrated (null = no row); lets the #753 repair see whether the on-disk form is canonical hex.
     public static boolean civAddressFormatKnown = false;//True when config "civFormat=hex" was present: the stored civ value is trusted verbatim, no model reconciliation (#753).
     public static int baudRate = 19200;//Baud rate

ft8af/app/src/main/java/com/k1af/ft8af/MainViewModel.java

@@ -2438,8 +2438,31 @@ private void connectRig() {
                 break;
             case InstructionSet.HAMLIB:
                 // hamlib model number is carried in the rig table's address column
-                // (parsed base-16), e.g. FT-891 = 1036 = 0x40C.
-                baseRig = new HamlibRig(GeneralVariables.civAddress);
+                // (parsed base-16), e.g. FT-891 = 1036 = 0x40C. Uses hamlibModelId,
+                // not civAddress -- civAddress's persistence is byte-masked (a real
+                // ICOM CI-V address is a byte; a hamlib model ID routinely isn't),
+                // which silently truncated it (2057 -> 9) on every app restart.
+                if (GeneralVariables.hamlibModelId <= 0) {
+                    // One-time migration for an install that selected a Hamlib rig
+                    // before the "hamlibModel" config key existed (or any other way
+                    // it ended up unset) -- self-heal from the still-correct selected
+                    // rig entry rather than leaving it at the unset default, which
+                    // HamlibRig.ensureOpen() now refuses to open at all (a real
+                    // model-0 hand-off previously aborted the whole process).
+                    android.content.Context ctx = GeneralVariables.getMainContext();
+                    if (ctx != null) {
+                        RigNameList.RigName model = RigNameList.getInstance(ctx)
+                                .getRigNameByIndex(GeneralVariables.modelNo);
+                        if (model != null && model.address > 0) {
+                            GeneralVariables.hamlibModelId = model.address;
+                            if (databaseOpr != null) {
+                                databaseOpr.writeConfig("hamlibModel",
+                                        String.valueOf(model.address), null);
+                            }
+                        }
+                    }
+                }
+                baseRig = new HamlibRig(GeneralVariables.hamlibModelId);
                 break;
         }
 

ft8af/app/src/main/java/com/k1af/ft8af/connector/CableSerialPort.java

@@ -380,6 +380,14 @@ public boolean connect() {
             usbIoManager = new SerialInputOutputManager(usbSerialPort, new SerialInputOutputManager.Listener() {
                 @Override
                 public void onNewData(byte[] data) {
+                    // DIAGNOSTIC (temporary): CableSerialPort had no logging at
+                    // all on the receive path, so "the radio never replied" was
+                    // never actually verified -- it could equally be a relay bug
+                    // further downstream. This logs every byte actually read off
+                    // the wire, before it reaches any rig-specific listener.
+                    StringBuilder hex = new StringBuilder();
+                    for (byte x : data) hex.append(String.format("%02X ", x));
+                    fileLog("serial.recv[" + data.length + "]: hex: " + hex.toString().trim());
                     if (ioListener != null) {
                         ioListener.onNewData(data);
                     }

ft8af/app/src/main/java/com/k1af/ft8af/database/DatabaseOpr.java

@@ -2772,6 +2772,11 @@ protected Void doInBackground(Void... voids) {
                             .decode(result, com.k1af.ft8af.rigs.CivAddressConfig.DEFAULT_ADDRESS);
                     GeneralVariables.civAddressStored = result;
                 }
+                if (name.equalsIgnoreCase("hamlibModel")) {
+                    // Plain decimal, not hex, and not byte-masked -- see civAddress's
+                    // javadoc in GeneralVariables for why this needs its own key.
+                    GeneralVariables.hamlibModelId = parseConfigInt(result, 0);
+                }
                 if (name.equalsIgnoreCase(com.k1af.ft8af.rigs.CivAddressConfig.FORMAT_KEY)) {
                     // Provenance marker: present only when a hex-aware writer stored "civ".
                     GeneralVariables.civAddressFormatKnown =

ft8af/app/src/main/java/com/k1af/ft8af/rigs/HamlibRig.java

@@ -27,8 +27,20 @@
 public class HamlibRig extends BaseRig {
     private static final String TAG = "HamlibRig";
 
-    /** hamlib mode for FT8/FT4: DATA-USB, so audio routes through the data path. */
-    private static final String FT_MODE = "PKTUSB";
+    /**
+     * hamlib mode string for FT8/FT4. NOT "PKTUSB": hamlib's generic Kenwood
+     * mode table (rigs/kenwood/kenwood.c's kenwood_mode_table, which the QMX
+     * backend falls back to since it sets no override) maps RIG_MODE_PKTUSB
+     * to table index 13, which kenwood_set_mode() then encodes as the ASCII
+     * character 'D' ('A' + 13 - 10) -- producing "MDD;". The QMX's own CAT
+     * manual (qrp-labs.com/images/qmx/manuals/cat_1_04_003.pdf) only accepts
+     * single-digit MD values (1,2,3,5,6,7,8,9); 'D' is not one of them and is
+     * silently rejected, so the radio's mode was never actually being set at
+     * all via this path. Table index 6 holds RIG_MODE_RTTY ("FSK Mode" per
+     * that table's own comment) -- exactly the QMX manual's own "6 (FSK)",
+     * its real digital mode. "RTTY" therefore produces the valid "MD6;".
+     */
+    private static final String FT_MODE = "RTTY";
 
     private final int modelId;
     private final ExecutorService worker = Executors.newSingleThreadExecutor();
@@ -47,6 +59,13 @@ public class HamlibRig extends BaseRig {
         BaseRigConnector c = getConnector();
         if (c != null) {
             c.sendData(data);
+        } else {
+            // DIAGNOSTIC (temporary): hamlib's CAT bytes are silently dropped
+            // here with no trace anywhere else in the app or native logs when
+            // the connector is null — suspected cause of commands beyond the
+            // initial rig_open() handshake (e.g. FA;) never reaching the radio.
+            Log.w(TAG, "sink.onBytesToRig: dropping " + data.length
+                    + " byte(s), getConnector() is null");
         }
     };
 
@@ -75,12 +94,38 @@ private synchronized boolean ensureOpen() {
         if (c == null || !c.isConnected() || !HamlibNative.isAvailable()) {
             return false;
         }
-        long h = HamlibNative.nativeOpen(modelId, sink);
+        if (modelId <= 0) {
+            // Never hand hamlib a placeholder/unset model ID: rig_init(0) was
+            // observed on real hardware to abort the whole process (native
+            // SIGABRT, "pthread_mutex_lock called on a destroyed mutex") a
+            // fraction of a second later, not just fail gracefully. 0 means
+            // "not yet persisted" (see GeneralVariables.hamlibModelId) and must
+            // never reach native code.
+            Log.w(TAG, "ensureOpen: refusing to open invalid hamlib model " + modelId);
+            return false;
+        }
+        long h = HamlibNative.nativeCreate(modelId, sink);
         if (h == 0) {
-            Log.w(TAG, "nativeOpen failed for hamlib model " + modelId);
+            Log.w(TAG, "nativeCreate failed for hamlib model " + modelId);
             return false;
         }
+        // Publish the handle BEFORE running the handshake: onReceiveData()
+        // only forwards radio bytes into the bridge once handle != 0, and
+        // rig_open() (called by nativeRigOpen below) does its own synchronous
+        // CAT handshake that needs those bytes fed back in real time. With a
+        // single combined open call, every byte the radio sent during that
+        // handshake was silently dropped (handle was still 0), so hamlib
+        // always saw a timeout — confirmed on real hardware: the radio was
+        // genuinely replying (serial.recv had real traffic) while hamlib's
+        // own trace reported "0 chars" for that exact window, every time.
         handle = h;
+        int rc = HamlibNative.nativeRigOpen(h);
+        if (rc != 0) {
+            Log.w(TAG, "nativeRigOpen failed for hamlib model " + modelId + " rc=" + rc);
+            // nativeRigOpen already tore down and freed the bridge on failure.
+            handle = 0;
+            return false;
+        }
         Log.i(TAG, "hamlib model " + modelId + " opened");
         return true;
     }