Repository navigation
Conversation
|
Hi, thanks for working on this. This PR is very similar in scope to my earlier PR #1463, which also implements asynchronous FTP/SFTP uploads with a persistent queue, retries and upload tracking. The main difference seems to be the architecture: #1603 uses a JSON queue processed through a background HTTP request, while #1463 uses SQLite and a dedicated Symfony/systemd worker. It may be worth comparing the approaches and seeing whether parts can be consolidated. |
|
Hi @flacoonb and sorry for not digging in the previous PRs before submitting mine. |
|
Hi @fmiccolis, no problem at all — it is easy to miss an older open PR, especially in a project with many ongoing contributions. I am glad you brought your approach forward, as the overlap gives us a good opportunity to compare both implementations and potentially combine the strongest parts. I chose SQLite because it provides a persistent transactional queue without requiring an external database server. It allows atomic job claiming, reliable recovery after crashes or restarts, and structured tracking of attempts, errors and filename mappings. The systemd worker keeps uploads fully independent from PHP and the web request lifecycle. This avoids request timeouts, session locks and occupied web-server workers during slow or unavailable FTP connections. It also gives automatic startup and restart on failure. The downside is additional installation complexity and a stronger dependency on Linux/systemd. Your HTTP-based approach is simpler and integrates more naturally into the existing Photobooth setup, so I can understand that choice. I agree that the remote-storage configuration should be simplified, and it may be useful to combine the strongest parts of both PRs. |
0f8afc8 to
c40084e
Compare
|
Thanks for the work on asynchronous uploads and for documenting the behavior changes and event testing. The JSON/flock approach avoids introducing SQLite or a dedicated systemd service, so the additional-dependency concern discussed on #1463 does not apply here. I am not ready to approve merging yet for the following reasons:
This is not a request to introduce SQLite or systemd. The immediate code blocker is the QR-link behavior with the default template; the remaining queue scenarios need focused validation before I can give merge approval. |
- Environment & network tab: one QR code per IPv4 address encoding the photobooth URL on that network, to open it from another device - New remote storage queue tab showing per-file upload status, attempts, errors and timestamps (works with the existing auto-refresh toggle) - Add missing label for the remote storage log tab Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The build workflow runs "npm run build", which sorts en.json with localeCompare and then requires a clean working tree. The new remoteUpload* keys were placed after "rembglog" but localeCompare orders them after "remoteStorageTemplate", so the build modified the file and the status check failed. it.json is aligned for consistency. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bundled gallery template ignores the photo query parameter, so the /?photo=<file> link opened the whole gallery instead of the picture. Keep the existing link; the deep link will be proposed separately together with template support.
51d76d5 to
5c8ff6c
Compare
|
I've reverted the change on In the end I think that all the settings regarding the FTP upload should be rewritten to give the power user more flexibility on the configuration. I will provide a new solution in another feature |
|
fyi @gambijul @fmiccolis @flacoonb @andi34 Thanks for your work on these improvements! We currently have three overlapping approaches to asynchronous FTP/SFTP uploads:
#1677 builds on #1603 and improves the settings, connection testing and gallery links. Would you agree with using #1603 as the foundation and bringing over the reliability improvements from #1674? In particular:
We should also add an admin retry button and migrate existing configurations when introducing the settings from #1677. For worker execution, I would prefer a detached CLI worker, so uploads do not occupy a webserver worker or depend on HTTP request timeouts. However, this requires PHP CLI and permission to start processes. What would you prefer: keep the HTTP-based approach from #1603 or adapt the detached CLI worker from #1674? I would leave SQLite and a permanent service out of this initial solution. Could we agree on a shared foundation and adapt the remaining improvements as follow-up PRs? |
… debug tab
Prerequisites checklist
What is the purpose of this pull request? (put an "x" next to an item)
What changes did you make? (Give an overview)
This PR makes the remote storage (FTP/SFTP) upload asynchronous, so the transition
from capture to the result screen is no longer blocked by a slow uplink, and adds
upload tracking plus two new admin debug panel tabs.
Asynchronous upload with tracking
applyEffects.phpno longer uploads synchronously: files are enqueued in a persistentJSON queue (
var/run/remotestorage_queue.json, newRemoteStorageQueueService) and theresponse returns immediately.
api/remoteStorageUpload.php, which drains the queue in the background:ignore_user_abort(), single-drainer lock (flock),session_write_close()(critical:without it the PHP session lock would freeze the booth UI during uploads), priority for
the just-captured photo, and a circuit breaker after 2 consecutive failures so an
unreachable server does not pile up ~90s FTP timeouts.
pending/uploading/done/failed), attempts and last error are persisted.Retries happen on later drains (max 3 attempts); failed uploads can be listed/retried/
cleared via the new admin-protected
api/remoteStorageAdmin.php.upload); a small spinner next to it turns into a check/error icon, driven by a cheap
status-polling mode of the same endpoint (same pattern as
gallery.php?status).deletePhoto.phpcancels queued uploads and skips remote round-trips for photos thatwere never uploaded.
createWebpage()fixes/improvementsconfig.php→config.inc.php): before this, the early-returncondition could never be true, so the webpage template + config were re-uploaded on
every single photo.
file) instead of doing remote
fileExistsround-trips per photo.ftp.template_config_locationoption (defaultprivate/template/config.inc.php):an optional PHP file returning an array that is merged over the generated
config.inc.phpdefaults — the counterpart of the already-customizabletemplate_locationfor people using a custom remote gallery template. Also applied tothe
test/remote-storage-template.phppreview.New debug panel tabs
CIDR network (via
net_get_interfaces()), Wi-Fi SSID (best-effort viaiwgetid), andone QR code per IPv4 address encoding the photobooth URL on that network — so an
operator can quickly open the booth (or SSH/VNC into it) from another device.
timestamps), compatible with the existing auto-refresh toggle.
Behavior changes to be aware of
keying.show_all = falseare no longer uploaded to remote storage(previously they were uploaded and then deleted locally; the remote gallery now mirrors
local visibility).
faileduntil retried via the admin endpoint —deliberately not retried on every capture so an offline server cannot slow down an event.
<gallery>/images/<file>). A?photo=<file>deeplink will be proposed in a separate PR, together with support for it in the bundled
gallery template.
Is there anything you'd like reviewers to focus on?
session_write_close()in the drain/status endpoints is load-bearing;please double-check I haven't missed a path that would hold the session while uploading.
a 1.5s linger re-check, a one-shot re-trigger from the status poller, and the next
capture's trigger), and stale
uploadingrecovery (entries reset after 10 min).set_time_limit(0)cannot override server-level timeouts — leftover entries are simplypicked up by the next drain.
Tested on a live installation during a real event (FTP over a slow uplink): the result
screen now appears immediately, the queue drains in the background, and the
failure/recovery paths were exercised by taking the FTP server offline mid-event.
AI used to create this Pull Request?
Yes. This PR was developed with Claude Code (Anthropic). The AI wrote the initial
implementation (queue service, endpoints, frontend changes, debug tabs) based on a design
we discussed and I decided on; I reviewed every change and tested the full flow on a real
installation at a live event (successful uploads, failure/retry paths, burst captures,
offline-server recovery).
npm run buildandnpm run eslintwere run (clean), and alltouched PHP files were syntax-checked.