Skip to content

Major Kotatsu-Redo port: tracker, reader, Cloudflare, library, sources and more - #137

Draft
PoonHarpoon wants to merge 27 commits into
AppFuton:develfrom
PoonHarpoon:pr/port
Draft

PoonHarpoon wants to merge 27 commits into
AppFuton:develfrom
PoonHarpoon:pr/port

Conversation

@PoonHarpoon

Copy link
Copy Markdown
Collaborator

This PR ports over several QoL Kotatsu-Redo features and fixes that were also missing in Futon. Below is a summary of what is contained; most features and fixes can be stripped out.

Downloads, backups and local storage

Five small, independent fixes for downloads, backups and local storage. The temp-file, decimal-settings and local-index fixes are ported from Kotatsu-Redo.

Downloads: queued chapters were silently dropped. Each manga's download is enqueued as unique work with KEEP, and the scheduler only waited while that work was RUNNING. If an earlier download of the same manga was still ENQUEUED or BLOCKED (for example, waiting for charging or an unmetered network), the next queue entry was discarded by KEEP and then removed from the queue, so its chapters were lost. Such entries now stay queued until the earlier download finishes, and an entry is removed only once its work is stored.

Downloads: concurrent downloads deleted each other's temp files. Two downloads to the same storage directory cleaned up each other's temp files. Each download now only removes its own.

Backup: restoring sources added to the current selection instead of replacing it. Keeps the welcome logic from interfering with restoring sources. Covered by a new case in AppBackupAgentTest.

Backup: decimal settings failed to load after restoring a JSON backup. JSON backups bring decimal values back as Double, which putAll skipped, and a value stored with the wrong type made getFloat throw. putAll now stores doubles as floats, and the decimal reader settings (double-page sensitivity, brightness/contrast, autoscroll speed and others) read through a helper that recovers a wrongly-typed value.

Local storage: the index rebuild blocked the database on cold start. The storage scan ran inside the index rebuild transaction and held Room's write connection the whole time. The scan now runs first, and the transaction only writes the results.

Suggestions and related manga

Ported from Kotatsu-Redo, local parts only, with a few changes noted below.

Fixes

  • The suggestions worker kept the least relevant suggestions. It sorted ascending by relevance, so it kept the wrong end. The notification picker also crashed with fewer than 3 suggestions (Random.nextInt(0, 0)).
  • Related manga ignored "disable NSFW". NSFW titles now stay hidden when NSFW content is disabled.

Smarter suggestions

  • Seed tags are now the reader's most telling tags, weighted by how rare each tag is across the local library (TF-IDF), rather than simply the most frequent ones. Blacklisted tags are never used as seeds.
  • Dropped manga (barely read, untouched for 3 weeks, not a favourite) no longer feed suggestions.
  • A source is never searched without a tag (that would return its whole catalogue). Two matched tags are searched together where the source supports it, falling back to one.
  • Rating sort is preferred over popularity.
  • Unlike Redo, suggestions stay off by default, so existing users who never opted in don't get a new background worker hitting every enabled source.

Related manga fallback
When a source links fewer than four related titles, the row is filled from the reader's stored suggestions that share this manga's tags, weighted by rarity. If that's still short, it's filled from one search of the manga's own source for its rarest tags. NSFW and the tag blacklist are respected. Extension sources skip the source search, because their shared repository has stateful paging. The related-list screen uses the same logic. Redo's community-server part is left out.

Covered by the new SuggestionRankingTest.

Webtoon reader

Three webtoon reader fixes, ported from Kotatsu-Redo.

Broken page sizing. Pages could collapse to zero height, render hugely zoomed in (infinite scale), or get cut off.

Pages cut off at page joins. A scroll now walks every page it reaches instead of only the first two, so a fast fling over short pages can no longer carry a tall page past the edge of the screen with its image unscrolled. Each page's inner scroll is re-pinned after every layout (decode, zoom, rotation). A page that finishes decoding after it has already scrolled partly off screen is placed without jumping to its end or moving what's on screen.

Pull-to-switch-chapter gesture rewritten. The EdgeEffect-based pull is replaced by a touch tracker that only starts a pull when both the list and the edge page's image are really at their first or last edge. The edge glow is turned off while the gesture is enabled. The feedback chip fades in from 0.15 to full opacity, switches to an outlined background once the threshold is reached, and hides as soon as you let go.

Tracker

Several fixes to how chapter progress is saved, to protect progress from being lost, and to how new chapters are fetched.

Counting fixes

  • Tracks reset or stuck. Tracks shared by history and favourites no longer have their baseline reset on every run, and sources with update checks disabled no longer hold batch slots forever.
  • New chapters not credited. Opening the reader on an older chapter moved the tracked chapter to the newest one without counting what had been released since. Those chapters now show up in Updates.
  • Sources that change chapter URLs. When the tracked chapter is gone from the fresh list, the comparison falls back to its upload date, then to the history chapter, before re-baselining. A track's first check anchors on the history chapter.
  • Read and old chapters reported as new. Chapters at or before the reading position (including the same title read on another source), or uploaded more than 90 days ago, are no longer reported.
  • Long-inactive tracks. Tracks with no new chapter, read or favourite in 90 days drop out of regular checks but are still checked weekly. Stale unread counters are cleared.
  • Clearing updates now resets each track to the history chapter, so unread chapters are recomputed from the reading position.
  • Manual checks check every track even when the system refuses a foreground service.
  • One-time recovery for tracks that were already advanced past unread chapters. It runs on first launch with network, needs no schema change, and is gated by two preference flags.

Throughput and save fixes (Not a port: New feat/fix)

  • Per-source limit. At most 2 checks run per source, within the existing 6. The source slot is taken before the global one, so a busy source can't hold global slots while it waits.
  • Blocked sources. After a source's first Cloudflare or rate-limit (429) failure, its remaining manga in the run aren't requested. They're stamped as checked in one update, so a blocked source can't fill the next batches either.
  • No network inside the save transaction. Progress refresh could fetch pages while holding the save transaction open. It now runs after the save, and a failed refresh no longer fails a check that was already saved.
  • The last-chapter date now comes from the tracked branch rather than from all branches.

Re-check after a captcha solve (Not a port: New feat/fix)

  • Captcha failures are stored as a new result, RESULT_CAPTCHA (5). last_result is a plain int, so no migration is needed.
  • Each successful solve queues one re-check of that source's captcha-blocked tracks. Solves come from the captcha notification, or from the automatic solver if that lands. It uses the existing CaptchaHandler.discard(), so it doesn't depend on any other PR.
  • It can't loop: a re-check that hits the captcha again only shows the notification.
  • New setting: "Re-check after solving a CAPTCHA", on by default.

New Features Ported

  • Swipe to remove a feed item, with undo.
  • New "Recently read manga" download strategy: new chapters of manga read in the last 30 days are downloaded, up to 5 per manga per run. Nothing is queued with less than 2 GB free or when the download quota is reached. Tracker downloads now go through the download queue, so the Wi-Fi, charging and off-peak settings apply to them. (Modified: in Redo the 5-chapter cap also hit the existing "Downloaded manga" strategy, so chapters over the cap were never downloaded; here only the new strategy is capped.)

Cloudflare

Fixes to how the app gets through Cloudflare, plus an experimental opt-in automatic captcha solver. Most of this is ported from Kotatsu-Redo.

Fixes

  • Request interception broke after a Cloudflare interstitial. The page script was injected only once, so captures timed out when a challenge page came before the real one. It's now re-injected on every navigation and polled while capturing. Captures are serialised and reuse a single WebView, and evaluation WebViews are now destroyed after use, so they no longer leak.
  • The clearance cookie didn't match the User-Agent. If the solver WebView had a non-Android User-Agent, Turnstile could fail. The UA is now realigned with the engine's default, and the one that earned clearance is stored per source, so OkHttp sends the agent the cf_clearance cookie is bound to.
  • The user's own User-Agent setting wins (Modified). If the user changes the source's User-Agent setting, the stored clearance agent is ignored. In Redo the stored agent was never cleared and silently overrode the user's setting.

Automatic captcha solving (experimental, off by default)

  • New global switch under Storage and network: "Solve CAPTCHAs automatically (experimental)". It's off by default, and there's also a per-source opt-out.
  • When it's on, opening a catalog, a manga or a chapter first tries to pass the challenge silently, in a translucent WebView activity. A read-only page-state probe decides when the real page has loaded.
  • Only one session runs at a time. Cooldowns are 30 s after a success and 3 min after a failure. It only starts while the app is in the foreground and is cancelled when it goes to the background. Background workers, prefetch and image loading never start a solve.
  • The headless WebViewExecutor.tryResolveCaptcha path and its two WebView clients are removed, because CaptchaHandler was their only user. AdBlock is kept.
  • It's marked experimental because it can't solve interactive challenges, and every challenge I got while testing was interactive. It should pass non-interactive ones, but I haven't been able to confirm that.

Library

  • Progress badges no longer show a leading "-" on "percent left" and "chapters left".
  • Hide NSFW from launcher shortcuts: a new setting, on by default. NSFW titles are also left out of shortcuts whenever NSFW content is disabled.
  • Pin favourites. Pin or unpin from the favourites selection mode. Pinned entries sort first under every sort order and show a pin badge in grid and detailed list modes. A manga pinned in any list counts as pinned in the all-favourites view, and pinning from that view pins it in every list it's in. It uses the existing favourites.pinned column, so no migration is needed.
  • Quick filters for favourites: Manga / Manhwa / Manhua chips, plus the three most common tags of the current list. Tags are matched by title, so the same genre from different sources is one chip, and blacklisted tags aren't offered. Mihon sources report everything as plain manga, so their content type comes from a manga/manhwa/manhua genre tag. A Mihon title without one is only listed when no content-type chip is selected, instead of being misfiled under "Manga".
  • Search within History and Favourites. On the History and Favourites screens, the search bar can now search just that screen. An option in the search bar switches between normal search and "History" or "Favourites". In screen mode, typing filters the list you're looking at instead of opening a new search. It matches the manga's title (including alternative titles), its source, and for favourites, the list it's in. While the filter is active it stays visible in the search bar with a button to clear it, and it's cleared automatically when you leave the screen.
    • Changes from Redo: it also works with Mihon sources, the search text is passed to the database safely rather than pasted into the query, and the Downloaded view is filtered the same way.

Covered by the new ContentTypeConditionTest and LibrarySearchTest.

Sources and alternatives

  • "Content unavailable" dialog. When a manga's source says it's gone (404 or removed), the app now says so and offers a shortcut to search for it on other sources. This also works with Mihon extension sources.
  • Source presets. Save named sets of languages and sources, and switch between them from Explore. While a preset is active, Explore and global search use only its sources, and adding or removing sources in the catalog edits the preset rather than your enabled sources. Uninstalled sources and deleted presets are handled cleanly. The active preset survives a settings backup and restore.
    • This adds a new database table, source_presets (migration 30 → 31). Note for sync: requests will now send X-Db-Version: 31.
  • Replace source (Modified). "Fix" in the favourites and history selection menus becomes "Replace source", and it's also in the manga details menu. It checks every candidate source against your library titles from the old source, lets you pick the best one, and moves the matches over automatically. Unlike Redo, it asks before replacing every matched title. Anything it can't match, you choose by hand. Each title is migrated on its own, so one failure doesn't stop the rest.
  • Search options for alternatives. The alternatives screen gets options for:
    • which sources to search (enabled, disabled, or a preset)
    • same language only
    • same content type only
    • hiding results without chapters
    • four sort orders
    • an editable search query
    • a progress counter
  • Covers from other sources. The manga edit screen can pick a cover from other sources. Duplicate covers are filtered out, and the chosen cover is saved locally.

I did not port Redo's community ranking system.

JPEG XL

Ported from Kotatsu-Redo.

  • JPEG XL images now display everywhere: covers, thumbnails, previews and reader pages. They're detected by their content as well as by the .jxl extension.
  • If the JPEG XL library fails to load on a device, only JPEG XL images fail; nothing else is affected.
  • New dependency: io.github.awxkee:jxl-coder 2.6.1. It bundles native libraries for each CPU type and adds about 11 MB to the debug APK (48.7 → 59.8 MB). It's a large dependency for a niche feature, so it's up to you whether it's worth adding.

Reader and history

  • Reading position kept when a source changes chapter IDs. Some sources give chapters new IDs between loads, which used to lose your place. The history now follows the same chapter in the new list (matched by URL, then volume and number). If it can't be matched, the position is estimated within the branch you were reading, not across all branches. If your history chapter is gone entirely, progress is rescaled to the new chapter count instead of being dropped; that estimate only ever lowers progress, and never marks a manga as completed.
  • Default branch re-picked once the fresh chapter list loads, not only from the cached one, so a recovered position opens in the right branch.
  • Smart resume fix (Not a port: New feat/fix). Continue reopened the finished chapter after a new release instead of moving on, because the progress recalculation overwrote the chapter count that smart resume relies on. The count is left to the reader again.
  • Double-page rework.
    • Two chapters never share a spread.
    • New option "First page as cover (solo)" shows each chapter's first page on its own.
    • Pinch-zoom and pan now work on the whole spread instead of one page at a time.
    • Pages are centred against each other instead of the screen edges.
    • Wide images span the full spread.
    • Right-to-left mode gets identical spreads.
  • E-Ink flash on page change. An optional full-screen white or black flash on page change clears ghosting on E-Ink screens. Duration and frequency are configurable. It doesn't flash when opening a manga, switching reader mode, or during webtoon autoscroll.

Overlaps with #118 in DoubleReaderFragment and ReversedDoubleReaderFragment; I'll rebase whichever lands second.

Testing

Unit tests pass (157, 0 failures) and the debug build succeeds. AppBackupAgentTest and ProgressUpdateRaceTest are instrumented tests and weren't run.

I tested all of these together and confirmed they work on my device (S26). Automatic captcha solving is the exception: it couldn't be verified, since every challenge I got was interactive.

@LandWarderer2772

Copy link
Copy Markdown
Member

feels the aura
holy moly this is gonna be a pain to review

@LandWarderer2772

Copy link
Copy Markdown
Member

brother u on discord?

@PoonHarpoon

Copy link
Copy Markdown
Collaborator Author

I have an old account but I do.

@PoonHarpoon

Copy link
Copy Markdown
Collaborator Author

Most of this is 99% the same code as Kotatsu-Redo but some more than others needed work to get them, well, working.

@LandWarderer2772

Copy link
Copy Markdown
Member

Most of this is 99% the same code as Kotatsu-Redo but some more than others needed work to get them, well, working.

promoting u to maintainer cause i need help 🙏🏻

@PoonHarpoon

Copy link
Copy Markdown
Collaborator Author

These issues are probably fixed by this PR:

#124 is partly covered: JPEG XL images now load, but there's no conversion feature.

@PoonHarpoon

Copy link
Copy Markdown
Collaborator Author

Currently fixing some regressions with this

@LandWarderer2772

Copy link
Copy Markdown
Member

Currently fixing some regressions with this

got it

fixes #125
fixes #27
fixes #102

…ownloads' temp files

* fix(downloads): only clean up a download's own temp files

Concurrent downloads to the same storage dir deleted each other's temp files.
Ported from Kotatsu-Redo ddc4be1.

* fix(downloads): stop the queue silently dropping chapters

Queued downloads are enqueued as unique work per manga with KEEP, and the
scheduler only waited while a download was RUNNING. A second queue entry
for a manga whose earlier download was still ENQUEUED or BLOCKED (e.g.
waiting for charging or an unmetered network) was discarded by KEEP and
then removed from the queue, losing its chapters.

Leave such entries in the queue until the earlier download finishes, and
only remove an entry after its work has been stored.
A backup only holds the enabled sources, and restoring them was a plain
upsert, so every source already enabled stayed on. On a fresh install
that is the welcome sheet's language defaults (or the 15 fallback
defaults), so the restored selection was buried among them.

Unless merging, disable all sources before restoring the backed-up ones.
Holding the write connection during the scan stalled Room on cold start.
Ported from Kotatsu-Redo 9.8.5.
The worker sorted ascending and kept the least relevant suggestions. The
notification picker also crashed with fewer than 3 suggestions
(Random.nextInt(0, 0)).

Ported from Kotatsu-Redo 9.8.5.
…ed manga

* feat(suggestions): rank local suggestion tags by rarity

- Skip dropped manga (barely read, untouched for 3 weeks, not a
  favourite) when building the seed.
- Pick the reader's most telling tags by TF-IDF against tag frequency
  across the local database (TagStatistics, TagsDao.countMangaPerTag)
  instead of the most frequent ones; blacklisted tags are not seeds.
- Never search a source with no tag (its whole catalogue); search two
  matched tags together when supported, falling back to one.
- Prefer rating over popularity sort order.
- Suggestions are now enabled by default.

Ported from Kotatsu-Redo 9.8.5 (9fb36ac), local parts only.

* fix(suggestions): keep suggestions off by default

Redo turned them on by default, which would start a background worker
hitting every enabled source for existing users who never opted in.

* feat(details): fall back to tag matching for related manga

When the source links fewer than four related titles, fill the row from
the reader's stored suggestions that share this manga's tags (weighted by
tag rarity), then from one search of the manga's source for its rarest
tags matched against the source's filter tags. NSFW and the tag blacklist
are respected; the source tag search is skipped for extension sources,
whose shared repository has stateful paging. The related list screen
uses the same use case.

Ported from Kotatsu-Redo 9.8.5 (9fb36ac), without the community server.
…hapter

* fix(webtoon): fix zero-height, infinite-scale and cut-off pages

Ported from Kotatsu-Redo 9.8.3/9.8.5.

* fix(webtoon): rewrite pull-to-switch-chapter gesture

Replace the EdgeEffect-based pull effect with a touch tracker in
dispatchTouchEvent that only starts a pull when the list and the edge
page's inner image scroll are really at the first/last edge. The edge
glow is disabled while the gesture is enabled. The feedback chip now
fades from 0.15 to full alpha and switches to an outlined background
once the threshold is reached, and hides immediately on release.

Ported from Kotatsu-Redo 45455ee.

* fix(webtoon): stop pages being cut off at page joins

Walk every page a scroll reaches instead of only the first two, so a
fast fling over short pages can no longer carry a tall page past the
viewport edge with its image unscrolled. Re-pin all pages' inner scroll
after every layout (decode, zoom, rotation), and place a page that
finishes decoding after the list has already scrolled it partly off
screen without jumping to its end or moving what is on screen.

Ported from Kotatsu-Redo 9.8.5 (9fb36ac).
* fix(tracker): stop resetting tracks shared by history and favourites, page past update-disabled sources

Ported from Kotatsu-Redo 9.8.0/9.8.1.

* fix(tracker): credit chapters released since the last tracked chapter

Opening the reader on an older chapter advanced last_chapter_id to the
newest chapter without counting chapters released since the previous
check, so they never showed up in Updates. Also cap the stored new
chapters counter at the branch chapter count.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23).

* fix(tracker): fall back to date and history anchors when comparing

When the tracked last chapter id is no longer in the fresh chapter list
(sources rotating chapter urls), compare by the last chapter upload date,
then by the history chapter, and only then re-baseline silently. The
first check of a track anchors on the history chapter instead of
swallowing chapters released since the user last read.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23) / 9.7.8 (f34b597).

* fix(tracker): drop already read and stale chapters from updates

Chapters at or before the reading position (also taking similar-titled
manga read from another source into account) or uploaded more than 90
days ago are not reported as new. Adds HistoryRepository.findSimilarByTitle.

Ported from Kotatsu-Redo 9.7.8 (f34b597) / 9.8.1 (cb430d3).

* fix(tracker): skip long inactive tracks and clear stale counters

Tracks without a chapter, read or favourite in the last 90 days are left
out of regular checks but still re-checked weekly, so a hiatus series is
not excluded forever. Unread counters of tracks whose last chapter is
older than 90 days are cleared on gc.

Ported from Kotatsu-Redo 9.7.8 (f34b597) / 9.8.0 (b1483eb).

* fix(tracker): rebase tracks on the history chapter when clearing counters

Clearing all updates now also resets last_chapter_id to the history
chapter (or 0 when there is none), so the tracker recomputes unread
chapters from the reading position.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23).

* fix(tracker): treat manual checks as full runs without foreground service

A manually triggered one-shot check now checks every track even when the
system refuses to promote the worker to a foreground service.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23).

* fix(tracker): one-shot migration for tracks advanced past unread chapters

The reader-side new chapters check used to move last_chapter_id to the
newest chapter without crediting chapters the user had not read. On first
start with network, credit chapters after the history chapter for tracks
with no new chapters whose history differs from last_chapter_id, then
refresh history chapter counts of tracks with new chapters so they no
longer show as completed. Gated by two SharedPreferences flags; no DB
schema change.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23) / 9.8.0 (b1483eb).
* feat(tracker): auto-download new chapters of recently read manga

Adds a "Recently read manga" tracker download strategy: new chapters of
manga read within the last 30 days are downloaded whether or not
anything is saved locally. For both strategies, at most the 5 newest
not yet downloaded chapters are taken per manga per run, and nothing is
queued when the target storage has less than 2 GB free or the download
storage quota is already reached (the worker would otherwise evict
older chapters).

Tracker downloads now go through the download queue instead of straight
to the download worker, so the Wi-Fi, charging and off-peak settings
apply to them as to smart downloads.

Ported from Kotatsu-Redo 9.8.3 (77d993d).

* fix(tracker): keep downloading every new chapter of downloaded manga

The 5-chapter cap from the recently-read strategy also hit the existing
downloaded-manga strategy, and chapters over the cap were never offered
again. Only cap the new recently-read strategy.
* fix(tracker): refresh progress outside the save transaction, date the tracked branch

saveUpdates recalculated reading progress inside its database
transaction, and that can load the chapter pages from the network: a
slow source kept the transaction open for the whole request. The track
and its feed entry are now saved first and the progress refreshed
after; a failed refresh no longer fails a check that was already saved.

When no new chapter has an upload date, lastChapterDate() fell back to
the last chapter across every branch instead of the tracked one.

* perf(tracker): limit per-source checks and stop at blocked sources

The tracker ran up to 6 checks at a time with no regard for the source,
so one source could take every slot, and a Cloudflare-protected or
rate-limited source failed once for each of its manga in the batch.

- At most 2 checks run per source. The source permit is taken before
  the global one, so a busy source does not hold global slots while
  its own checks wait.
- The first Cloudflare or rate-limit (429) failure of a source stops it
  for the rest of the run: its other manga are not requested, and are
  stamped as checked with that failure in one update, so a blocked
  source still cannot fill the following batches.
- Captcha failures are stored as a new result, RESULT_CAPTCHA, so they
  can be found again once the captcha is solved. The tracker debug
  screen shows them like failures.
- The captcha notification also finds a Cloudflare error wrapped in
  another exception.
Manga whose check stopped at a Cloudflare captcha used to wait for
their next turn in the queue, up to a full tracking cycle, even after
the captcha was solved.

CaptchaHandler now reports each solve, whether automatic or from the
captcha notification, through solvedSources. Swiping the notification
away is not a solve. WorkScheduleManager listens and queues one
re-check per source: a one-time TrackWorker run that checks only the
tracks of that source stored as RESULT_CAPTCHA. A re-check already
waiting or running for the source is kept.

It cannot loop. A re-check that meets the captcha again only shows the
captcha notification, and only another solve queues a new one.

On by default: "Re-check after solving a CAPTCHA" in the tracker
settings.
…arance User-Agent

* fix(webview): make request interception survive Cloudflare interstitials

Re-inject the page script on every navigation and poll it every 250 ms
while capturing, instead of injecting it once (captures used to time out
when a challenge page preceded the real one). Serialise captures with a
mutex and reuse a single WebView, releasing it asynchronously instead of
synchronously destroying it. Evaluation WebViews in WebViewExecutor are
now destroyed after use, and evaluateJs uses cancellation-safe
suspendCancellableCoroutine.

Ported from Kotatsu-Redo 9.8.2 (dc0de75).

* fix(cloudflare): remember the User-Agent that earned clearance per source

When the CloudFlare solver WebView is given a non-Android (e.g. hard-coded
desktop) User-Agent that contradicts the engine, realign it with the
engine's default UA so Turnstile can pass, and once the check passes
store it in SourceSettings.cloudFlareUserAgent. CommonHeadersInterceptor
then forces that UA for the source, so OkHttp presents the agent the
cf_clearance cookie is bound to. The background WebViewExecutor solver
uses the same stored UA.

Ported from Kotatsu-Redo 9.8.1 (cb430d3).

* fix(cloudflare): let the user's User-Agent setting win over the stored clearance agent

The agent recorded after a Cloudflare solve was never cleared and silently
overrode a User-Agent the user had configured for the source. Record the
user's setting alongside it and ignore the stored agent once that changes.
…xperimental, off by default)

Opening a catalog, a manga or a chapter now first tries to pass a
CloudFlare challenge silently: a translucent, focused
CloudFlareHiddenActivity loads the challenge and a read-only JS
page-state probe (CF_STATE_JS) decides when the real page is shown.
CaptchaAutoResolveCoordinator owns the single session (de-duplicated,
30 s success / 3 min failure cooldowns, cancelled when the app goes to
background, never started from background). CloudFlareActivity gets the
resolve loop with viewport wait, stable-pass counting, one auto-recreate
with a cleared clearance and a reveal for manual completion; the visible
solver uses the same loop. ChaptersLoader and PageLoader run through
runWithVerification (prefetch never starts a solve), the reader retries
loading details after a solve and the catalog shows "Solving captcha
automatically…". Background workers and coil errors never auto-solve.
New per-source opt-out "Disable automatic CAPTCHA solving".

The headless WebViewExecutor.tryResolveCaptcha and its
CaptchaContinuationClient / ContinuationResumeWebViewClient are removed:
CaptchaHandler was their only user and it now delegates to the
coordinator. AdBlock is kept.

Ported from Kotatsu-Redo 9.7.4-9.8.3 (eb2dd23, d4bb9af, c24bd1b,
b1483eb, cb430d3, 77d993d).

Automatic solving is experimental and off by default: a global switch
in the network settings, "Solve CAPTCHAs automatically (experimental)".
SourceSettings.isCaptchaAutoResolveDisabled reads it as well as the
per-source opt-out, so every caller follows it: the solver itself, the
catalog and the details screen. The per-source switch is greyed out
while the global one is off.
The "percent left" and "chapters left" badges no longer show a minus sign.

Ported from Kotatsu-Redo 9.7.2 (b81c174).
Adds a "Hide NSFW from shortcuts" toggle (on by default); NSFW titles
are also skipped whenever NSFW content is disabled globally.

Ported from Kotatsu-Redo 9.8.3 (77d993d).
…y and Favourites

* feat(favourites): pin favourites to the top of the list

Pin/unpin from the favourites selection mode. Pinned entries sort first
under every sort order (MAX(pinned) for queries grouped across
categories, so a manga pinned in any of its lists is consistently
treated as pinned in the all-favourites view) and show a pin badge in
the grid and detailed list modes. Pinning from the all-favourites view
pins the manga in every category it is in. Uses the existing
favourites.pinned column; no migration.

Ported from Kotatsu-Redo 9.7.0 (cf3625d).

* feat(favourites): quick filters by content type and popular tags

Adds Manga / Manhwa / Manhua and the three most common tags of the
current list (or of all favourites) to the favourites quick filters.
Tags are matched by title so the same genre from different sources is
one chip; blacklisted tags are not offered.

Content type is resolved per kind of source: built-in parsers by their
declared ContentType (as in Redo), Mihon sources - whose wrapper reports
every source as plain manga - by a "manga"/"manhwa"/"manhua" genre tag
on the manga itself. A Mihon manga without such a tag is only listed
when no content type chip is selected, instead of being silently
misfiled under "Manga" or dropped.

Ported from Kotatsu-Redo 9.7.2 (b81c174).

* feat(library): search within History and Favourites

On History and Favourites the shared search bar offers a scope chip
("Search" / "History" or "Favorites"). In screen mode it filters the
list underneath instead of navigating: typed text matches the title,
alternative title, source (by display title, resolved to stored source
names) and favourites list name; suggestions offer matching lists, the
sources present on the screen and matching entries. The active filter
is shown in the collapsed bar with a clear action, is dropped when
leaving the screen, and counts as a filter for the empty state and
"Reset filter".

Differences from Redo:
- SourceSearchFilter also resolves Mihon sources, from the installed
  extensions and the titles persisted in the sources table.
- The search text is bound as SQL arguments (MangaQueryBuilder gains
  bound WHERE args) with LIKE wildcards escaped, instead of being
  inlined; the source list is bound too.
- The on-device (Downloaded) path is filtered in SQL through the
  same DAO query rather than by title in memory.
- ScreenFilterLog/HistoryWriteLog debug logging is not ported.

Ported from Kotatsu-Redo 9.8.3 (77d993d) and 9.8.4 (ef09cb2).
Show a "Content unavailable" dialog when opening a manga whose source
answers 404 / removed, with a shortcut to the alternatives screen.
Also recognises HTTP 404 thrown by Mihon extension sources.

Ported from Kotatsu-Redo 9.8.3 (77d993d).
Named sets of languages and sources, created and edited from Explore
(presets button, dropdown switcher and overflow menu). While a preset is
active, Explore lists its sources, "disable" removes a source from the
preset, global search queries the preset's sources, and the sources
catalog adds/removes sources to/from the preset instead of enabling them.
Presets resolve built-in parsers, installed Mihon extensions and external
plugins by name; uninstalled sources and a dangling active preset id
(deleted preset, settings restored without the table) fall back cleanly.

New table source_presets (Migration30To31, DB version 31). The active
preset id survives settings backup/restore (Long values restored from
JSON as Integer are normalized).

Ported from Kotatsu-Redo 9.7.0 (cf3625d), 9.7.1 (32b2923).
…vers

* feat(alternatives): replace source and alternatives search options

"Replace source" replaces "Fix" in the favourites/history selection modes
(single-source selections) and is added to the details menu. The new
source replacement screen samples the library titles of the source,
compares every candidate source (enabled, disabled or any preset), lets
the user pick the best one and applies matches automatically with a
smart mix of the next best sources, then falls back to choosing a source
per remaining title manually. Each migration stays its own transaction;
a failed one no longer aborts the batch and replaced titles leave the
queue even when the job is cancelled.

The alternatives screen gets search options: source scope (enabled,
disabled, any preset), same language, same content type, hide results
without chapters, four sort orders, a query editor and a progress
counter. Language and content-type rules also cover Mihon extensions.
Auto fix searches the default scope (active preset or enabled sources).
Adds MangaDao.findLibraryBySource. Community ranking is not ported.

Ported from Kotatsu-Redo 9.8.2 (dc0de75), 4a74fd4, 9.8.3 (77d993d),
9.8.5 (9fb36ac).

* fix(alternatives): confirm before replacing the source of every matched title

* feat(override): pick a cover from other sources

Adds "Covers from other sources" to the manga edit screen. Candidates come
from AlternativesUseCase; covers are deduplicated by file name and by a
perceptual DCT hash (CoverHash), previewed, and the chosen remote cover is
downloaded into the override cover store on save, with the source's
request headers.

Ported from Kotatsu-Redo 9.8.3 (77d993d).
Adds a Coil decoder (covers, thumbnails, previews) and JPEG XL handling
in BitmapDecoderCompat, which also covers reader pages through the
existing SSIV convert-to-PNG fallback. Detection is by content as well
as by .jxl extension. Native library loading is guarded by JxlSupport so
a failing load only fails the JPEG XL image.

Ported from Kotatsu-Redo 9.8.5 (9fb36ac, d45b7c1).
… smart resume

* fix(details): estimate progress when the history chapter id is gone

When the history chapter is not in the fresh chapter list, rescale the
stored percent to the new chapter count instead of giving up. The
estimate may only lower progress and never marks a manga completed.
Progress recalculation also keeps history.chaptersCount up to date.
Futon's guard against overwriting a newer history row is kept for both
writes.

Ported from Kotatsu-Redo 9.7.4 (eb2dd23) / 9.8.1 (cb430d3).

* fix(history): keep the reading position when a source changes chapter ids

Some sources derive chapter ids from data that changes between loads.
Before a stored chapter list is replaced, move the history onto the same
chapter of the new list (same url/branch, else same volume and number,
else same url). Saving history maps a stale chapter id the same way.
When the chapter cannot be identified, the position is estimated from
the percentage in the branch it was read in (or the branch closest in
size) rounding up at chapter boundaries, instead of
(chapters.size * percent) over all branches.

Ported from Kotatsu-Redo 9.7.8 (f34b597) / 9.8.0 (b1483eb) / 9.8.5 (9fb36ac).

* fix(details): re-pick the default branch after the fresh chapter list loads

The default branch was only picked from the first (usually cached)
chapter list. Pick it again when the selected branch is gone from the
fresh list, or when it was not picked from history and the user has not
changed it, so a history recovered onto a re-identified chapter selects
the right branch.

Ported from Kotatsu-Redo 9.8.5 (9fb36ac).

* fix(history): keep the reader's chapter count so smart resume can advance

Smart resume only moves on to a new chapter when the saved chapter was the
last one at the reader's last save, which it reads from
history.chaptersCount. The ported progress recalculation (run when details
open and when the tracker finds new chapters) and the one-shot tracker
migration both overwrote that count with the current total, so after a new
release Continue reopened the finished chapter instead.

Leave chaptersCount to the reader again, as upstream did. The count-based
estimate for a vanished chapter id still updates it, since it scales the
percent by the old count and smart resume does not apply there.
* feat(reader): keep chapters apart in double-page mode, add cover page option

Pad every chapter with blank spacer pages to an even number of slots so
that two chapters never share a spread, and optionally show the first
page of each chapter alone ("First page as cover (solo)" in the reader
config sheet). Adapter positions are mapped back to the real page
positions for the page counter, slider jumps, saved state and history;
the reversed reader now pads in reading order and reverses afterwards so
spreads stay identical in RTL. Toggling the cover option re-lays out the
pages in place, keeping the current page.

Ported from Kotatsu-Redo 9.7.1 (32b2923).

* feat(reader): zoom double-page spreads as a whole, push pages together

Replace the per-page zoom of the double-page reader with a pinch-zoom
and pan over the whole spread (DoublePageScalingFrame), center each page
against the other one of its spread instead of the screen edges, and let
wide images (aspect ratio above 1.3) span the full spread.

Ported from Kotatsu-Redo 9.7.0 (cf3625d) and 9.7.1 (32b2923).
* feat(reader): E-Ink flash on page change

Optional full-screen white or black flash on page change to clear E-Ink
ghosting, with configurable duration and frequency (every N pages).
Page changes are counted once the reader has settled (150 ms), using the
reader's own position: one per page in the pagers, one per spread in the
double-page reader (spacer pages excluded), one per centre page in
webtoon mode. Restoring the position on open, switching reader mode and
webtoon autoscroll do not flash.

Ported from Kotatsu-Redo 9.7.8 (f34b597).
A manual check now checks every track even when it can't run in the
foreground. As plain background work, a large library can run into
the job time limit or doze, and WorkManager then re-queues the
request and starts it again from scratch later. With the tracker set
to manual, that shows up as an automatic check hours after the user
asked for one.

Stamp manual requests and skip a run that only starts more than 30
minutes after it was requested. Also let the progress notification
time out, so a worker killed before its finally block can't leave it
stuck, usually at 100%.
finishSuccess() ran discard() in the Activity's scope and closed only
once it returned. discard() waits for the captcha lock and rebuilds
the remaining captcha notifications, favicons included, so the screen
could stay open on the solved page. Tapping X then destroyed the
Activity, which cancelled discard(): the source stayed marked as
captcha-protected and the tracker re-check never ran.

Close right away and record the solve in the process scope instead.
Also bound each page probe, so a WebView that never answers
evaluateJavascript can't stall the loop, and log challenge state
changes to make failed solves diagnosable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

2 participants