feat(job): configure the jobs tick route's batch size, timeout and queues

changelog.d/jobs-tick-settings.added.md

@@ -0,0 +1 @@
+- The jobs tick route (`/wheels/jobs/tick`) takes its batch from settings: `jobsRunnerTickMaxJobs` (most jobs per call, `1`-`1000`, default `1`), `jobsRunnerTickTimeout` (seconds per job, `1`-`86400`, default `300`) and `jobsRunnerTickQueues` (comma list, default every queue); a fractional or out-of-range value falls back to the default. The defaults keep today's behaviour of one job per call. They are settings rather than request parameters, so a caller holding the token can't widen the work one request does.

vendor/wheels/JobTickEndpoint.cfc

@@ -10,7 +10,9 @@
  * - a request carrying any forwarding header (see $forwardingHeaders()), even an empty one,
  *   came through a proxy, so it is refused (403) whatever its token;
  * - a request whose method or headers can't be read is refused (500) without running a tick,
- *   rather than being judged on missing information.
+ *   rather than being judged on missing information;
+ * - each tick runs with the jobsRunnerTickMaxJobs / jobsRunnerTickTimeout / jobsRunnerTickQueues
+ *   settings, never with values from the request.
  *
  * handle() takes the request's parts and returns {status, contentType, body}, so the rules are
  * testable without HTTP. Dispatch calls it before routing and before the public-component gate.
@@ -63,17 +65,19 @@ component {
 
 	/**
 	 * Internal: one tick under a per-host lock taken try-once, so a call that overlaps a running
-	 * tick on this server returns at once instead of stacking.
+	 * tick on this server returns at once instead of stacking. Its batch size, timeout and queues
+	 * come from the app's settings ($tickArguments()), never from the request.
 	 */
 	public struct function $runTick() {
 		local.started = GetTickCount();
 		local.job = new wheels.Job();
 		local.host = local.job.$jobHostName();
+		local.tickArgs = $tickArguments();
 		var outcome = {ran = false, summary = {}, error = ""};
 		lock name="wheels.jobs.tick.#local.host#" type="exclusive" timeout="1" throwOnTimeout="false" {
 			outcome.ran = true;
 			try {
-				outcome.summary = $newRunner().tick();
+				outcome.summary = $newRunner().tick(argumentCollection = local.tickArgs);
 			} catch (any e) {
 				outcome.error = e.message;
 			}
@@ -93,6 +97,46 @@ component {
 		return local.rv;
 	}
 
+	/**
+	 * Internal: the arguments each tick runs with, from settings so that a caller holding the token
+	 * can't widen the work one request does: jobsRunnerTickMaxJobs (most jobs per call, default 1),
+	 * jobsRunnerTickTimeout (seconds per job, default 300) and jobsRunnerTickQueues (comma list,
+	 * default every queue). maxJobs must be a whole number from 1 to 1000 and timeout one from 1 to
+	 * 86400 (a day); a missing, fractional or out-of-range value falls back to its default.
+	 */
+	public struct function $tickArguments() {
+		local.queues = $setting("jobsRunnerTickQueues", "");
+		return {
+			maxJobs = $wholeSetting(name = "jobsRunnerTickMaxJobs", fallback = 1, maximum = 1000),
+			timeout = $wholeSetting(name = "jobsRunnerTickTimeout", fallback = 300, maximum = 86400),
+			queues = IsSimpleValue(local.queues) ? Trim(local.queues) : ""
+		};
+	}
+
+	/**
+	 * Internal: a setting that must be a whole number from 1 to `maximum` (well inside the INTEGER
+	 * columns it can end up in, such as a claimed job's claimTimeout), written as plain digits.
+	 * Anything else (missing, a fraction, zero, a sign, exponent notation such as "1e3", or above
+	 * the maximum) is the fallback: a value that isn't plainly a whole number is never coerced
+	 * into one. Never through Int(), which Lucee truncates to 32 bits.
+	 */
+	public numeric function $wholeSetting(required string name, required numeric fallback, required numeric maximum) {
+		local.value = $setting(arguments.name, arguments.fallback);
+		if (!IsSimpleValue(local.value)) {
+			return arguments.fallback;
+		}
+		// A whole number reads as digits, possibly with a ".0" tail (an engine's double).
+		local.text = Trim(ToString(local.value));
+		if (!ReFind("^[0-9]+(\.0+)?$", local.text)) {
+			return arguments.fallback;
+		}
+		local.number = Val(ListFirst(local.text, "."));
+		if (local.number < 1 || local.number > arguments.maximum) {
+			return arguments.fallback;
+		}
+		return local.number;
+	}
+
 	/**
 	 * Internal: compares two tokens in time that doesn't depend on where they differ. Both are
 	 * hashed first, so the comparison is always over two 64-character digests.

vendor/wheels/events/onapplicationstart.cfc

@@ -353,6 +353,12 @@ component {
 		// (a token in a URL ends up in access logs).
 		application.$wheels.jobsRunnerToken = "";
 		application.$wheels.jobsRunnerTokenInQuery = false;
+		// What each call to the tick route runs (JobRunner.tick()): at most this many jobs (1-1000),
+		// each capped at this many seconds (1-86400), from these queues ("" = every queue). Settings,
+		// not request parameters, so a caller holding the token can't widen the work one request does.
+		application.$wheels.jobsRunnerTickMaxJobs = 1;
+		application.$wheels.jobsRunnerTickTimeout = 300;
+		application.$wheels.jobsRunnerTickQueues = "";
 		// Job schedules (config/schedules.cfm) are checked and their due slots enqueued at most
 		// this often, from every worker poll, processQueue() and JobRunner.tick(). 0 = never.
 		application.$wheels.jobsScheduleCheckSeconds = 15;

vendor/wheels/tests/specs/jobs/JobTickEndpointSpec.cfc

@@ -20,6 +20,10 @@ component extends="wheels.WheelsTest" {
 			afterEach(function() {
 				application.wheels.jobsRunnerToken = "";
 				application.wheels.jobsRunnerTokenInQuery = false;
+				application.wheels.jobsRunnerTickMaxJobs = 1;
+				application.wheels.jobsRunnerTickTimeout = 300;
+				application.wheels.jobsRunnerTickQueues = "";
+				queryExecute("DELETE FROM wheels_jobs WHERE queue = 'test_tick_batch'", {}, {datasource = application.wheels.dataSourceName});
 			});
 
 			it("recognises the route, with or without a trailing slash", function() {
@@ -113,6 +117,70 @@ component extends="wheels.WheelsTest" {
 				expect(StructKeyExists(body, "durationMs")).toBeTrue();
 			});
 
+			it("runs each tick with the configured batch size, timeout and queues", function() {
+				application.wheels.jobsRunnerTickMaxJobs = 25;
+				application.wheels.jobsRunnerTickTimeout = 900;
+				application.wheels.jobsRunnerTickQueues = "mail, reports";
+				var endpoint = new wheels.JobTickEndpoint();
+				prepareMock(endpoint);
+				var runner = createStub();
+				runner.$("tick", {processed = 0});
+				endpoint.$("$newRunner", runner);
+				endpoint.$runTick();
+				var passed = runner.$callLog().tick[1];
+				expect(passed.maxJobs).toBe(25);
+				expect(passed.timeout).toBe(900);
+				expect(passed.queues).toBe("mail, reports");
+			});
+
+			it("keeps one job, 300 seconds and every queue when nothing is set", function() {
+				StructDelete(application.wheels, "jobsRunnerTickMaxJobs");
+				StructDelete(application.wheels, "jobsRunnerTickTimeout");
+				StructDelete(application.wheels, "jobsRunnerTickQueues");
+				expect(new wheels.JobTickEndpoint().$tickArguments()).toBe({maxJobs = 1, timeout = 300, queues = ""});
+			});
+
+			it("takes maxJobs from 1 to 1000 and a timeout from 1 to 86400, whole numbers only", function() {
+				var endpoint = new wheels.JobTickEndpoint();
+				// Each setting on its own, with the other at its default.
+				var cases = [
+					{name = "jobsRunnerTickMaxJobs", key = "maxJobs", fallback = 1, invalid = [0, -1, 0.5, 1.5, 1001, 2147483648, "1e3", "1e99", "+5", "0x10", "soon", "", "2 jobs", [5], {n = 5}], valid = [{value = 1, expected = 1}, {value = "10", expected = 10}, {value = " 10 ", expected = 10}, {value = 1000, expected = 1000}]},
+					{name = "jobsRunnerTickTimeout", key = "timeout", fallback = 300, invalid = [0, -300, 0.5, 300.25, 86401, 2147483648, "1e3", "1e99", "+600", "soon", "", "1 hour", [5], {n = 5}], valid = [{value = 1, expected = 1}, {value = "600", expected = 600}, {value = 86400, expected = 86400}]}
+				];
+				for (var c in cases) {
+					for (var value in c.invalid) {
+						application.wheels[c.name] = value;
+						var label = IsSimpleValue(value) ? "[" & value & "]" : "a complex value";
+						expect(endpoint.$tickArguments()[c.key]).toBe(c.fallback, c.name & " " & label);
+					}
+					for (var v in c.valid) {
+						application.wheels[c.name] = v.value;
+						expect(endpoint.$tickArguments()[c.key]).toBe(v.expected, c.name & " [" & v.value & "]");
+					}
+					application.wheels[c.name] = c.fallback;
+				}
+				application.wheels.jobsRunnerTickQueues = ["mail"];
+				expect(endpoint.$tickArguments().queues).toBe("");
+			});
+
+			it("drains several jobs in one call when jobsRunnerTickMaxJobs is above 1", function() {
+				var job = new wheels.tests._assets.jobs.ProcessOrdersJob();
+				for (var i = 1; i <= 3; i++) {
+					job.enqueue(queue = "test_tick_batch");
+				}
+				application.wheels.jobsRunnerTickMaxJobs = 3;
+				application.wheels.jobsRunnerTickQueues = "test_tick_batch";
+				var response = new wheels.JobTickEndpoint().handle(method = "POST", headers = {"X-Wheels-Jobs-Token" = "spec-tick-token"});
+				expect(response.status).toBe(200);
+				expect(DeserializeJSON(response.body).processed).toBe(3);
+				var done = queryExecute(
+					"SELECT COUNT(*) AS cnt FROM wheels_jobs WHERE queue = 'test_tick_batch' AND status = 'completed'",
+					{},
+					{datasource = application.wheels.dataSourceName}
+				);
+				expect(done.cnt).toBe(3);
+			});
+
 			it("compares tokens in full, whatever their length", function() {
 				var endpoint = new wheels.JobTickEndpoint();
 				expect(endpoint.$tokensMatch("abc", "abc")).toBeTrue();

web/sites/guides/src/content/docs/v4-2-0/digging-deeper/running-jobs-on-multiple-servers.mdx

@@ -251,12 +251,22 @@ new wheels.JobRunner().tick(
 
 ### Drive it over HTTP: the tick route
 
-When your scheduler can only make an HTTP request — an engine scheduled task, a cron `curl`, a systemd timer — point it at `/wheels/jobs/tick`. Each call runs one `JobRunner.tick()` on that server with its defaults — so it reaps, enqueues any due schedules, and runs **at most one job** (capped at 300 s) — and returns a JSON summary. There's no request parameter to raise either limit yet ([#4520](https://github.com/wheels-dev/wheels/issues/4520) will make the route's batch size and timeout configurable), so one call a minute is one job a minute per server: fine for light, steady queues, but for volume run `wheels jobs work` (a continuous loop) or call `JobRunner.tick(maxJobs = n, timeout = …)` from CFML instead. Turn the route on by setting a token:
+When your scheduler can only make an HTTP request — an engine scheduled task, a cron `curl`, a systemd timer — point it at `/wheels/jobs/tick`. Each call runs one `JobRunner.tick()` on that server — so it reaps, enqueues any due schedules, and runs a bounded batch of jobs — and returns a JSON summary. By default that batch is **one job** (capped at 300 s) from any queue, so one call a minute is one job a minute per server. Raise it with settings — not request parameters, so a caller holding the token can't widen the work one request does. For a steady high volume, `wheels jobs work` (a continuous loop) is still the better fit. Turn the route on by setting a token:
 
 ```cfm {test:compile} title="config/settings.cfm"
 set(jobsRunnerToken = "a-long-random-secret");   // the route is 404 until this is set
+
+// Optional: what each call runs (the defaults are 1, 300 and every queue).
+set(jobsRunnerTickMaxJobs = 10);        // at most this many jobs per call, one after another (1-1000)
+set(jobsRunnerTickTimeout = 600);       // seconds each job may run (1-86400)
+set(jobsRunnerTickQueues = "default");  // comma list of queues; "" = every queue
 ```
 
+A fractional or out-of-range value falls back to the default. When sizing a batch:
+
+- **It has to fit inside one HTTP request.** The jobs in a call run one after another, inside that request, so `jobsRunnerTickMaxJobs` × the time a job takes, plus a little overhead, must fit within **your engine's request timeout**. `jobsRunnerTickTimeout` caps each job; it doesn't extend the request's timeout. It should also fit inside your scheduler's interval, because an overlapping call on the same server is skipped rather than stacked. For long or many jobs, run `wheels jobs work` (a continuous worker) instead.
+- **A request the engine kills mid-job leaves that job claimed.** The reaper recovers it like any job whose worker died: once its claim or heartbeat goes stale it is retried, or, for a job with `this.idempotent = false`, marked `interrupted` (see [Idempotency and `this.idempotent`](#idempotency-and-thisidempotent)). Keep your jobs idempotent.
+
 Then hit it with the token in a header, once a minute, from each server:
 
 ```bash title="a cron entry / systemd timer"