Skip to content

Upgrade to upstream GraphJin v3.21.19 - #9

Draft
Graloth wants to merge 4 commits into
masterfrom
feat/graphjin-v3.21.19
Draft

Graloth wants to merge 4 commits into
masterfrom
feat/graphjin-v3.21.19

Conversation

@Graloth

@Graloth Graloth commented Nov 8, 2026

Copy link
Copy Markdown

What is the purpose of this pull request?

Updates the fork from upstream GraphJin 0.17.x (its last upstream merge is from 2022-02-18, a day before v0.17.16 was released) to upstream v3.21.19, plus a few small changes on top, and updates Go and all dependencies to the newest stable versions.

What has changed?

Four commits, meant to be read in order:

  1. Replace the fork with upstream v3.21.19: upstream's tracked files, unchanged, minus what is left out on purpose: tests-large, wasm, training, website, upstream's own CI workflows and release tooling (goreleaser, npm, install script), the web UI sources and lockfile (the prebuilt bundle in serv/web/build stays because serv embeds it), and upstream's AI-tooling files. The old fork's docs, chart, Jenkinsfiles and changelog are dropped.
  2. Return a single object when a unique key pins one row (MySQL, MariaDB):
    • A selection with an explicit limit: 1, or one whose where clause and join cover a unique or primary key, returns an object (queries only; unique indexes are read from information_schema.STATISTICS).
    • The MySQL IN operator in a where clause renders as the old fork did.
    • Only the snake_case identity variables (user_id, user_id_provider, ...) are reserved; the camelCase spellings are ordinary variables.
    • The schema cache is versioned, so a cache written without index data is rebuilt.
    • Ten of upstream's database-backed tests expect an array for such a selection; they skip on MySQL and MariaDB with the reason in the code and still run on the other databases.
  3. Update to Go 1.27.3 and the newest dependencies, fix lint findings: modernc.org/sqlite stays at v1.60.1 (v1.61.0 and later return SQLite JSON values as strings, which breaks upstream's own tests) and goja-babel stays at upstream's pin (the newest commit has no tag and broke 17 cmd tests). golangci-lint v2.15.0 reports 0 issues in every module (about 5,000 findings before, most of them errcheck on dialect.Context.WriteString). The commit message lists the changes that are more than mechanical.
  4. Add the image build and a public CI workflow: a static, cross-compiled Dockerfile (hadolint-clean) and .github/workflows/build.yml.

Notes for your reviewer

  • Tests pass in every module in a golang:1.27.3 container without Docker. The Postgres suite (about 90 s) and the MySQL suite (about 65 s, with the ten skips above) pass against containers. One serv test that relies on a read-only file now skips when run as root (it failed the same on upstream).
  • Vulnerability scan: nothing High or Critical. Remaining: gosnowflake GO-2026-6665 (Medium, no fixed version), golang.org/x/crypto GO-2026-5932 (fix only as an untagged commit), busybox CVE-2025-60876 in the Alpine image (no fix yet). Nothing is suppressed.
  • .github/workflows/build.yml runs lint and unit tests in every module, the Postgres and MySQL suites (one job each), the vulnerability scan (fails on High), hadolint and an image build with no push. Actions are pinned to commit SHAs, the token is read-only and there are no secrets. This is its first run, so the checks on this PR are the real test of it.
  • Behaviour that differs from upstream v3 on purpose: the single-object selection above (MySQL, MariaDB), and the identity variable names. Behaviour that differs from the old fork: the web UI is only served in dev mode unless web_ui is set explicitly.

Risk assessment

This replaces the whole code base, so treat it as high risk for anything that talks to the server. The behaviour that callers rely on was compared query by query against the previous build outside this repository, and the database-backed suites above pass; nothing is deployed from this PR. The image is built and published separately, so merging does not roll anything out.

Replace the 2022 fork with the tracked files of upstream v3.21.19 (tag v3.21.19,
commit 5914e90), unchanged, except for what is left out on purpose:

- tests-large, wasm, training and website.
- upstream's own CI workflows, and its release tooling: goreleaser, the npm
  package and the install script.
- the web UI sources and lockfile. The prebuilt bundle in serv/web/build stays
  because serv embeds it. The UI is only served in dev mode unless web_ui is set
  explicitly.
- upstream's AI-tooling files.
- go.work no longer lists wasm.

Everything from the old fork (docs, chart, Jenkinsfiles, CHANGELOG and so on) is
dropped.
Port a few behaviour changes of the old fork onto v3:

- A selection with an explicit `limit: 1` returns a single object, and so does a
  selection whose where clause and join cover every column of a unique or primary
  key (MySQL and MariaDB, queries only). The unique indexes are read from
  information_schema.STATISTICS and stored in the schema snapshot. Mutations and
  cursor pagination keep upstream's behaviour.
- The MySQL `IN` operator in a where clause renders as the old fork did.
- v3 reserves the camelCase identity variables (userId, userIdProvider and so on).
  Only the snake_case names are reserved now; the camelCase spellings are ordinary
  variables.
- The schema cache format is versioned (snapshot version 2, discovery generation
  format version 2), so a cache written without index data is rebuilt instead of
  loaded, and the DDL fallbacks are only used when they can carry index data.

Ten of upstream's database-backed tests expect an array for such a selection. They
skip on MySQL and MariaDB with the reason in the code, and still run on every other
database.
Security update: Go 1.27.3 (checked against go.dev) and the newest stable version of
every dependency in all modules, including tests and conf (go get -u, go mod tidy).
Exceptions, each for a reason:

- modernc.org/sqlite stays at v1.60.1: v1.61.0 and later return SQLite JSON values
  as strings, which breaks upstream's own tests.
- goja-babel stays at upstream's pin: the newest commit has no tag and broke 17 cmd
  tests.

Code changes the newer modules force: moby Port.Int() became Port.Num() in
cmd/cmd_demo.go and tests, a vet %w error in codesql/indexer.go. cmd's test that reads
the training directory skips when it is absent.

Fix every golangci-lint v2.15.0 finding (default linters, no config file). The default
output hides most findings behind per-linter caps; the real count was about 5,000,
mostly errcheck on dialect.Context.WriteString in core. All modules now report 0 issues.
Changes that are more than mechanical:

- core: dialect.Context.Write and WriteString return nothing (internal interface; every
  implementation writes to a buffer). Two CompileQuery errors in internal/psql/mutate.go
  now propagate. sub.s in subs.go is a *gstate so the struct literal no longer copies
  atomic counters. The MongoDB geo render error goes to ctx.SetError. strings.Title in
  the Oracle dialect is replaced by an equivalent helper. Dead branches and unused code
  are removed.
- serv: about 1,500 lines of unreachable MCP tool code are removed. The two scoped
  service copies in mcp_config.go share the parent's config lock, as scopedCopy does. A
  test that relies on a read-only file skips when run as root (it failed the same on
  upstream in a root container).
- mongodriver: nil contexts in rows.go became context.Background().
- cmd: the deprecated testcontainers BindMount became HostConfigModifier in the demo
  command.
- agent: lead_test.go was vacuous (marshalling a func field always failed and the error
  was dropped); it now checks the request data. Two SA4000 test patterns are rewritten.
- tests: 44 findings, mostly unchecked Close calls.

Tests pass in every module in a golang:1.27.3 container without Docker, and the Postgres
and MySQL suites pass against containers.
- Dockerfile: static build of the cmd module on the build platform, cross-compiled for
  the target, with the time zone database embedded so the runtime image needs no tzdata
  package. The runtime image runs apk upgrade for security fixes the base tag has not
  picked up. Passes hadolint at the warning level.
- .github/workflows/build.yml: lint (golangci-lint v2.15.0) and unit tests in every
  module, the Postgres and MySQL suites of the tests module (one job each), a
  vulnerability scan that fails on High, hadolint and an image build with no push.
  Public actions pinned to commit SHAs, a read-only token and no secrets. Docs-only
  changes do not trigger it.
- .gitignore: ignore local tooling and working notes.
@Graloth Graloth self-assigned this Nov 8, 2026
@Graloth
Graloth requested a balanced review from Copilot November 8, 2026 22:21

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants