Repository navigation
refactor(sql): read every SQL span through SQLNonCodeSpan and run one shared script corpus against the core - #4289
Merged
Merged
Conversation
… shared script corpus against the core
…ent limit instead of holding it
…ame in the fingerprint
… any directive as what it sends
…and read a MySQL hint as a comment
… by a new shape column
…d a versioned MySQL comment both ways on iOS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SQLFileParser) reads every comment, literal, quoted name and SQLite parameter withSQLNonCodeSpanon its chunk buffer. Its own quote, comment, dollar andq''state machine is gone (SQLFileParser+Literals.swiftdeleted).E'...'string continued after a line break, a SQLite$name(...)parameter holding a quote, a dropped comment gluing its neighbours (a/* x */FROMsent asaFROM), a client delimiter of 3+ units cut by a chunk.SQLNonCodeSpan.needsMoreText), and the next read at least doubles what is held, so a literal of many megabytes is lexed a few times, not once per chunk.SQLFileParseError.spanTooLong), by the count and the import, instead of being held in memory.QueryBracketScanner) read with the session's grammar: no false "No matching opening bracket" inside$$...$$,[a)], a MySQL#comment, PostgreSQL'C:\'or a nested comment; a)inside a run/*! */is code and its*/is not.COPY ... FROM STDINkeeps its command in the shape, a SurrealQL⟨name⟩stays a name, and@@versionstays a variable. History and plan snapshots group and match by a newshape_hashcolumn the backfill fills.fingerprint_hashkeeps the previous stable's reading, from a frozen copy (SQLLegacyFingerprint, pinned to v0.86.0's shapes) that nothing else reads and that goes once no supported release reads the column.# undoon MySQL, nested comments on PostgreSQL).core/crates/core/tests/fixtures/sql_script_corpus.json(52 cases: strings, comments, dollar quotes,DELIMITER,GObatches, per dialect), runs in the core (ScriptPlan, andScriptReadercut 1, 2, 7 and whole), the statement scanner, the import cut 1, 7 and 64 KiB, and the kit's lexer. The core reads every case right, so it is unchanged.SQLFeatureLexerandSQLiteSQLScanner: a plugin cannot link the grammar package. The corpus now holdsSQLFeatureLexerto the app's lexer.detect-changed-paths.sh:macosandiosrun when the corpus changes.SQLFingerprintGoldenTestsandsystemVariableFingerprint(shapes now follow the engine's grammar),QueryHistoryMigrationTests(plan snapshots gainshape_hash),SQLTokenizerTests(grammarless cases name their grammar),QueryDiagnosticsTestsandTransactionRollBackClassifierTests(take a grammar).Test plan
SQLScriptSharedCorpusTests,SQLFileParserSharedCorpusTests,script_shared_corpus.rs,SQLLexicalFeatureMappingTests.kitLexerAgreesOnTheSharedCorpus,SQLNonCodeSpanTestsbuffer-end cases,QueryDiagnosticsTestsgrammar cases,SQLFileParserTestscontinuation, Tcl parameter, comment, long tag, split comment, long delimiter and long literal cases,SQLFileParserTests.spanLongerThanTheLimitIsRefused,QueryHistoryFingerprintMigrationTests.shapeColumnLeavesThePreviousStableItsFingerprint,QueryPlanSnapshotStorageTests.planKeepsThePreviousStablesFingerprint,SQLLegacyFingerprintTests,TransactionRollBackClassifierTests.commentsFollowTheGrammar.swift testfor TableProSQLGrammar, TableProAutocomplete, TableProPluginKit and TableProQuery,cargo test -p tablepro-core --test script_shared_corpusand clippy. The iOS suite runs in CI only: this Mac has no iOS simulator runtime.Closes #4271