Repository navigation
feat(database): Supports CSV import and semantic layer analysis - #183
Merged
Merged
Conversation
- 区分 CSV 托管来源与 JDBC 引擎,复用 MySQL 数据源和分析链路 - 增加上传预览、字段类型确认、异步批量导入及原子语义发布 - 支持失败重试、删除补偿、任务恢复和过期文件清理 - 在查询及元数据入口校验表归属和就绪状态,沿用角色表权限 - 增加前端 CSV 分析空间、MySQL 5.7 兼容迁移及 Docker 配置说明 验证: - 后端主代码及既有测试编译通过,Checkstyle 和本次 Java 改动的 Spotless 检查通过 - 前端类型检查、生产构建及本次改动的 ESLint 检查通过 - MySQL 8.0 独立临时库验证导入、精确聚合、权限隔离、失败重试和删除 - 迁移脚本重复执行通过,未新增或执行单测,仅同步既有测试构造依赖 - MySQL 5.7 尚未实机验证;启用步骤见 docs/csv-import.md Refs #165
|
Thank you for your contribution! We will review your request as soon as possible. Please review the code yourself using ponytail https://github.com/DietrichGebert/ponytail. |
Preserve the application executor and align schema inference with empty-value handling. Isolate import attempts, bind retries to the submitting user, and guard publishing, deletion, and orphan cleanup against stale workers. Validate metric table references before deletion, share semantic conversion, and ignore stale upload and pagination responses. Make concurrent confirmations idempotent and return HTTP 413 for oversized multipart uploads. Document dedicated CSV database setup and upgrade requirements. Verified with 57 live integration checks and a successful backend build.
Extract shared datasource validation and inject the CSV executor by name. Generate preview metadata mappings with MapStruct while keeping task lifecycle initialization in the service. Remove the exception factory from the schema converter and introduce CSV-specific error codes through the existing exception handling flow. Document error codes and HTTP status mappings. Validated with a successful backend build and 24 temporary Spring context checks.
Centralize date parsing, the default timezone, and MySQL date range validation. Reuse these rules in CSV inference and conversion, the date tool, and agent prompts. Extract physical naming rules, organize worker execution and cleanup into focused methods, and tighten frontend column and task status types. Validated backend and frontend builds, lint, 9 existing date tool tests, 34 date and naming checks, 24 Spring context checks, and 47 live API checks.
Move shared table and column snapshots into model.bo and update semantic sync consumers. Deep-clone dialog input to keep nested edits independent of the parent list. Persist the final row count and DATA_READY state in one guarded update. Reuse UTF-8 byte counts from record validation when sizing import batches. Validated backend and frontend builds, lint, the existing semantic sync test, and focused checks for editor isolation, stale attempts, atomic completion, and byte limits.
Rename shared schema snapshots with the Bo suffix and use MapStruct for mechanical metadata mappings while keeping business defaults in services. Reuse existing datasource enums and retain contextual asynchronous failure diagnostics before compensation.
Separate upload forms and cleanup responsibilities, reuse JSON utilities, and preserve local date and time values. Keep cleanup failures polling, serialize deletion and reference writes, and protect semantic refreshes from concurrent publication.
Accept Tab delimiters and reject cross-database metric references. Add datasource-aware semantic navigation and isolate state when switching sources. Remove the single-datasource chat selector, clear CSV configuration defaults, and update portable setup documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
验证:
Refs #165