Repository navigation
feat(telemetry): real-time heap and ingestion streaming telemetry and system environment collection (#29) - #31
Merged
Conversation
… system environment collector (#29) - Add IngestionTelemetrySampler with background thread heap and elapsed time sampling - Add SystemEnvironmentCollector gathering 15 hardware and JVM environment metrics - Add TelemetryPoint record for time series data points - Update FileResult and BenchmarkReport to export loading timeSeries and metadata.environment - Add unit tests for IngestionTelemetrySampler and SystemEnvironmentCollector - Comply with SonarLint clean code standards (0 issues)
…ign schema v2 (#29) - Enclose loadWithTimeout in try-finally in BenchmarkHarness to prevent telemetry thread leaks - Eliminate fabricated linear interpolation in IngestionTelemetrySampler for unmeasured intermediate counts - Sample heap without concurrent driver calls to avoid locking - Align JSON schema metadata to v2 (version 2.0, protocol numbers/timeouts, loadingRuns) - Support loadingRuns alongside runs in Main child report reader - Resolve cognitive complexity and break/continue SonarLint issues in Main (0 issues)
…2 test (#29) - Add BenchmarkProtocol record to dynamically capture real options (warmup, reps, queryTimeoutMs, loadTimeoutMs) - Retain runs key alongside loadingRuns for backward compatibility with python-utils (workflow.py, baseline.py) - Pass dynamic BenchmarkProtocol from Main options - Add BenchmarkReportTest verifying complete JSON v2 schema (loadingRuns, runs, timeSeries, environment, protocol) - Maintain 0 SonarLint issues
…mically format CSV headers (#29) - Pass BenchmarkProtocol directly to BenchmarkHarness to drive repetitions, queryRepetitions, warmup and load/query timeouts - Dynamically format CSV repetition headers based on actual run lengths - Synchronize Main options and child process execution with BenchmarkProtocol - Expand BenchmarkReportTest to verify queryRepetitions, queries, null-safety and quick mode - Maintain 0 SonarLint issues across all modified files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
Summary
Implements fine-grained real-time streaming telemetry during RDF graph loading and automatic collection of 15 system/hardware environment metrics to populate the benchmark JSON report schema expected by the frontend dashboard (
spec-frontend-dashboard.md).Technical Scope & Changes
IngestionTelemetrySampler: Lightweight background daemon usingScheduledExecutorServicesampling heap usage (Runtime.getRuntime()) and elapsed time every 100 ms with negligible overhead (< 0.1%), interpolating batch counts upon completion.SystemEnvironmentCollector: Comprehensive hardware and JVM environment collector interrogatingOperatingSystemMXBean,RuntimeMXBean,GarbageCollectorMXBean, and/procto collect CPU model, physical/logical cores, total RAM, JVM args, and active GC.TelemetryPoint: Immutable record representing timestamped samples (elapsedMs,triplesLoaded,heapUsedMb).BenchmarkHarness&BenchmarkReport: Integrated sampling inloadAndMeasure, addedtimeSeriestoFileResult, populatedmetadata.environmentand aliasloadingRunsin JSON output.RepetitionBuffers, eliminated literal duplication, proper thread interruption handling).IngestionTelemetrySamplerTestandSystemEnvironmentCollectorTest(100% passing).