Fix sign dependent flat line detection in get_railed_percentage - #848
Fix sign dependent flat line detection in get_railed_percentage#848adityasingh2400 wants to merge 3 commits into
Conversation
The straight line check compared abs (raw_data[i - 1]) against the signed raw_data[i] because the closing parenthesis was placed after the abs call instead of around the difference. That made the test sign dependent, so a channel stuck at a negative value was never reported as railed and a monotonically rising positive signal was always reported as 100 percent railed. The running maximum was also stored in an int, which discarded the fractional part of every peak and collapsed sub microvolt peaks to zero. Store the maximum as a double and take the absolute value of the difference between consecutive samples, matching the pattern already used by detect_peaks_z_score in the same file.
|
The consecutive-difference correction looks right, and preserving the fractional maximum also fixes a real precision bug. However, this PR contains two observable behavior changes, so the statement that behavior for previously correct signals is unchanged should be narrowed. Please add focused regression coverage for:
The zero case matters because the existing |
|
Sorry, that was codex and autoreview, change looks fine, thanks |
Covers positive and negative flat signals, ascending and descending ramps, fractional amplitudes below 1, and signals crossing or ending at zero. The zero cases pin the existing semantics rather than changing them: a step into zero is suppressed, a step back out of zero is not.
|
Added
So three observable changes, not two. The ramp one is the worst: with the old comparison an increasing ramp always produced a negative difference, so every ascending ramp was reported as fully railed. On the zero semantics, the test pins the current behavior rather than changing it. |
|
I missed your follow-up before posting that, sorry. I had started on the tests when the first comment came in and did not re-check the thread before pushing. The test and the CI line are a separate commit, so if you would rather merge this as the one-line fix you already reviewed, say so and I will drop it. If you want to keep it, nothing further is needed. |
DataFilter.get_railed_percentagedecides whether a channel is flat before it converts the peak amplitude into a percentage of the ADC range. The flat line test inget_railed_percentageis written as(abs (raw_data[i - 1]) - raw_data[i]) > 0.00001, so the closing parenthesis sits after theabscall rather than around the difference. It compares the absolute value of the previous sample against the signed value of the current sample instead of measuring how far apart two consecutive samples are.That makes the check depend on the sign of the data. For a positive signal the condition only becomes true when the signal is decreasing, so a monotonically rising channel never clears the flag and is reported as 100 percent railed. For a negative signal
abs (prev) - curis-prev - cur, which is positive for essentially any pair of negative samples, so the flag is cleared immediately and a channel stuck at a negative value is never detected as railed. Detecting a stuck channel is the main thing this function exists to do, and a negative rail is just as common as a positive one. Separately, the running maximum is declaredint cur_max = abs (raw_data[0]), so the fractional part of every peak is discarded and peaks below 1 uV collapse to exactly 0.The fix moves the parenthesis so the difference is taken before
abs, which matches the patternabs (data[i] - avg_filter[i - 1])already used bydetect_peaks_z_scorea hundred lines further down in the same file, and stores the running maximum as adouble. The behavior for signals that were already handled correctly is unchanged.I built the project on macOS arm64 with
cmake .. && make -j8and ran the reproduction through the Python binding against the freshly builtlibDataHandler. Before the change a flat line at +100 uV returned 100.0 while a flat line at -100 uV returned 0.0533, a rising ramp from 100 uV to 5000 uV returned 100.0, and uniform noise bounded by 0.9 uV returned exactly 0.0. After the change both flat lines return 100.0, the ramp returns 2.6667 in both signs, and the noise case returns 0.00047889, which matchesmax (abs (data)) / max_val * 100computed independently in NumPy. Signals at the actual rail and a flat line at zero still return 100.0, and uniform noise bounded by 90000 uV returns 47.989 which again matches the closed form.There is no existing test that touches
get_railed_percentage, so I could not extend one. As a regression check I ran the synthetic board signal processing examples that CI already runs,denoising.py,signal_filtering.py,transforms.py,downsampling.py,band_power.pyandwindowing.py, and all six pass.clang-formatwith the repository.clang-formatreports no change in the edited region.