Squashed commit of the following: - #22
Merged
martinsumner merged 1 commit intoSep 3, 2026
Merged
Conversation
commit 2b56df8 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Aug 27 16:35:51 2026 +0100 Maybe push ledger cache when SQN changes beyond threshold (martinsumner#498) * Fix jitter and integer requirement * Format fix commit ba4464e Merge: edbce08 fa947f7 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Aug 27 12:37:20 2026 +0100 Merge pull request martinsumner#491 from martinsumner/mas-d34-i477-binaryledgervalue Mas d34 i477 binaryledgervalue commit fa947f7 Author: Thomas Arts <thomas.arts@quviq.com> Date: Tue Aug 25 19:58:20 2026 +0200 Reduce test scope to more liberal decoding commit 815a9b6 Author: Thomas Arts <thomas.arts@quviq.com> Date: Thu Aug 20 16:23:02 2026 +0200 Fix the creation of data and better failure printing commit 257d15d Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Aug 20 11:54:40 2026 +0100 Remove unused function from export list commit d228948 Merge: 16763e1 3db9353 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 18:19:32 2026 +0100 Merge remote-tracking branch 'quviq/mas-d34-i477-binaryledgervalue' into mas-d34-i477-binaryledgervalue commit 16763e1 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 17:54:49 2026 +0100 Type deduplication commit 28593a1 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 17:51:47 2026 +0100 Clarify type on hash commit 50a9eb6 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 17:45:41 2026 +0100 Support variable length metadata i.e. The bytes required to store the length of the binary can vary with the size of the binary - between 1 and 15 bytes can be used to store the length commit e03603c Merge: 96250dd 14e300c Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 13:10:41 2026 +0100 Merge branch 'mas-d34-i477-binaryledgervalue' of https://github.com/martinsumner/leveled into mas-d34-i477-binaryledgervalue commit 14e300c Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 13:10:21 2026 +0100 Update src/leveled_codec.erl Co-authored-by: Thomas Arts <thomas.arts@quviq.com> commit 96250dd Merge: c940f49 edbce08 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 12:54:41 2026 +0100 Merge branch 'develop-3.4' into mas-d34-i477-binaryledgervalue commit edbce08 Merge: 776eba8 6268903 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 12:54:11 2026 +0100 Merge pull request martinsumner#495 from martinsumner/mas-d34-i494-standardtree Simplify leveled_tree commit 6268903 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Aug 18 11:47:54 2026 +0100 Comment typos commit 6f47429 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Mon Aug 17 14:15:45 2026 +0100 Clarify types following review commit d7cce42 Merge: 6493e1b 776eba8 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Mon Aug 17 12:51:02 2026 +0100 Merge branch 'develop-3.4' into mas-d34-i494-standardtree commit 776eba8 Merge: 6dd1a91 bf3c923 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Mon Aug 17 12:50:39 2026 +0100 Merge pull request martinsumner#493 from martinsumner/mas-d34-i492-map Replace dict with map commit bf3c923 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Mon Aug 17 11:14:05 2026 +0100 Use maps:foreach following review commit 3db9353 Author: Thomas Arts <thomas.arts@quviq.com> Date: Wed Aug 12 15:58:59 2026 +0200 Generate the binaries from a grammar (as reference for futire changes) commit 28e8926 Author: Thomas Arts <thomas.arts@quviq.com> Date: Wed Aug 12 13:46:31 2026 +0200 simplistic generator commit a77b499 Author: Thomas Arts <thomas.arts@quviq.com> Date: Wed Aug 12 13:24:27 2026 +0200 Some more exports for test commit 6dd1a91 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Jul 28 11:55:55 2026 +0100 README.md - CI branch reference commit 6493e1b Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Fri Jul 17 17:09:44 2026 +0100 Add empty search test back (i.e. search when whole tree inside of key) commit 8fa2ea8 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Fri Jul 17 14:19:14 2026 +0100 Change API to use simpler function names Also strip out redundant functions commit 73c8ae9 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Fri Jul 17 11:16:11 2026 +0100 Tree should just be a pure gb_tree or an idxt Remove skpl and old tree type commit 9e1c470 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Jul 16 19:05:13 2026 +0100 Optimisations of idxt tree - and switch to use it everywhere Extend eunit testing to better prove relative performance. Handle issues with inefficient ++ commit c940f49 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Jul 14 00:51:32 2026 +0100 Remove unnecessary short circuit commit 31993d7 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Mon Jul 13 23:30:20 2026 +0100 Add short circuits into extractions For each extraction function, allow the most common expected scenario to be accelerated through a simple pattern match. commit 7787e20 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Fri Jul 10 11:57:13 2026 +0100 Experiment with pattern matching common cases commit 74c8131 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Jul 9 20:16:57 2026 +0100 Replace dict with map The bloom fetch may happen multiple times per request, and so avoiding the expensive hash-lookup in dict should be a marginal gain. commit 80ca7fe Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Wed Jul 8 16:52:58 2026 +0100 Refine tests commit a10c8df Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Wed Jul 8 13:54:25 2026 +0100 Fix fmt commit 35e16dc Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Wed Jul 8 13:44:07 2026 +0100 Expand testing to include v2 and v3 ledger values Also the switching between v2 and v3. commit 1490a83 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Jul 7 19:20:14 2026 +0100 Add version, change ordering Order of elements matters for performance. Significant difference seen with perf_prof. Version added to front to allow for future changes commit bde71ad Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Tue Jul 7 14:40:47 2026 +0100 Make ledger value version configurable Allow for a version 3 which is a binary version. Expectation is that in making blocks less complex terms (as the majority of the term is hidden in a binary), the binary_to_term/term_to_binary overhead when creating and reading slots will be reduced. commit 7f08bba Merge: 8934604 f10abba Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Feb 12 12:21:24 2026 +0000 Merge pull request martinsumner#487 from martinsumner/mas-d34-or34update leveled_bookie:status/1 (#17) commit f10abba Merge: a57cf88 8934604 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Feb 12 11:08:43 2026 +0000 Merge branch 'develop-3.4' into mas-d34-or34update commit 8934604 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Feb 12 11:08:03 2026 +0000 Rename any cdb files outside of the manifest (martinsumner#486) * Rename any cdb files outside of the manifest Rename them as *.bak files. This aligns with the ledger, which does the same thing at startup. These files are expected when a stop or crash happens during a journal compaction. In this case, files may be created - but not added to the manifest. These orphaned files will be in the post_compact folder. It is less expected for folders in the non-compact area, but can still happen when there is a crash during the delete_pending state. The garbage is not collected (deleted) - but this just makes it easier for OS admins to clear it, as they don't need erlang functions, *.bak files can be assumed to be garbage. * Update src/leveled_inker.erl Co-authored-by: Thomas Arts <thomas.arts@quviq.com> * Add comment following review * Format correctly --------- Co-authored-by: Thomas Arts <thomas.arts@quviq.com> commit a57cf88 Author: Andriy Zavada <johnhommer@gmail.com> Date: Thu Feb 12 13:02:59 2026 +0200 leveled_bookie:status/1 (#17) Return a map of status information about the bookie (when the bookie is not a snapshot). The content of the status may change in future releases. --------- Co-authored-by: Andriy Zavada <andriy.zavada@tiot.jp> Co-authored-by: Martin Sumner <martin.sumner@adaptip.co.uk> commit bb9e78d Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Thu Dec 18 10:56:36 2025 +0000 Don't require regex to be compiled (martinsumner#484) Ad don't compile regex in evaluation expressions commit 4583ee6 Author: Martin Sumner <martin.sumner@adaptip.co.uk> Date: Wed Nov 19 13:35:55 2025 +0000 Switch to using domain filtering not log_type (martinsumner#483) * Switch to using domain filtering not log_type This looks like it is a more correct way of doing this. Given we have no agreed hierarchy as part of OpenRiak, will use [backend, leveled] as a start. * Align log format with Erlang standards The leveled_log code is not aligned well with the expectations of riak kernel logger. In particular log function calls were centralised, and so MFA metadata is irrelevant. This change implements the log function calls as a macro, so that the should_i_log logic cna be maintained, but MFA metadata will be correctly present in the log. It also ends the practice of adding the pid() directly to the log, instead relying on the metadata * Format via erlfmt * ensure logger:allow/2 is called Will be filtered by global and ?MODULE log filters. Outstanding code coverage issue caused by detecting `false` branch when macro is substituted over multiple lines. * Add missing space to logs with no db_id * Coverage code coverage is an issue with the switch to macros for logging. Not every log line will have a `false` from the log allow - and that will fail code coverage ... but only if the log covers multiple lines. Most logs covering multiple lines, do do needlessly. So avoid the code coverage issue by making those logs single line. * Update include/leveled.hrl Co-authored-by: Thomas Arts <thomas.arts@quviq.com> * Updates post review * Remove unused get_log Code uses get_loglevel/1 --------- Co-authored-by: Thomas Arts <thomas.arts@quviq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Amalgamation of upstream PRs:
martinsumner#491 - Make the metadata within the ledger a binary (rather than a term) which can be directly decoded (i.e. individual elements can be extracted).
martinsumner#493 - Replace use of dict() with map().
martinsumner#495 - Standardise tree structure on an optimised adaptation of gb_trees.
martinsumner#498 - A fix to an issue whereby otherwise repeated changes to a small set of keys could lead to unbounded journal growth even with compaction (or trim).
Co-authored-by: Thomas Arts thomas.arts@quviq.com