Skip to content

⚡ Bolt: Optimize LanguageDetector regex split overhead #729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LeanBitLab wants to merge 1 commit into from
Closed

⚡ Bolt: Optimize LanguageDetector regex split overhead #729

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,7 @@
## 2024-05-18 - Optimize Regex pattern detection in DictionaryFacilitatorImpl
**Learning:** `DictionaryFacilitatorImpl.isRegexPattern` used chained `String.contains()` and `String.startsWith()` calls, which internally execute full loops over the string (or delegate to regex). Replacing these chained calls with a single manual O(N) iteration reduces overhead and prevents repeatedly traversing the string.
**Action:** Replace chained string detection methods (`contains`, `startsWith`, `endsWith`) with a single manual loop over the string's indices when checking multiple characters in performance hot paths.

## 2024-05-18 - Optimize Regex split overhead
**Learning:** `LanguageDetector.detectByHeuristics` used `text.lowercase().split(Regex("[^\\p{L}]+")).filter { it.length >= 2 }`. While the Regex was pre-compiled, the split and filter operations still allocated intermediate lists and required regex engine execution, causing GC pressure in a frequently invoked text processing path.
**Action:** Replace `Regex.split` combined with `.filter` in hot paths with manual O(N) string index iteration using character checks (like `char.isLetter()`) and a `StringBuilder` to eliminate regex execution and avoid allocating intermediate collections.
41 changes: 28 additions & 13 deletions app/src/main/java/helium314/keyboard/latin/utils/LanguageDetector.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,6 @@ object LanguageDetector {
return null
}

private val NON_LETTER_REGEX = Regex("[^\\p{L}]+")

private fun detectByHeuristics(text: String, targetLangCode: String?): String? {
val scores = mutableMapOf<String, Int>()

Expand All @@ -175,18 +173,35 @@ object LanguageDetector {
}
}

val words = text.lowercase().split(NON_LETTER_REGEX).filter { it.length >= 2 }
for (word in words) {
if (SPANISH_WORDS.contains(word)) addScore("es", 2)
if (FRENCH_WORDS.contains(word)) addScore("fr", 2)
if (GERMAN_WORDS.contains(word)) addScore("de", 2)
if (ITALIAN_WORDS.contains(word)) addScore("it", 2)
if (PORTUGUESE_WORDS.contains(word)) addScore("pt", 2)
if (DUTCH_WORDS.contains(word)) addScore("nl", 2)
if (TURKISH_WORDS.contains(word)) addScore("tr", 2)
if (POLISH_WORDS.contains(word)) addScore("pl", 2)
if (ENGLISH_WORDS.contains(word)) addScore("en", 2)
val textLower = text.lowercase()
val len = textLower.length
val wordBuilder = StringBuilder()

fun processWord() {
if (wordBuilder.length >= 2) {
val word = wordBuilder.toString()
if (SPANISH_WORDS.contains(word)) addScore("es", 2)
if (FRENCH_WORDS.contains(word)) addScore("fr", 2)
if (GERMAN_WORDS.contains(word)) addScore("de", 2)
if (ITALIAN_WORDS.contains(word)) addScore("it", 2)
if (PORTUGUESE_WORDS.contains(word)) addScore("pt", 2)
if (DUTCH_WORDS.contains(word)) addScore("nl", 2)
if (TURKISH_WORDS.contains(word)) addScore("tr", 2)
if (POLISH_WORDS.contains(word)) addScore("pl", 2)
if (ENGLISH_WORDS.contains(word)) addScore("en", 2)
}
wordBuilder.clear()
}

for (i in 0 until len) {
val char = textLower[i]
if (char.isLetter()) {
wordBuilder.append(char)
} else {
processWord()
}
}
processWord()

if (scores.isEmpty()) return null

Expand Down
Loading