perf: merge and recount each unique subgraph object once in incremental loop

Duplicated words share one subgraph object via the build cache, but the
incremental update loop merged and recounted per occurrence, so duplicates
diverged into equal-but-distinct copies on their first merge. Memoizing by
object identity does the work once per unique word; the resulting Counter is
shared across occurrences (comp_counts entries are only ever read).

Zipf corpus (20k words, 2k vocab, 300 merges), byte-identical output:
- BPE: 0.362s -> 0.172s, peak 35.8MB -> 31.5MB
- BNE: 0.501s -> 0.352s, peak 37.8MB -> 35.3MB

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: AmitMY

complex_tokenization/trainer.py

@@ -95,10 +95,17 @@ def _train_incremental(self, num_merges: int, progress: bool = False):
             nodes = next(m for cc in comp_counts for m in cc if (len(m) - 1) * total[m] == best)
             token = reduce(lambda x, y: x + y, nodes)
 
+            # Duplicated words share one subgraph object (the build cache), so
+            # merge and recount each distinct object once, not per occurrence.
+            # The shared Counter is safe: comp_counts entries are only read.
+            merged: dict[int, tuple[GraphVertex, Counter]] = {}
             for i in list(index[nodes]):
                 _index_remove(total, index, i, comp_counts[i])
-                components[i] = components[i].merge(token, nodes)
-                comp_counts[i] = Counter(components[i].get_merges())
+                pair = merged.get(id(components[i]))
+                if pair is None:
+                    component = components[i].merge(token, nodes)
+                    pair = merged[id(components[i])] = (component, Counter(component.get_merges()))
+                components[i], comp_counts[i] = pair
                 _index_add(total, index, i, comp_counts[i])
 
             self.merges.append((token, nodes))