I was just going through cleaning up my version of this code that I ported to C# and noticed a possible problem with the way the trie data is generated for the line break data.
See this line of code
|
if ((type != null) && (rangeType !== type)) { |
Seems this code is trying to coalesce consecutive ranges with the same class into a single trie.setRange call. The problem is that it's checking for a change of class, but not that the ranges are consecutive and without gaps.
Shouldn't that line read something like this:
if (((type != null) && (rangeType !== type)) || parseInt(rangeStart, 16) != parseInt(end, 16) + 1)
Also, it seems the UnicodeTrieBuilder already coalesces runs so this isn't even necessary. I reduced it to this:
var re = /^([0-9A-F]+)(?:\.\.([0-9A-F]+))?\s*;\s*(.*?)\s*#/gm
var m;
while (m = re.exec(data))
{
var from = parseInt(m[1], 16);
var to = m[2] === undefined ? from : parseInt(m[2], 16);
var prop = m[3];
lineBreakClassesTrie.setRange(from, to, LineBreakClass[prop], true);
}Unfortunately none of this resolves any of those 30 non-passing tests.
I was just going through cleaning up my version of this code that I ported to C# and noticed a possible problem with the way the trie data is generated for the line break data.
See this line of code
linebreak/src/generate_data.js
Line 31 in 9096a22
Seems this code is trying to coalesce consecutive ranges with the same class into a single trie.setRange call. The problem is that it's checking for a change of class, but not that the ranges are consecutive and without gaps.
Shouldn't that line read something like this:
Also, it seems the UnicodeTrieBuilder already coalesces runs so this isn't even necessary. I reduced it to this:
Unfortunately none of this resolves any of those 30 non-passing tests.