Re: RFC: Small mod to fl_utf8len() to detect invalid single char UTF-8 bytes
"'Albrecht Schlosser' via fltk.coredev"
14:14 Aug 12
On 8/12/26 19:02 Greg Ercolano wrote:
On 8/12/26 09:30, Greg Ercolano wrote:
I
should mention though; FWIW, codex initially suggested rewriting
the code to be as follows, as per RFC 3629:
int fl_utf8len(char c) { const unsigned char u = (unsigned char)c; if (u <= 0x7f) return 1; if (u >= 0xc2 && u <= 0xdf) return 2; if (u >= 0xe0 && u <= 0xef) return 3; if (u >= 0xf0 && u <= 0xf4) return 4; return -1; }
int fl_utf8len1(char c) {
const int len = fl_utf8len(c);
return len < 0 ? 1 : len;
}
[..]
+1 on this version of the code, for these reasons:
- clarity and maintainability (fl_utf8len)
- correctness for UTF-8, according to RFC 3629 (fl_utf8len)
- no duplication of code in fl_utf8len1
BTW, you wrote in an earlier message:
> I guess it depends on whether our fl_utf8 functions should
allow for full utf-8 encoding, perhaps to support utf-8 encoding of
other types of data that might allow 6 bytes, or if we want to limit
it to Unicode purposes only.
FLTK's purpose is to display Unicode text, encoded as UTF-8 strings
(besides any other GUI specific stuff, of course). The functions in
question deal with Unicode text, not any binary data that is -
intentionally or not - encoded in UTF-8. If this is not clearly
documented, then we should do this.
In old documentation we had these tables with up to 6 bytes for
encoding, but this was not RFC 3629 compliant. Historically Unicode
was designed for 31-bit Code Points, but that was revised long ago,
and now Unicode supports only 21 bits (U+0000 .. U+10FFFF), and
that's what we should document and use. I also read in RFC 3629 that
doing the conversion wrong can result in security issues.
Note: meanwhile I'm convinced that we should make this kind
of changes NOW, i.e. before we release FLTK 1.5.0 !
Comments are owned by the poster. All other content is copyright 1998-2026 by Bill Spitzak and others. This project is hosted by The FLTK Team. Please report site problems to 'erco@seriss.com'.