FLTK logo

Re: [fltk.coredev] RFC: Small mod to fl_utf8len() to detect invalid single char UTF-8 bytes

FLTK matrix user chat room
(using Element browser app)   FLTK gitter user chat room   GitHub FLTK Project   FLTK News RSS Feed  
  FLTK Library      Forums      Links      Apps     Login 
[  All Forums  |  Back to fltk.coredev  ]
 
[ Previous Message ] [ New Message | Reply ] [ Next Message ]

Re: RFC: Small mod to fl_utf8len() to detect invalid single char UTF-8 bytes "'Albrecht Schlosser' via fltk.coredev" 14:14 Aug 12  
 
On 8/12/26 19:02 Greg Ercolano wrote:

On 8/12/26 09:30, Greg Ercolano wrote:

    I should mention though; FWIW, codex initially suggested rewriting the code to be as follows, as per RFC 3629:
  int fl_utf8len(char c) {
    const unsigned char u = (unsigned char)c;
    if (u <= 0x7f) return 1;
    if (u >= 0xc2 && u <= 0xdf) return 2;
    if (u >= 0xe0 && u <= 0xef) return 3;
    if (u >= 0xf0 && u <= 0xf4) return 4;
    return -1;
  }

  int fl_utf8len1(char c) {
    const int len = fl_utf8len(c);
    return len < 0 ? 1 : len;
  }
[..]

+1 on this version of the code, for these reasons:

- clarity and maintainability (fl_utf8len)
- correctness for UTF-8, according to RFC 3629 (fl_utf8len)
- no duplication of code in fl_utf8len1

BTW, you wrote in an earlier message:
> I guess it depends on whether our fl_utf8 functions should allow for full utf-8 encoding, perhaps to support utf-8 encoding of other types of data that might allow 6 bytes, or if we want to limit it to Unicode purposes only.

FLTK's purpose is to display Unicode text, encoded as UTF-8 strings (besides any other GUI specific stuff, of course). The functions in question deal with Unicode text, not any binary data that is - intentionally or not - encoded in UTF-8. If this is not clearly documented, then we should do this.

In old documentation we had these tables with up to 6 bytes for encoding, but this was not RFC 3629 compliant. Historically Unicode was designed for 31-bit Code Points, but that was revised long ago, and now Unicode supports only 21 bits (U+0000 .. U+10FFFF), and that's what we should document and use. I also read in RFC 3629 that doing the conversion wrong can result in security issues.

Note: meanwhile I'm convinced that we should make this kind of changes NOW, i.e. before we release FLTK 1.5.0 !

--
You received this message because you are subscribed to the Google Groups "fltk.coredev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkcoredev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/fltkcoredev/ced69e1d-8e88-4fac-9b11-050b6e4f471b%40aljus.de.
[ Direct Link to Message ]
 
     
[ Previous Message ] [ New Message | Reply ] [ Next Message ]
 
 

Comments are owned by the poster. All other content is copyright 1998-2026 by Bill Spitzak and others. This project is hosted by The FLTK Team. Please report site problems to 'erco@seriss.com'.