Repository navigation
Description
DSharpPlus.Commands currently lowercases user input if necessary based on System.Char, which is an UCS2 code point. (I am aware that UCS2 is not "up to date" terminology, the intent is to express that while the characters in the full string are represented as correct UTF16 characters, enumerating them yields them without accounting for combining characters or characters that require four bytes in UTF16 encoding.) This may produce incorrect results on any sufficiently complex script using diacritics in a way that is asymmetrically represented by Unicode, will produce incorrect results on any script where the relationship between uppercase and lowercase versions cannot be symmetrically represented in UCS2 code points or is asymmetric entirely and will produce incorrect results on any script that cannot be represented as merely UCS2 characters, with each char mapping to exactly one character (that is, alphabets using any characters greater than 0xFFFF). Affected scripts include, but are not limited to:
- Osmanya, one of the scripts used for the Somali Language
- Osage, used by the Osage language
- Garay, occasionally used in the Wolof language
- Khema, used by the Gurung language
- Bamum, rarely used by the Bamum and Shümom languages
- Mro, occastionally used by the Mru language
- Tangsa, rarely used by the Tangsa language
- Pahawh Hmong, used by the Hmong Daw and Hmong Njua languages
- Kirat Rai, used by the Bantawa language
- Nyiakeng Puachue Hmong, used by the White Hmong and Green Hmong languages
- Toto, used by the Toto language
- Wancho, rarely used by the Wancho language
- Mundari Bani, rarely used by the Mundari language
- Ol Onal, rarely used by the Bhumij language
- Mende Kikakui, rarely used by the Mende language
- Adlam, used by the Fula language
Furthermore, other scripts may need to rely on "extensions" in affected unicode planes, which will exhibit similar issues. Planes 1 (Supplementary Multilingual Plane), 2 (Supplementary Ideographic Plane) and 3 (Tertiary Ideographic Plane) are handled incorrectly today, and behaviour exhibited should be considered erroneous by default.
Since System.Char will slice UTF16 surrogate pairs into two System.Chars, which will then be treated as separate UCS2 characters, the value of the character during lowercasing may change to where it now represents two entirely distinct characters or it may render the string incomprehensible in its entirety: attempting to use Adlam characters, for example, will break the command in its entirety and it will not be uploaded to Discord at all, irrespective of its validity otherwise.
Metadata
Metadata
Assignees
Labels
Type
Fields
Effort
Activity