Sets are limited to "ordinal type with a range between 0 and 255."
The compiler converts your widechar code points to chars silently!! Which turns $FFFD into 63, making it between [58..64] (58 & 64 also came from similar silent conversions).
I believe this deserve a hint/warning.