"UTF8CodepointSizeFast" — question about existing optimization

[flowCRANE]

The UTF8CodepointSizeFast is implemented in this way (comments are original):

Code: Pascal  [Select][+][-]

1. function UTF8CodepointSizeFast(p: PChar): integer;
2. begin
3.   case p^ of
4.     #0..#191   : Result := 1;
5.     #192..#223 : Result := 2;
6.     #224..#239 : Result := 3;
7.     #240..#247 : Result := 4;
8.     //#248..#255 : Result := 1;
9.     // Theoretically UTF-8 supports length 1-7, but since 2003, RFC 3629 limits
10.     // it to 1-4 bytes.
11.     // This is an inline function, so keep the function short.
12.     //#248..#251   : Result := 5;
13.     //#252, #253   : Result := 6;
14.     //#254         : Result := 7;
15.  
16.     else Result := 1; // An optimization + prevents compiler warning about uninitialized Result.
17.   end;
18. end;

Can somebody explain me please why the else statement is an optimization, as the comment says?

[Thaddy]

1. makes the case stay in order for most of the part. In order case generates efficient code.
2. unexpected result are treated as resolving to case 1, which means result is always valid
3. Prevents the Jonas' case without else warning in trunk (6060)

[domasz]

Is the above any better than the below?

Code: Pascal  [Select][+][-]

1.     function UTF8CodepointSizeFast(p: PChar): integer;
2.     begin
3.   Result := 1;
4.       case p^ of
5.         #0..#191   : Result := 1;
6.         #192..#223 : Result := 2;
7.         #224..#239 : Result := 3;
8.         #240..#247 : Result := 4;
9.         //#248..#255 : Result := 1;
10.         // Theoretically UTF-8 supports length 1-7, but since 2003, RFC 3629 limits
11.         // it to 1-4 bytes.
12.         // This is an inline function, so keep the function short.
13.         //#248..#251   : Result := 5;
14.         //#252, #253   : Result := 6;
15.         //#254         : Result := 7;
16.       end;
17.     end;

[Thaddy]

No it fails my points 2 and 3. At least 2 needs to be taken into account. Since it has a less likelihood, solving it in the else will result in faster code in the common case. Unless you want exception frames and these are dead slow.

[domasz]

Thank you!

[Thaddy]

Note I believe the compiler can already optimize case loops in higher optimization settings: it is an "easy" one.

[RayoGlauco]

I think originally the code was:

Code: Pascal  [Select][+][-]

1.     function UTF8CodepointSizeFast(p: PChar): integer;
2.     begin
3.       case p^ of
4.         #0..#191   : Result := 1;
5.         #192..#223 : Result := 2;
6.         #224..#239 : Result := 3;
7.         #240..#247 : Result := 4;
8.         #248..#255 : Result := 1;
9.       end;
10.     end;
11.  

Then, someone commented this

Code: Pascal  [Select][+][-]

1. #248..#255 : Result := 1;

and added this

Code: Pascal  [Select][+][-]

1. else Result := 1;

[RayoGlauco]

The final version seems faster, and avoids the compiler warning

[Thaddy]

Less comparisons at the cost of one jump.
Usually this is such a case that compilers can optimize better than humans, well humans who do not take into account extra complexity. It is a kind of pre-AI type AI, well it is AI: making compilers smarter than people.

[flowCRANE]

I still don't know what type of optimization would be used. This code might as well look like this:

Code: Pascal  [Select][+][-]

1. function UTF8CodepointSizeFast(p: PChar): integer;
2. begin
3.   case p^ of
4.     #0   .. #191: Result := 1;
5.     #192 .. #223: Result := 2;
6.     #224 .. #239: Result := 3;
7.     #240 .. #247: Result := 4;
8.     #248 .. #255: Result := 1;
9.   end;
10. end;

The order is correct, the full range of byte values has been used. A redundant compiler message about a possible missing result does not bother (although such a case does not exist for single-byte Char). So what advantage does the else version give in practice (at the asm level and various optimizations)?

Code: Pascal  [Select][+][-]

1. function UTF8CodepointSizeFast(p: PChar): integer;
2. begin
3.   case p^ of
4.     #0   .. #191: Result := 1;
5.     #192 .. #223: Result := 2;
6.     #224 .. #239: Result := 3;
7.     #240 .. #247: Result := 4;
8.   else
9.     Result := 1;
10.   end;
11. end;

Does it even matter?

[TRon]

Does it even matter?

Not really as the true winner is an 256 byte array 

[JuhaManninen]

I removed the inaccurate part of the comment about optimization in fee8dbf80e.