Print Page - Alternative set of string-to-int conversion routines

Announcements => Third party => Topic started by: avk on January 19, 2022, 06:45:12 am

Title: Alternative set of string-to-int conversion routines
Post by: avk on January 19, 2022, 06:45:12 am

Hi!
For a long time I was going to try to create a string-to-integer conversion function that would be noticeably faster than the built-in Val().
There seems to be something like this (https://github.com/avk959/str2int-alter) at the moment. Anyway, a quick and dirty benchmark against Val() from the current development version of FPC on my machine looks like this:

Code: Text [Select][+]

Int32:
Val(), score:        2853
TryChars2Int, score: 1137
Int64:
Val(), score:        3045
TryChars2Int, score: 1121
 

Parsing is carried out basically according to the same rules as in the built-in Val() (well, as far as I understood them). IMO there is only one significant difference from Val(): since the lone leading zero is the prefix of an octal string, decimal strings cannot start with a zero.
The set also includes functions that accept an array of char and a PChar.
Feedback and comments are highly appreciated, if any.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 19, 2022, 08:06:46 am

Nice. One remark: there is a lot of duplicate code and that can be avoided.
There are two options I can see:
1. You could write generic functions, implemented and declared in the implementation section. That will (should) not affect speed since generics are templates.
2. Use the technique sometimes used in the rtl, where a header include uses the same implementation include per routine wherein the type is changed in the header, so the implementation include knows the correct type. (predates generics) Also should not affect speed.

But compliments. I find the same speed gain, percent wise.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 19, 2022, 08:15:50 am

Please add

Code: Pascal [Select][+]

function TryStr2xxx(const s: string; const aIndex, aCount: SizeInt; out aValue: xxx): Boolean; inline;

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 19, 2022, 08:35:43 am

@Thaddy, thank you.
You're right, a lot of code is duplicated and I don't have an option yet to avoid this without sacrificing performance. On the other hand, this set is still a proof of concept.

@zamtmn, doesn't this code

Code: Pascal [Select][+]

  if TryChars2Int(s[5..15], MyInt) then
    ...
 

cover your case?

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 19, 2022, 08:50:10 am

Code: Pascal [Select][+]

TryStrToInt(Copy(s,5,10),MyInt)

also covers my case, but the point is to make it fast, without unnecessary allocations and copies

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 19, 2022, 09:27:07 am

I just compiled this example:

Code: Pascal [Select][+]

program project1;
{$MODE OBJFPC}{$H+}
uses
  SysUtils, Str2IntAlter;
 
procedure Test(const s: string);
var
  I: Integer;
begin
  if TryChars2Int(s[22..Length(s)], I) then
    WriteLn('I = ', I)
  else
    WriteLn('Ooooppss!');
end;
 
var
  s: string = '';
begin
  Randomize;
  s := 'it is random integer:  ' + Random(MaxInt).ToString;
  Test(s);
  ReadLn;
end.
 

To call TryChars2Int(s[22..Length(s)], I) FPC generated the following code:
32 bits

Code: ASM [Select][+]

# [10] if TryChars2Int(s[22..Length(s)], I) then
        movl    %eax,%edx
        testl   %eax,%eax
        je      .Lj5
        movl    -4(%edx),%edx
.Lj5:
        subl    $22,%edx
        movl    %esp,%ecx
        addl    $21,%eax
        call    STR2INTALTER_$$_TRYCHARS2INT$array_of_CHAR$LONGINT$$BOOLEAN
        testb   %al,%al
        je      .Lj7
# [11] WriteLn('I = ', I)
 

64 bits

Code: ASM [Select][+]

# [10] if TryChars2Int(s[22..Length(s)], I) then
        movq    %rcx,%rdx
# Peephole Optimization: %rdx = %rcx; changed to minimise pipeline stall (MovXXX2MovXXX)
        testq   %rcx,%rcx
        je      .Lj5
        movq    -8(%rdx),%rdx
.Lj5:
        subq    $22,%rdx
        leaq    32(%rsp),%r8
        leaq    21(%rax),%rcx
        call    STR2INTALTER_$$_TRYCHARS2INT$array_of_CHAR$LONGINT$$BOOLEAN
        testb   %al,%al
        je      .Lj7
# [11] WriteLn('I = ', I)
 

There doesn't seem to be any unnecessary copying going on.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 19, 2022, 09:54:08 am

Oh yes, I completely forgot that

Code: Pascal [Select][+]

TryChars2Int(const a: array of char; ...

this is not equivalent to

Code: Pascal [Select][+]

type
  TTest=array of char;
procedure TryChars2Int(const a:TTest; ...

the request is closed :-[

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 19, 2022, 09:22:23 pm

Благодарю за выложенный код!
Я буду знать, что я иду в правильном направлении! :)

google translate:
Thanks for posting the code!
I will know that I am going in the right direction! :)

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 20, 2022, 02:25:55 pm

As per Thaddy's remarks, tried to somehow reduce code duplication.
Generic functions need to be declared in the interface part of the unit, and since they are helpers, this is not very good.
Includes are also not suitable, because I want everything to stay in one unit.
As a result, as an experiment, I settled on macros. The code looks unreadable, of course, but it seems that it would not be much better with includes.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 20, 2022, 03:09:18 pm

It became very ugly. In my opinion, the best solution would still be genetics and two modules:
Str2IntAlter - for uses in main unit
Str2IntAlterInternal - for uses in Str2IntAlter, contains generic help functions

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 20, 2022, 03:36:03 pm

Ok, I get it, I need to think about it some more. In any case, there is no problem to roll back to the previous version.
But I really want everything to be in one self-sufficient unit.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 20, 2022, 04:38:47 pm

In principle, using generics (as I suggested too) should not have a speed penalty at all.

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on January 20, 2022, 06:58:36 pm

Quote from: Thaddy on January 20, 2022, 04:38:47 pm

In principle, using generics (as I suggested too) should not have a speed penalty at all.

In principle. In practice I find that debug sessions on my code regularly take me through the generics unit, which suggests that it /does/ interpose code in various places even when not used explicitly.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 20, 2022, 07:20:06 pm

Lyrical digression.
As I mentioned above, I had a desire to create a faster function for converting a string to an integer quite a long time ago, or more precisely, at the time of this epic story (https://forum.lazarus.freepascal.org/index.php/topic,46103.msg327460.html#msg327460).
At that time, in order to defeat Windows file mapping in the company with NtDll, it came to some completely crazy decisions.
I hope now it can be done without much straining.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 20, 2022, 10:43:05 pm

Quote

Int32:
Val(), score: 3073
rejected: 0

TryChars2Int, score: 1486
rejected: 0

sc_StrToLongWord or sc_StrToInt, score: 1104
rejected: 0

Int64:
Val(), score: 3075
rejected: 0

TryChars2Int, score: 1627
rejected: 0

sc_StrToQWord or sc_StrToInt64, score: 1072
rejected: 0

Press any key to exit...

Я проверил свой код в вашей демке. Не воспринимайте это как "вызов". Ваш код людям нравится больше. И они считают, что вашим кодом можно "выдёргивать" цифры из текста. Возможно они правы.
Не всегда "вычурные" решения бывают самыми лучшими. При переводе строк, табличными значениями удобно пользоваться только в шестнадцатеричной системе. Ваш код помог мне решить несколько мелких проблем в моём коде. Хотя идеи далеко не новы... я просто забыл о них.
google translate:
I checked my code in your demo. Don't take it as a "challenge". People like your code more. And they think that your code can "pull out" numbers from the text. Perhaps they are right.
Not always "artsy" solutions are the best. When translating strings, it is convenient to use table values only in the hexadecimal system. Your code helped me solve a few small problems in my code. Although the ideas are far from new... I just forgot about them.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 21, 2022, 06:44:01 am

I'm glad I was able to help you.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 08:53:02 am

Quote from: MarkMLl on January 20, 2022, 06:58:36 pm

Quote from: Thaddy on January 20, 2022, 04:38:47 pm
In principle, using generics (as I suggested too) should not have a speed penalty at all.

In principle. In practice I find that debug sessions on my code regularly take me through the generics unit, which suggests that it /does/ interpose code in various places even when not used explicitly.

MarkMLl

I believe this is only true if you use one of the generic libraries, NOT for completely self defined generic code as one can see from the generated assembler.
E.g. this code, that only uses the system unit:

Code: Pascal [Select][+]

{$mode delphi}
type
  TIntArray = TArray<integer>;
  TStringArray = TArray<string>;
  procedure swap<T>(var left,right:T);inline;
  var
    temp:T;
  begin
    temp:=left;
    left:=right;
    right:=temp;
  end;
 
  procedure Reverse<T>(var Value:TArray<T>);inline;
  var
    i: Integer = 0;
  begin
    if Length(Value) > 0 then
      for i := Low(Value) to High(Value) div 2 do
        Swap<T>(Value[i],Value[High(Value) - i]);
  end;
 
var
  i:integer;
  s:string;
  si:TintArray = [1,2,3,4,5,6,7,8,9,10];
  ss:TStringArray = ['there', 'are', 'more', 'things', 'in', 'heaven', 'and', 'earth',', ', 'Horatio'];
begin
  reverse<integer>(si);
  for i in si do write(i:2);
  writeln;
  reverse<string>(ss);
  for s in ss do write(s:length(s)+1);
end.

The assembler shows that the "templates" are filled in *before* any assembler code is generated.
There is simply no reference to anything "generics" in the generated code apart from possibly internal namings.
There is simply no codepath related to generics at all.
I compiled this with -al -CX -XXs -O4
I also tested a non-generic version if Int reverse: Hey, near identical code!

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on January 21, 2022, 09:31:58 am

Quote from: Thaddy on January 21, 2022, 08:53:02 am

I believe this is only true if you use one of the generic libraries, NOT for completely self defined generic code as one can see from the generated assembler.

I can only assume that it's somewhere in the standard FPC or LCL libraries. It's definitely not anything that I've done explicitly.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 09:41:37 am

Quote from: MarkMLl on January 21, 2022, 09:31:58 am

I can only assume that it's somewhere in the standard FPC or LCL libraries. It's definitely not anything that I've done explicitly.

MarkMLl

That seems strange to me. My example only uses system and no libraries at all.
In fact, it shows using generics for those string to int conversions is a viable option without loss of speed.
The generated code is even inlined.

If you can disproof this, pls show me what happens at your side.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 21, 2022, 10:02:32 am

Geneticists are still making debugging very difficult, and may even affect the build due to compiler bugs. But they make readable code and errors will be fixed (but not as fast as I would like)

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 10:09:06 am

So what is difficult? As long as you specialize at type level the debugger has no issues, at least I did not encounter them. I am aware that if you specialize at var level the debugger does not always play nice. That is an important distinction.

Code: Pascal [Select][+]

type
  TIntArray = TArray<integer>;
  TStringArray = TArray<string>;

Note I use fp + GDB.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 21, 2022, 10:14:27 am

I have often encountered an incorrect display of the current executed line, it is shifted 1-2 up or down

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 10:15:49 am

Quote from: zamtmn on January 21, 2022, 10:14:27 am

I have often encountered an incorrect display of the current executed line, it is shifted 1-2 up or down

How did you specialize? type level or var level? Can you give us a small example to reproduce?

GDB is GNU gdb unicode (GDB) 9.2

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on January 21, 2022, 10:22:27 am

Quote from: Thaddy on January 21, 2022, 09:41:37 am

Quote from: MarkMLl on January 21, 2022, 09:31:58 am
I can only assume that it's somewhere in the standard FPC or LCL libraries. It's definitely not anything that I've done explicitly.

MarkMLl
That seems strange to me. My example only uses system and no libraries at all.
In fact, it shows using generics for those string to int conversions is a viable option without loss of speed.
The generated code is even inlined.

If you can disproof this, pls show me what happens at your side.

I'll report back if I can spot what's going on.

Please note that I'm not commenting on an observed performance issue, merely that the generics library is insinuating itself /somewhere/ so there's the /potential/ for a performance issue if a shim is being traversed e.g. inside a tight loop.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 21, 2022, 10:23:59 am

I specialize in different ways, but I try to do in types, I'm talking about debugging inside generics classes

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 10:25:38 am

Quote from: MarkMLl on January 21, 2022, 10:22:27 am

Please note that I'm not commenting on an observed performance issue, merely that the generics library is insinuating itself /somewhere/ so there's the /potential/ for a performance issue if a shim is being traversed e.g. inside a tight loop.

MarkMLl

I think the latter is wrong. But I am open to further testing.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 10:27:29 am

Quote from: zamtmn on January 21, 2022, 10:23:59 am

I specialize in different ways, but I try to do in types, I'm talking about debugging inside generics classes

That is no issue if you specialize at type level and debug the specialization, not the template - which is not there for the debugger to see.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 21, 2022, 10:32:26 am

Quote from: Thaddy on January 21, 2022, 10:27:29 am

That is no issue if you specialize at type level and debug the specialization, not the template - which is not there for the debugger to see.

I know your point of view - it's not a compiler problem, it's a bad programmer's problem)) But no. Debugging the code is also necessary inside the genetic classes and it is very difficult

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 21, 2022, 10:38:18 am

Quote from: zamtmn on January 21, 2022, 10:32:26 am

I know your point of view - it's not a compiler problem, it's a bad programmer's problem)) But no. Debugging the code is also necessary inside the genetic classes and it is very difficult

Well, no, that is not what I wrote. Both ways to specialize are fully legal.
Specialized at type level it isn't even possible - by sheer logic - to debug the template code, afaik there is none.
Maybe @PascalDragon can help us here.... Sarah? Or @FPK himself?

Note that this may seem to some a side discussion but it isn't: once we fully understand what is going on, we can help and improve on the original subject's code. At least that is how I think: we have to establish the facts.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 21, 2022, 12:03:08 pm

Maybe this is Lazarus' problem, I can't reproduce this in a minimal example. I think that with some changes in the generic source code, EXE or PPU remain old, and this causes incorrect display of the executed lines. It's related https://gitlab.com/freepascal.org/fpc/source/-/issues/39387

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 21, 2022, 01:08:26 pm

Of course, I have some understanding of the current possibilities of generics, but so far this does not in any way stimulate the choice in their favor due to the above reason.
On the other hand, is duplication of code in a library so bad? Indeed, in this case, changes in the code of some function will not affect others.

Title: Re: Alternative set of string-to-int conversion routines
Post by: PascalDragon on January 21, 2022, 03:51:40 pm

Quote from: avk on January 20, 2022, 02:25:55 pm

Generic functions need to be declared in the interface part of the unit, and since they are helpers, this is not very good.

Why do you say that generic functions need to be declared in the interface part of the unit?

Quote from: Thaddy on January 21, 2022, 10:38:18 am

Quote from: zamtmn on January 21, 2022, 10:32:26 am
I know your point of view - it's not a compiler problem, it's a bad programmer's problem)) But no. Debugging the code is also necessary inside the genetic classes and it is very difficult
Well, no, that is not what I wrote. Both ways to specialize are fully legal.
Specialized at type level it isn't even possible - by sheer logic - to debug the template code, afaik there is none.
Maybe @PascalDragon can help us here.... Sarah? Or @FPK himself?

If there are concrete issues with debugging then we need a reproducible example no matter the issue, cause, yes, such things should be fixed, no matter if it's about stepping through generic code correctly or not accidentally entering generic code when one doesn't want it as MarkMLI hinted at.

One potential issue might be (not tested): the generic code is compiled without debug information, but code that uses it is compiled with and thus the specialization will have debug information as well. Thus when stepping through the latter code one might suddenly land in the generic code which one assumed is without debug information.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 21, 2022, 04:13:01 pm

Quote from: PascalDragon on January 21, 2022, 03:51:40 pm

Why do you say that generic functions need to be declared in the interface part of the unit?

Probably lagged behind life, some time ago it was like that, but now I didn’t even try to check. :-[
Thank you.

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on January 21, 2022, 04:49:00 pm

Quote from: PascalDragon on January 21, 2022, 03:51:40 pm

One potential issue might be (not tested): the generic code is compiled without debug information, but code that uses it is compiled with and thus the specialization will have debug information as well. Thus when stepping through the latter code one might suddenly land in the generic code which one assumed is without debug information.

On the other hand, I invariably use a locally-built FPC+Lazarus and looking at my build log for FPC 3.2.0 I see

make NO_GDB=1 CPU_TARGET=i386 OPT='-V3.0.4 -O- -gl -Xs- -vt' all |/usr/local/bin/fpc-filter-vt

Lazarus+LCL though is built with default options, so it could be there.

I'll try to keep an eye open for what I think I've seen, and will report back.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 22, 2022, 11:40:01 am

Some news from str2int-alter:
current version has switched to generics;
added define to force full compatibility with Val();
added benchmarks for unsigned integers, now it looks like this on my virtual Linux machine:

Code: Text [Select][+]

SInt32: 
Val(), score:        3071
TryChars2Int, score: 1250
 
SInt64: 
Val(), score:        3131
TryChars2Int, score: 1177
 
UInt32: 
Val(), score:        5619
TryChars2Int, score: 1248
 
UInt64: 
Val(), score:        7015
TryChars2Int, score: 1280
 

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 22, 2022, 12:01:58 pm

Nice gain.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 24, 2022, 09:37:56 am

Sorry, couldn't resist, found the old benchmark archive from "Sorting and Counting" and ran it again using Str2IntAlter on a win-64 machine:

Code: Text [Select][+]

RandomRange = 1
Julkas1's time: 3.3850  #unique: 100000 #total: 10000000
Julkas2's time: 3.4820  #unique: 100000 #total: 10000000
  Akira's time: 3.1700  #unique: 100000 #total: 10000000
 Howard's time: 4.6500  #unique: 100000 #total: 10000000
   Avk1's time: 1.3300  #unique: 100000 #total: 10000000
   Avk2's time: 0.6100  #unique: 100000 #total: 10000000
  440bx's time: 1.5100  #unique: 100000 #total: 10000000
 BrunoK's time: 1.6900  #unique: 100000 #total: 10000000
BrunoK1's time: 1.1200  #unique: 100000 #total: 10000000
 
RandomRange = 2
Julkas1's time: 3.4000  #unique: 200000 #total: 10000000
Julkas2's time: 3.4500  #unique: 200000 #total: 10000000
  Akira's time: 3.3400  #unique: 200000 #total: 10000000
 Howard's time: 4.6900  #unique: 200000 #total: 10000000
   Avk1's time: 1.3600  #unique: 200000 #total: 10000000
   Avk2's time: 0.6400  #unique: 200000 #total: 10000000
  440bx's time: 1.6200  #unique: 200000 #total: 10000000
 BrunoK's time: 1.7800  #unique: 200000 #total: 10000000
BrunoK1's time: 1.1700  #unique: 200000 #total: 10000000
 
RandomRange = 3
Julkas1's time: 3.4300  #unique: 300000 #total: 10000000
Julkas2's time: 3.4300  #unique: 300000 #total: 10000000
  Akira's time: 3.4300  #unique: 300000 #total: 10000000
 Howard's time: 4.7500  #unique: 300000 #total: 10000000
   Avk1's time: 1.3900  #unique: 300000 #total: 10000000
   Avk2's time: 0.6700  #unique: 300000 #total: 10000000
  440bx's time: 1.6800  #unique: 300000 #total: 10000000
 BrunoK's time: 1.8300  #unique: 300000 #total: 10000000
BrunoK1's time: 1.2500  #unique: 300000 #total: 10000000
 
RandomRange = 4
Julkas1's time: 3.5400  #unique: 400000 #total: 10000000
Julkas2's time: 3.4700  #unique: 400000 #total: 10000000
  Akira's time: 3.6200  #unique: 400000 #total: 10000000
 Howard's time: 4.8200  #unique: 400000 #total: 10000000
   Avk1's time: 1.4200  #unique: 400000 #total: 10000000
   Avk2's time: 0.7100  #unique: 400000 #total: 10000000
  440bx's time: 1.7400  #unique: 400000 #total: 10000000
 BrunoK's time: 1.8900  #unique: 400000 #total: 10000000
BrunoK1's time: 1.3100  #unique: 400000 #total: 10000000
 
RandomRange = 5
Julkas1's time: 3.5000  #unique: 500000 #total: 10000000
Julkas2's time: 3.5000  #unique: 500000 #total: 10000000
  Akira's time: 3.7200  #unique: 500000 #total: 10000000
 Howard's time: 4.9000  #unique: 500000 #total: 10000000
   Avk1's time: 1.4600  #unique: 500000 #total: 10000000
   Avk2's time: 0.7400  #unique: 500000 #total: 10000000
  440bx's time: 1.7900  #unique: 500000 #total: 10000000
 BrunoK's time: 1.9300  #unique: 500000 #total: 10000000
BrunoK1's time: 1.3400  #unique: 500000 #total: 10000000
 
RandomRange = 6
Julkas1's time: 3.5400  #unique: 600000 #total: 10000000
Julkas2's time: 3.5200  #unique: 600000 #total: 10000000
  Akira's time: 3.7700  #unique: 600000 #total: 10000000
 Howard's time: 5.0000  #unique: 600000 #total: 10000000
   Avk1's time: 1.4800  #unique: 600000 #total: 10000000
   Avk2's time: 0.7800  #unique: 600000 #total: 10000000
  440bx's time: 1.8400  #unique: 600000 #total: 10000000
 BrunoK's time: 1.9900  #unique: 600000 #total: 10000000
BrunoK1's time: 1.3900  #unique: 600000 #total: 10000000
 
RandomRange = 7
Julkas1's time: 3.5700  #unique: 700000 #total: 10000000
Julkas2's time: 3.5800  #unique: 700000 #total: 10000000
  Akira's time: 3.8300  #unique: 700000 #total: 10000000
 Howard's time: 5.0600  #unique: 700000 #total: 10000000
   Avk1's time: 1.5300  #unique: 700000 #total: 10000000
   Avk2's time: 0.8000  #unique: 700000 #total: 10000000
  440bx's time: 1.9000  #unique: 700000 #total: 10000000
 BrunoK's time: 2.0100  #unique: 700000 #total: 10000000
BrunoK1's time: 1.4200  #unique: 700000 #total: 10000000
 
RandomRange = 8
Julkas1's time: 3.5900  #unique: 799994 #total: 10000000
Julkas2's time: 3.6100  #unique: 799994 #total: 10000000
  Akira's time: 3.9000  #unique: 799994 #total: 10000000
 Howard's time: 5.1400  #unique: 799994 #total: 10000000
   Avk1's time: 1.5800  #unique: 799994 #total: 10000000
   Avk2's time: 0.8400  #unique: 799994 #total: 10000000
  440bx's time: 1.9400  #unique: 799994 #total: 10000000
 BrunoK's time: 2.0800  #unique: 799994 #total: 10000000
BrunoK1's time: 1.4700  #unique: 799994 #total: 10000000
 
RandomRange = 9
Julkas1's time: 3.6300  #unique: 899990 #total: 10000000
Julkas2's time: 3.6300  #unique: 899990 #total: 10000000
  Akira's time: 3.9400  #unique: 899990 #total: 10000000
 Howard's time: 5.2100  #unique: 899990 #total: 10000000
   Avk1's time: 1.5800  #unique: 899990 #total: 10000000
   Avk2's time: 0.8600  #unique: 899990 #total: 10000000
  440bx's time: 1.9800  #unique: 899990 #total: 10000000
 BrunoK's time: 2.0900  #unique: 899990 #total: 10000000
BrunoK1's time: 1.5200  #unique: 899990 #total: 10000000
 
RandomRange = 10
Julkas1's time: 3.6600  #unique: 999962 #total: 10000000
Julkas2's time: 3.6400  #unique: 999962 #total: 10000000
  Akira's time: 4.0100  #unique: 999962 #total: 10000000
 Howard's time: 5.2700  #unique: 999962 #total: 10000000
   Avk1's time: 1.6100  #unique: 999962 #total: 10000000
   Avk2's time: 0.9000  #unique: 999962 #total: 10000000
  440bx's time: 2.0200  #unique: 999962 #total: 10000000
 BrunoK's time: 2.1300  #unique: 999962 #total: 10000000
BrunoK1's time: 1.5500  #unique: 999962 #total: 10000000
 
repeatMillionsCount = 2
Julkas1's time: 0.8800  #unique: 734359 #total: 2000000
Julkas2's time: 0.8600  #unique: 734359 #total: 2000000
  Akira's time: 1.0700  #unique: 734359 #total: 2000000
 Howard's time: 1.2010  #unique: 734359 #total: 2000000
   Avk1's time: 0.4200  #unique: 734359 #total: 2000000
   Avk2's time: 0.2700  #unique: 734359 #total: 2000000
  440bx's time: 0.5000  #unique: 734359 #total: 2000000
 BrunoK's time: 0.5800  #unique: 734359 #total: 2000000
BrunoK1's time: 0.4700  #unique: 734359 #total: 2000000
 
repeatMillionsCount = 4
Julkas1's time: 1.7400  #unique: 794586 #total: 4000000
Julkas2's time: 1.5600  #unique: 794586 #total: 4000000
  Akira's time: 1.8300  #unique: 794586 #total: 4000000
 Howard's time: 2.1900  #unique: 794586 #total: 4000000
   Avk1's time: 0.7000  #unique: 794586 #total: 4000000
   Avk2's time: 0.4100  #unique: 794586 #total: 4000000
  440bx's time: 0.8900  #unique: 794586 #total: 4000000
 BrunoK's time: 0.9500  #unique: 794586 #total: 4000000
BrunoK1's time: 0.7400  #unique: 794586 #total: 4000000
 
repeatMillionsCount = 6
Julkas1's time: 2.2500  #unique: 799541 #total: 6000000
Julkas2's time: 2.2400  #unique: 799541 #total: 6000000
  Akira's time: 2.5200  #unique: 799541 #total: 6000000
 Howard's time: 3.1700  #unique: 799541 #total: 6000000
   Avk1's time: 0.9900  #unique: 799541 #total: 6000000
   Avk2's time: 0.5700  #unique: 799541 #total: 6000000
  440bx's time: 1.2500  #unique: 799541 #total: 6000000
 BrunoK's time: 1.3400  #unique: 799541 #total: 6000000
BrunoK1's time: 0.9800  #unique: 799541 #total: 6000000
 
repeatMillionsCount = 8
Julkas1's time: 2.9200  #unique: 799966 #total: 8000000
Julkas2's time: 2.9200  #unique: 799966 #total: 8000000
  Akira's time: 3.2100  #unique: 799966 #total: 8000000
 Howard's time: 4.5520  #unique: 799966 #total: 8000000
   Avk1's time: 1.8860  #unique: 799966 #total: 8000000
   Avk2's time: 0.9000  #unique: 799966 #total: 8000000
  440bx's time: 1.8020  #unique: 799966 #total: 8000000
 BrunoK's time: 1.8100  #unique: 799966 #total: 8000000
BrunoK1's time: 1.3400  #unique: 799966 #total: 8000000
 
repeatMillionsCount = 10
Julkas1's time: 3.7200  #unique: 799998 #total: 10000000
Julkas2's time: 3.6100  #unique: 799998 #total: 10000000
  Akira's time: 3.9300  #unique: 799998 #total: 10000000
 Howard's time: 5.1600  #unique: 799998 #total: 10000000
   Avk1's time: 1.5500  #unique: 799998 #total: 10000000
   Avk2's time: 0.8500  #unique: 799998 #total: 10000000
  440bx's time: 1.9400  #unique: 799998 #total: 10000000
 BrunoK's time: 2.0800  #unique: 799998 #total: 10000000
BrunoK1's time: 1.4800  #unique: 799998 #total: 10000000
 
repeatMillionsCount = 12
Julkas1's time: 4.3900  #unique: 799999 #total: 12000000
Julkas2's time: 4.2600  #unique: 799999 #total: 12000000
  Akira's time: 4.6000  #unique: 799999 #total: 12000000
 Howard's time: 6.1300  #unique: 799999 #total: 12000000
   Avk1's time: 1.9300  #unique: 799999 #total: 12000000
   Avk2's time: 0.9700  #unique: 799999 #total: 12000000
  440bx's time: 2.2900  #unique: 799999 #total: 12000000
 BrunoK's time: 2.4300  #unique: 799999 #total: 12000000
BrunoK1's time: 1.7100  #unique: 799999 #total: 12000000
 
repeatMillionsCount = 14
Julkas1's time: 4.9900  #unique: 799999 #total: 14000000
Julkas2's time: 4.9600  #unique: 799999 #total: 14000000
  Akira's time: 5.3200  #unique: 799999 #total: 14000000
 Howard's time: 7.1200  #unique: 799999 #total: 14000000
   Avk1's time: 2.0900  #unique: 799999 #total: 14000000
   Avk2's time: 1.0800  #unique: 799999 #total: 14000000
  440bx's time: 2.6300  #unique: 799999 #total: 14000000
 BrunoK's time: 2.8000  #unique: 799999 #total: 14000000
BrunoK1's time: 1.9800  #unique: 799999 #total: 14000000
 
repeatMillionsCount = 16
Julkas1's time: 5.6700  #unique: 800000 #total: 16000000
Julkas2's time: 5.6200  #unique: 800000 #total: 16000000
  Akira's time: 5.9900  #unique: 800000 #total: 16000000
 Howard's time: 8.1000  #unique: 800000 #total: 16000000
   Avk1's time: 2.3600  #unique: 800000 #total: 16000000
   Avk2's time: 1.2200  #unique: 800000 #total: 16000000
  440bx's time: 2.9800  #unique: 800000 #total: 16000000
 BrunoK's time: 3.1800  #unique: 800000 #total: 16000000
BrunoK1's time: 2.2000  #unique: 800000 #total: 16000000
 
repeatMillionsCount = 18
Julkas1's time: 6.3200  #unique: 800000 #total: 18000000
Julkas2's time: 6.2900  #unique: 800000 #total: 18000000
  Akira's time: 6.6800  #unique: 800000 #total: 18000000
 Howard's time: 9.0600  #unique: 800000 #total: 18000000
   Avk1's time: 2.6700  #unique: 800000 #total: 18000000
   Avk2's time: 1.3900  #unique: 800000 #total: 18000000
  440bx's time: 3.3400  #unique: 800000 #total: 18000000
 BrunoK's time: 3.5700  #unique: 800000 #total: 18000000
BrunoK1's time: 2.4400  #unique: 800000 #total: 18000000
 
repeatMillionsCount = 20
Julkas1's time: 7.0000  #unique: 800000 #total: 20000000
Julkas2's time: 6.9700  #unique: 800000 #total: 20000000
  Akira's time: 7.4000  #unique: 800000 #total: 20000000
 Howard's time: 10.0600 #unique: 800000 #total: 20000000
   Avk1's time: 2.9500  #unique: 800000 #total: 20000000
   Avk2's time: 1.5100  #unique: 800000 #total: 20000000
  440bx's time: 3.6700  #unique: 800000 #total: 20000000
 BrunoK's time: 3.9000  #unique: 800000 #total: 20000000
BrunoK1's time: 2.6900  #unique: 800000 #total: 20000000
 

Procedure Avk1 reads the input data from stdin, Avk2 loads the entire input file into TMemoryStream.

Title: Re: Alternative set of string-to-int conversion routines
Post by: PascalDragon on January 24, 2022, 01:31:49 pm

Quote from: avk on January 21, 2022, 04:13:01 pm

Quote from: PascalDragon on January 21, 2022, 03:51:40 pm
Why do you say that generic functions need to be declared in the interface part of the unit?
Probably lagged behind life, some time ago it was like that, but now I didn’t even try to check. :-[

This works since the introduction of generic functions. Maybe you confused it with the common workaround before generic functions (though even then one does not need to use the interface section):

Code: Pascal [Select][+]

{$mode objfpc}
{$modeswitch advancedrecords}
 
type
  generic TGenFuncs<T> = record
    class function Add(aArg1, aArg2: T): T; static;
  end;
 
class function TGenFuncs.Add(aArg1, aArg2: T): T; static;
begin
  Result := aArg1 + aArg2;
end;
 
begin
  Writeln(specialize TGenFuncs<LongInt>(3, 5));
end.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 24, 2022, 01:57:25 pm

IIRC at the time when FPC-3.2.0 was not yet released, this method was the only possible one and attempts to declare a generic type in the implementation section were unconditionally suppressed by the compiler.
I just haven't had a need for something like this in a long time.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Zoran on January 24, 2022, 03:55:12 pm

I find it strange that RTL doesn't have a low-level string-to-int function which accepts only decimal (base 10) numbers.

Okay, we don't have it because Delphi doesn't have it. ::)
Which is, again... strange.

Of course it is easy to write a naive implementation, but...
Now we have to check if the two starting characters are decimal digits (not only the first, but the second as well, as val accepts 0x... notation for hex numbers) and only then call val (or TryStrToInt, or your Char2Int), which will again check for binary or hexadecimal notation.

For example:

Code: Pascal [Select][+]

uses
  SysUtils;
 
function StrToIntDecimal(const S: String; out N: Int32): Boolean;
var
  P: PAnsiChar;
begin
  P := PAnsiChar(TrimLeft(S));
  if P^ = '-' then
    Inc(P);
  case P^ of
    '0':
      if not ((P + 1)^ in [#0, '0'..'9']) then
        Exit(False);
    '1'..'9':
      ;
  otherwise
    Exit(False);
  end;
 
  Result := TryStrToInt(S, N);
end;
 

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 24, 2022, 05:04:56 pm

avk
Perhaps it is worth adding unicode versions? Are there any plans for StrToFloat?

Title: Re: Alternative set of string-to-int conversion routines
Post by: Bart on January 24, 2022, 06:44:33 pm

Quote from: Zoran on January 24, 2022, 03:55:12 pm

I find it strange that RTL doesn't have a low-level string-to-int function which accepts only decimal (base 10) numbers.

...

Of course it is easy to write a naive implementation, but...
Now we have to check if the two starting characters are decimal digits (not only the first, but the second as well, as val accepts 0x... notation for hex numbers) and only then call val (or TryStrToInt, or your Char2Int), which will again check for binary or hexadecimal notation.

Val() is the standard function for string to a number, and has been for a very long time.
Specialized and optimized functions (e.g. only base 2) should go into a separate unit IMO.
The checking for base and negativity does not take that much of the time. Most time is doen doing the multiplications and adding and checking that the result won't overflow.
It does the job, and is reliable (at least it is supposed to be and I think it is now).
TryStrToInt and family are supposed to handle the same input as Val() in the same way, so you cannot specialize them for e.g. only base 10.

Personally I think that when you require zillions of string to a number conversions in your program, you should go for a specialized solution.
Such a function can (and probably will) assume certain things about the input (always base 10, not negative, no leading whitespace etc. etc) and may raise exceptions if the input does not match the expectation. That's the trade off.
Such a function most likely will not work on (or be optimized for) all supported fpc platforms (e.g. 8-bit CPU's).
As you can see in this thread many have already implemented such a solution, just pick what you need.

Just my 2 cents.

Bart

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 24, 2022, 06:59:26 pm

@Zoran:
It seems that such function, if don't resort to various tricks, will be only slightly faster than the universal one. But it will be possible to try.

@zamtmn:
To be honest, I have not even thought about unicodestring yet.
Some implementation of the string-to-double conversion function is available in the LGenerics package in the lgJson unit, but it was made purely for the needs of JSON and without much flexibility.
Have you seen this (https://github.com/BeRo1985/pasdblstrutils) library?

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 24, 2022, 09:17:31 pm

avk
>>Have you seen this library?
no, thanks!

Title: Re: Alternative set of string-to-int conversion routines
Post by: BobDog on January 24, 2022, 11:02:34 pm

Here is an int64 conversion from ansistring.
float would be the const array extended using 1/10,1/100,1/1000 e.t.c., but i have not tried it yet.
EDITED 25/01/22

Code: Pascal [Select][+]

 
uses
sysutils;
 
function val64( const x:ansistring):int64; inline;
 
const
 p: array[1..20]of qword=(1,10,100,1000,10000,100000,1000000,10000000,100000000,1000000000,10000000000,100000000000, 1000000000000,10000000000000,
                              100000000000000,1000000000000000,10000000000000000,100000000000000000,1000000000000000000,10000000000000000000);
 var
 count:integer=0;
 sp:boolean=false;
 sg:boolean=false;
 n:integer;
 ans:int64=0;
 sign:shortint=1;
 b:byte;
    begin
    for n := length(x) downto  1 do
    begin
    count:=count+1;
    b:=byte(x[n]);
    case b of
    
    32:
    begin
    sp:=true;
    continue;
    end;
    
    45:    
    begin
    sg:=true;
    sign:=-sign;
    if (sign<>-1) then exit(0);
    continue
    end;
    
    48..57:if sp or sg  then exit(0);
    else
    exit(0);
    end;
    
     ans:= ans+p[count] * (b-48);
    end;
    exit( sign*ans);
end;
 
var
 t:int64;
 res:int64;
 code:word;
 k,i:longword;
 
 number:ansistring=' -998880776665550';
 
 
begin
 
 
for k:=1 to 5 do
begin
t:=gettickcount64;
for i:=0 to 10000000 do
res:=val64(number);
writeln(gettickcount64-t,' milliseconds val64 = ',res);
 
t:=gettickcount64;
for i:=0 to 10000000 do
val(number,res,code);
writeln(gettickcount64-t,' milliseconds val =   ',res);
writeln;
end;
writeln('Press enter to end . . .');
readln;
end.
 
 

Title: Re: Alternative set of string-to-int conversion routines
Post by: Bart on January 24, 2022, 11:43:21 pm

Any illegal string will return 0?
You can have multiple '-'?
'1 2 3' converts to 10203?
'--1' converts to -1
'-1-1-1' converts to -10101
'999999999999999999999999999999999999999999999999999999' gives Aritmetic Overflow.
'000000000000000000000' gives Range Check Error (accessing p[21]).

But yes, it's faster than Val().

This one is also fater than Val():

Code: Pascal [Select][+]

function BartsVal(const S: String): Int64; inline;
begin
  Result := 42;
end;

It's only slightly less accurate than yours, and it will never raise an exception.
And it's the answer to life, the universe and everything.

Bart

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 25, 2022, 02:13:19 am

Quote from: Bart on January 24, 2022, 11:43:21 pm

This one is also fater than Val():
Code: Pascal [Select][+][-]
function BartsVal(const S: String): Int64; inline;
begin
Result := 42;
end;

Вот оно, идеальное решение!!! :D Этот метод я не смогу ни как ускорить. :(
Eng:
Here it is, the perfect solution! :D I can't speed up this method. :(

Title: Re: Alternative set of string-to-int conversion routines
Post by: PascalDragon on January 25, 2022, 09:12:48 am

Quote from: avk on January 24, 2022, 01:57:25 pm

IIRC at the time when FPC-3.2.0 was not yet released, this method was the only possible one and attempts to declare a generic type in the implementation section were unconditionally suppressed by the compiler.

You could always declare generic types in the implementation section, so I don't know what problems exactly you remember.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 25, 2022, 10:34:33 am

It seems there is no more problem (at least so far :)), thank you.

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 25, 2022, 11:18:13 am

PascalDragon
I think it means this

Code: Pascal [Select][+]

unit Unit1;
{$mode objfpc}{$H+}
interface
 
generic procedure GTest<T>(aArg:T);
 
implementation
 
generic procedure GImpl<T>(aArg:T);
begin
  writeln(aArg);
end;
 
generic procedure GTest<T>(aArg:T);
begin
  specialize GImpl<T>(aArg);
end;
 
end.

Quote from: PascalDragon on January 21, 2022, 03:51:40 pm

If there are concrete issues with debugging then we need a reproducible example no matter the issue, cause, yes, such things should be fixed, no matter if it's about stepping through generic code correctly or not accidentally entering generic code when one doesn't want it as MarkMLI hinted at.

It looks like it's a bug lazarus https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39584
Could you comment on which way to dig here https://gitlab.com/freepascal.org/fpc/source/-/issues/39387

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 25, 2022, 01:18:38 pm

Why? You only need place

Code: Pascal [Select][+]

generic procedure GImpl<T>(aArg:T);

to interface section, and everything works. I'm not saying it's a compiler error, this is just an illustration of how I understood avk

Quote from: avk on January 20, 2022, 02:25:55 pm

As per Thaddy's remarks, tried to somehow reduce code duplication.
Generic functions need to be declared in the interface part of the unit, and since they are helpers, this is not very good.
Includes are also not suitable, because I want everything to stay in one unit.
As a result, as an experiment, I settled on macros. The code looks unreadable, of course, but it seems that it would not be much better with includes.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 25, 2022, 01:22:26 pm

That is simply not needed.

Title: Re: Alternative set of string-to-int conversion routines
Post by: PascalDragon on January 25, 2022, 01:33:30 pm

Quote from: zamtmn on January 25, 2022, 11:18:13 am

PascalDragon
I think it means this
Code: Pascal [Select][+][-]
unit Unit1;
{$mode objfpc}{$H+}
interface

generic procedure GTest<T>(aArg:T);

implementation

generic procedure GImpl<T>(aArg:T);
begin
writeln(aArg);
end;

generic procedure GTest<T>(aArg:T);
begin
specialize GImpl<T>(aArg);
end;

end.

avk had said pre-3.2.0. Pre-3.2.0 generic functions didn't exist at all.

But yes, in the example you mentioned it is indeed the case that GImpl<> needs to be declared in the interface section. This is by design.

Quote from: zamtmn on January 25, 2022, 11:18:13 am

Could you comment on which way to dig here https://gitlab.com/freepascal.org/fpc/source/-/issues/39387

No, I can't, because if I knew that, then I'd already be at the fix essentially.

(Edit: fixed quote)

Title: Re: Alternative set of string-to-int conversion routines
Post by: BobDog on January 26, 2022, 12:17:17 am

Quote from: Bart on January 24, 2022, 11:43:21 pm

Any illegal string will return 0?
You can have multiple '-'?
'1 2 3' converts to 10203?
'--1' converts to -1
'-1-1-1' converts to -10101
'999999999999999999999999999999999999999999999999999999' gives Aritmetic Overflow.
'000000000000000000000' gives Range Check Error (accessing p[21]).

But yes, it's faster than Val().

This one is also fater than Val():
Code: Pascal [Select][+][-]
function BartsVal(const S: String): Int64; inline;
begin
Result := 42;
end;

It's only slightly less accurate than yours, and it will never raise an exception.
And it's the answer to life, the universe and everything.

Bart

I have edited.
I liked a little drink myself when I was 42.
No harm in a drop of John Barleycorn, heck, I live in the country where it was invented .
Have a nice Burn's night.
"And it's the answer to life, the universe and everything."
Very true.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 26, 2022, 01:12:23 am

Quote from: BobDog on January 26, 2022, 12:17:17 am

I have edited.

что изменилось? Вы пробовали тестировать свой код?
Eng:
what changed? Have you tried testing your code?

Title: Re: Alternative set of string-to-int conversion routines
Post by: BobDog on January 26, 2022, 01:32:13 am

Quote from: Seenkao on January 26, 2022, 01:12:23 am

Quote from: BobDog on January 26, 2022, 12:17:17 am
I have edited.
что изменилось? Вы пробовали тестировать свой код?
Eng:
what changed? Have you tried testing your code?

Yes, tested 32 and 64 bit Windows on Geany ide.
It is better than it was anyway.
Not perfect by any means of course.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 26, 2022, 04:45:22 am

Quote from: BobDog on January 26, 2022, 01:32:13 am

Yes, tested 32 and 64 bit Windows on Geany ide.
It is better than it was anyway.
Not perfect by any means of course.

Вы пробовали вывести отрицательные числа? Числа, пред которыми пробел?
Eng: Have you tried displaying negative numbers? Numbers preceded by a space?

Look at here (https://forum.lazarus.freepascal.org/index.php/topic,57819.0.html) - the solution has already been provided and it works a little faster. Quite a bit faster than yours.

Title: Re: Alternative set of string-to-int conversion routines
Post by: BobDog on January 26, 2022, 10:40:00 am

Quote from: Seenkao on January 26, 2022, 04:45:22 am

Quote from: BobDog on January 26, 2022, 01:32:13 am
Yes, tested 32 and 64 bit Windows on Geany ide.
It is better than it was anyway.
Not perfect by any means of course.
Вы пробовали вывести отрицательные числа? Числа, пред которыми пробел?
Eng: Have you tried displaying negative numbers? Numbers preceded by a space?

Look at here (https://forum.lazarus.freepascal.org/index.php/topic,57819.0.html) - the solution has already been provided and it works a little faster. Quite a bit faster than yours.

Hi Seenkao
My example uses -ve number preceded by a space
(' -998880776665550')
I cannot run project files here on Geany, but I compiled your unit and used
sc_StrToInt64(number,res);
in my code instead of val.
If I set a space in front(as in my example) your function gives 0.
Maybe the solution has already been provided as you say, but surely a little input from members does no harm.
And yes, your library function is faster for int64 for strings >5 characters, and a little slower for smaller strings.
Thank you for the unit.
I will try out some code for floats, maybe later.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 26, 2022, 03:00:39 pm

Quote from: BobDog on January 26, 2022, 10:40:00 am

I cannot run project files here on Geany, but I compiled your unit and used

Edit the geany configuration file. filetype_extensions.conf
You may also want to add the missing keywords. That is another conf file that I forgot, but I will look it up.
You may also want to edit the templates, add a unit.pas etc, and remove the licence if so required.

There are many more options to let geany play nice with Pascal sources.
I am thinking about a wiki entry, since I tend to use geany over fp for simple fpc code.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 26, 2022, 03:46:04 pm

As an experiment, I added a couple of functions(TryDecimals2Int()) that only accept strings in decimal notation using an idea I saw on the forum.

Some quick and dirty benchmark:

Code: Pascal [Select][+]

program bench2;
 
{$MODE OBJFPC}{$H+}
 
uses
  SysUtils, DateUtils, StrUtils, Str2IntAlter;
 
var
  a: TStringArray = nil;
  Trash: TStringArray = (
    ' --', ' ++', ' +-',' --0', ' ++0', ' +-0',' -0-1', ' +0--1', ' 0-1-1',
    '9223372036854775808', '-9223372036854775809');
 
function NextInt: string;
begin
  case Random(4) of
    0: Result := '  '#9'-' + Random(High(Int64)).ToString;
    1: Result := '  '#9'+' + Random(High(Int64)).ToString;
    2: Result := '  '#9'-' + Random(Int64(High(Dword))).ToString;
  else
    Result := '  '#9'+' + Random(Int64(High(Dword))).ToString;
  end;
end;
 
procedure GenTestData;
var
  I: Integer;
const
  TestSize = 1000;
begin
  SetLength(a, TestSize);
  for I := 0 to High(a) do
    a[I] := NextInt;
end;
 
procedure RunVal;
var
  I, J, r, c: Integer;
  Start: TTime;
  v, Score: Int64;
begin
  for I := 0 to High(Trash) do
    begin
      Val(Trash[I], v, c);
      if c = 0 then
        begin
          WriteLn('Val() accepts trash');
          exit;
        end;
    end;
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      begin
        Val(a[I], v, c);
        Inc(r, Ord(c <> 0));
      end;
  Score := MillisecondsBetween(Time, Start);
  WriteLn('Val(), score:           ', Score);
  WriteLn('rejected:               ', r);
end;
 
procedure RunAlt;
var
  I, J, r: Integer;
  Start: TTime;
  v, Score: Int64;
begin
  for I := 0 to High(Trash) do
    if TryChars2Int(Trash[I][1..Length(Trash[I])], v) then
      begin
        WriteLn('TryChars2Int accepts trash');
        exit;
      end;
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      if not TryChars2Int(a[I][1..Length(a[I])], v) then
        Inc(r);
  Score := MillisecondsBetween(Time, Start);
  WriteLn('TryChars2Int, score:    ', Score);
  WriteLn('rejected:               ', r);
end;
 
procedure RunAltDec;
var
  I, J, r: Integer;
  Start: TTime;
  v, Score: Int64;
begin
for I := 0 to High(Trash) do
  if TryDecimals2Int(Trash[I][1..Length(Trash[I])], v) then
    begin
      WriteLn('TryDecimals2Int accepts trash');
      exit;
    end;
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      if not TryDecimals2Int(a[I][1..Length(a[I])], v) then
        Inc(r);
  Score := MillisecondsBetween(Time, Start);
  WriteLn('TryDecimals2Int, score: ', Score);
  WriteLn('rejected:               ', r);
end;
 
begin
  Randomize;
  GenTestData;
 
  RunVal;
  WriteLn;
  RunAlt;
  WriteLn;
  RunAltDec;
  WriteLn;
 
  WriteLn('Press any key to exit...');
  ReadLn;
end.
 

output x64:

Code: Text [Select][+]

Val(), score:           2445
rejected:               0
 
TryChars2Int, score:    923
rejected:               0
 
TryDecimals2Int, score: 770
rejected:               0
 
Press any key to exit...
 

output x32:

Code: Text [Select][+]

Val(), score:           4598
rejected:               0
 
TryChars2Int, score:    1891
rejected:               0
 
TryDecimals2Int, score: 1178
rejected:               0
 
Press any key to exit...
 

I would like to remind that in this thread we are talking about functions to replace Val(), and therefore these should parse strings according to the same rules as Val().

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 26, 2022, 06:43:45 pm

BobDog, на самом деле это не критично. Не такая большая разница, чтоб обращать на разницу в исполнении.
Да, функции предоставленные мной проверяют числа в чистом виде, без пробелов и прочего (исключая знак минус), всё это можно проверить перед функцией и убрать по надобности. Потому и выдало ноль, потому что встретился пробел.
Google translate:
BobDog, actually it is not critical. Not such a big difference to pay attention to the difference in performance.
Yes, the functions provided by me check numbers in their pure form, without spaces and other things (excluding the minus sign), all this can be checked before the function and removed as needed. That's why it gave out zero, because there was a gap.

avk, я не думал, что этот метод сильно ускорит работу с числами. Я ошибался! Увеличение примерно в 1.5 раза. ))) Но только для длинных чисел.
Google translate:
avk, I did not think that this method would greatly speed up the work with numbers. I was wrong! An increase of about 1.5 times. ))) But only for long numbers.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 27, 2022, 09:30:40 am

Quote from: Seenkao on January 26, 2022, 06:43:45 pm

avk, я не думал, что этот метод сильно ускорит работу с числами. Я ошибался! Увеличение примерно в 1.5 раза. ))) Но только для длинных чисел.
Google translate:
avk, I did not think that this method would greatly speed up the work with numbers. I was wrong! An increase of about 1.5 times. ))) But only for long numbers.

Досадно, но я был прав изначально... как только мы начинаем работать с настоящими строками, а не массивом из Pchar сразу же всё теряется.
При необходимости можно полностью уйти от использования строк и тогда скорость может в два раза возрастёт. Вроде как пользовательские процедуры должны быть, а не для собственного использования. Хотя это на ваше усмотрение.
google translate:
It's a shame, but I was right initially... as soon as we start working with real strings, and not an array from Pchar, everything is immediately lost.
If necessary, you can completely avoid using strings and then the speed can double. Like as the user procedures should be, instead of for own use. Although this is up to you.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 28, 2022, 10:20:12 am

Added more overloaded functions for converting strings in decimal notation to integers.

@Seenkao, could you clarify what you are talking about?

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 28, 2022, 01:23:56 pm

Quote from: Seenkao on January 27, 2022, 09:30:40 am

as soon as we start working with real strings, and not an array from Pchar, everything is immediately lost.
If necessary, you can completely avoid using strings and then the speed can double.

On the contrary: Pascal strings are more flexible and faster than PChars. That is because strlen (slow, O(n)) to determine length for a pchar vs reading the size of the string at once in Pascal strings (Fast, O(1)). Also PChars can not contain embedded zero's, you would need a char array for that and with a known length. Pascal strings can contain embedded zero's. Using PChars mixed with Pascal strings can lead to strange results, as I already demonstrated.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 30, 2022, 06:57:20 am

Added some overloads to support unicodestring.
A quick and dirty benchmark against Val() on my machine looks like this:

Code: Text [Select][+]

SInt32:
Val(), score:        3559
rejected:            0
TryChars2Int, score: 780
rejected:            0
 
SInt64:
Val(), score:        4820
rejected:            0
TryChars2Int, score: 1232
rejected:            0
 
UInt32:
Val(), score:        5195
rejected:            0
TryChars2Int, score: 811
rejected:            0
 
UInt64:
Val(), score:        8549
rejected:            0
TryChars2Int, score: 1248
rejected:            0
 

Title: Re: Alternative set of string-to-int conversion routines
Post by: zamtmn on January 30, 2022, 06:59:42 am

Thanks!
Please add unicode version to bench (TryChars2Int(widechars) vs Val(unicodestring))

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 30, 2022, 07:19:05 am

Located in the folder bench_uni, this is its output.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Thaddy on January 30, 2022, 10:24:58 am

Works well, and tnx to include proper tests!

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 30, 2022, 03:59:51 pm

Quote from: avk on January 28, 2022, 10:20:12 am

@Seenkao, could you clarify what you are talking about?

О том, что пользователь будет использовать StrToInt('-758343');, а не StrToInt('-758343'[1..length('-758343')]. Я поздравляю вас, вы добились скорости. Но вы потеряли в удобстве использования. А когда возвращаем к обычному представлению String вся ваша скорость теряется.
google translate:
That the user will use StrToInt('-758343'); rather than StrToInt('-758343'[1..length('-758343')] I congratulate you, you have achieved speed, but you have lost in usability, and when we return to the usual String representation, all your speed is lost.

Quote from: Thaddy on January 28, 2022, 01:23:56 pm

On the contrary: Pascal strings are more flexible and faster than PChars. That is because strlen (slow, O(n)) to determine length for a pchar vs reading the size of the string at once in Pascal strings (Fast, O(1)). Also PChars can not contain embedded zero's, you would need a char array for that and with a known length. Pascal strings can contain embedded zero's. Using PChars mixed with Pascal strings can lead to strange results, as I already demonstrated.

Уж извиняюсь, взял код из того что я сделал: fast_StrToInt (https://github.com/Seenkao/fast_StrToInt)
Eng: Sorry, I took the code from what I did: fast_StrToInt (https://github.com/Seenkao/fast_StrToInt)

Code: Pascal [Select][+]

function geStrToInt(const Str: String; out Value: maxIntVal; Size: LongWord = isInteger): Boolean;
var
  lenStr, i: maxUIntVal;
  m, n, z: maxUIntVal;
  useParametr: PgeUseParametr;
  IntMinus: Boolean;
  correct: maxUIntVal = 0;
label
  jmpEndStr, loopZero;
begin
  {$push}
  {$Q-}{$R-}
  Result := False;
  Value := 0;
  if Size > maxSize then
    Exit;
  IntMinus := False;
  lenStr := Length(Str);
  if lenStr = 0 then
    exit;
  i := 1;
  m := Byte(Str[i]);
 
  if (lenStr = 1) and (m = 48) then
  begin
    Result := True;
    exit;
  end;
  if m = 45 then
  begin
    if lenStr = 1 then
      exit;
    IntMinus := True;
    inc(i);
    m := Byte(Str[2]);
  end;
 
loopZero:
  if m = 48 then
  begin
    inc(i);
    inc(correct);
    m := Byte(Str[i]);
    goto loopZero;
  end;
  
  inc(i);
  m := m - 48;
  if m > 9 then
    exit;
 
  useParametr := @allIntParametr[Size];
  if i > lenStr then
  begin
    z := 0;
    goto jmpEndStr;
  end;
  if (lenStr - correct) > useParametr^.maxLen then
    Exit;
  while i < lenStr do
  begin
    n := (Byte(Str[i]) - 48);
    if n > 9 then
      Exit;
    m := m * 10 + n;
    inc(i);
  end;
 
  if m > useParametr^.maxNumDiv10 then
    exit;
  m := m * 10;
  z := Byte(Str[i]) - 48;
  if z > 9 then
    exit;
 
jmpEndStr:
  if IntMinus then
    n := useParametr^.maxNumeric + 1 - m
  else
    n := useParametr^.maxNumeric - m;
  if z > n then
    exit;
 
  if IntMinus then
    Value := - m - z
  else
    Value := m + z;
  Result := true;
  {$pop}
end;

Переделал:
Remade:

Code: Pascal [Select][+]

function geStrToInt(const Str: array of char; out Value: maxIntVal; Size: LongWord = isInteger): Boolean;
var
  lenStr, i: maxUIntVal;
  m, n, z: maxUIntVal;
  useParametr: PgeUseParametr;
  IntMinus: Boolean = false;
  correct: maxUIntVal = 0;
  v: LongWord;
  pv: PByte;
label
  jmpEndStr, loopZero;
begin
  {$push}
  {$Q-}{$R-}
  Result := False;
  Value := 0;
  if Size > maxSize then
    Exit;
  lenStr := Length(Str);
  if lenStr = 0 then
    exit;
  i := 0;
  m := Byte(Str[i]);
  if (lenStr = 1) and (m = 48) then
  begin
    Result := True;
    exit;
  end;
  if m = 45 then
  begin
    if lenStr = 1 then
      exit;
    IntMinus := True;
    inc(i);
    m := Byte(Str[2]);
  end;
loopZero:
  if m = 48 then
  begin
    inc(i);
    inc(correct);
    m := Byte(Str[i]);
    goto loopZero;
  end;  
  inc(i);
  m := m - 48;
  if m > 9 then
    exit;
  useParametr := @allIntParametr[Size];
  if i > lenStr - 1 then
  begin
    z := 0;
    goto jmpEndStr;
  end;
 
  if (lenStr - correct) > useParametr^.maxLen then
    Exit;
  dec(lenStr);
  while i < lenStr do
  begin
    n := (Byte(Str[i]) - 48);
    if n > 9 then
      Exit;
    m := m * 10 + n;
    inc(i);
  end;
  if m > useParametr^.maxNumDiv10 then
    exit;
  m := m * 10;
  z := Byte(Str[i]) - 48;
  if z > 9 then
    exit;
 
jmpEndStr:
  if IntMinus then
    n := useParametr^.maxNumeric + 1 - m
  else
    n := useParametr^.maxNumeric - m;
  if z > n then
    exit;
 
  if IntMinus then
    Value := - m - z
  else
    Value := m + z;
  Result := true;
  {$pop}
end;

И при тестах в первом варианте (работа со String) function geStrToInt(const Str: String; out Value: maxIntVal; Size: LongWord = isInteger): Boolean; я получаю меньшую скорость обработки, чем во втором (работа с массивом PChar) function geStrToInt(const Str: array of char; out Value: maxIntVal; Size: LongWord = isInteger): Boolean;. Вторая функция работает в два с лишним раза быстрее!!!
Не хотите прокомментировать что происходит? И почему вы утверждаете обратное?
Google translate:
And when testing in the first option (working with String) function geStrToInt(const Str: String; out Value: maxIntVal; Size: LongWord = isInteger): Boolean; I get a lower processing speed than in the second ( working with a PChar array) function geStrToInt(const Str: array of char; out Value: maxIntVal; Size: LongWord = isInteger): Boolean;. The second function works more than twice as fast!!!
Would you like to comment on what's going on? And why are you saying otherwise?

avk, по сути я приложил пример (и не я один), в котором простые вещи обходят слишком замудрёные. Эту функцию function geStrToInt(const Str: array of char; out Value: maxIntVal; Size: LongWord = isInteger): Boolean; сложно обойти по скорости, там уже практически нечего ускорять. Даже используя ваш метод с QWord и DWord.
Google translate:
avk, in fact, I attached an example (and I'm not the only one) in which simple things bypass too complicated ones. This function function geStrToInt(const Str: array of char; out Value: maxIntVal; Size: LongWord = isInteger): Boolean; is hard to get around in terms of speed, there is practically nothing to speed up. Even using your method with QWord and DWord.

My test:

Quote

StrToInt standard 7.8757494776072852E+000
StrToInt made by me 1.5074538423153692E+000
StrToInt made by avk 2.0989100899762976E+000

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 30, 2022, 04:56:22 pm

First of all, I would like to remind you that this section of the forum is called "Third Party announcements", and this topic discusses a specific library.
If you want to discuss something different, it is better to create a new thread for this.
As for Str2IntAlter, it don't seem to be restricting the user in any way in choosing a particular version of the function.
One could try to compare their performance(adding function from fast_StrToInt at the same time), for example:

Code: Pascal [Select][+]

program bench;
 
{$MODE OBJFPC}{$H+}
 
uses
  SysUtils, DateUtils, StrUtils, Str2IntAlter, ge_external_Utils;
 
function NextRandomQWord: QWord;
begin
  Result := QWord(Random(Int64($100000000))) shl 32 or QWord(Random(Int64($100000000)));
end;
 
function NextUInt64: string;
begin
  case Random(4) of
    0: Result := ' '#9'+%' + NextRandomQWord.ToBinString;
    1: Result := ' '#9'+&' + OctStr(NextRandomQWord, 22);
    2: Result := ' '#9'+' + NextRandomQWord.ToString;
  else
    Result := ' '#9'+$' + NextRandomQWord.ToHexString;
  end;
end;
 
var
  a: TStringArray = nil;
 
procedure GenTestData;
var
  I: Integer;
const
  TestSize = 1000;
begin
  SetLength(a, TestSize);
  for I := 0 to High(a) do
    a[I] := NextUInt64;
end;
 
procedure RunChars;
var
  I, J, r: Integer;
  Start: TTime;
  Score: Int64;
  v: QWord;
begin
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      if not TryChars2Int(a[I][1..Length(a[I])], v) then
        Inc(r);
  Score := MillisecondsBetween(Time, Start);
  WriteLn('TryChars2Int, score:  ', Score);
  WriteLn('rejected:             ', r);
end;
 
procedure RunStr;
var
  I, J, r: Integer;
  Start: TTime;
  Score: Int64;
  v: QWord;
begin
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      if not TryStr2Int(a[I], v) then
        Inc(r);
  Score := MillisecondsBetween(Time, Start);
  WriteLn('TryStr2Int, score:    ', Score);
  WriteLn('rejected:             ', r);
end;
 
procedure RunSeen;
var
  I, J, r: Integer;
  Start: TTime;
  Score: Int64;
  v: QWord;
begin
  r := 0;
  Start := Time;
  for J := 1 to 30000 do
    for I := 0 to High(a) do
      if not sc_StrToQWord(a[I], v) then
        Inc(r);
  Score := MillisecondsBetween(Time, Start);
  WriteLn('sc_StrToQWord, score: ', Score);
  WriteLn('rejected:             ', r);
end;
 
begin
  Randomize;
  GenTestData;
 
  RunChars;
  WriteLn;
  RunStr;
  WriteLn;
  RunSeen;
  WriteLn;
 
  WriteLn('Press enter to exit...');
  ReadLn;
end.
 

I get such result:

Code: Text [Select][+]

TryChars2Int, score:  1762
rejected:             0
 
TryStr2Int, score:    1740
rejected:             0
 
sc_StrToQWord, score: 130
rejected:             30000000
 
Press enter to exit...
 

So I still don't understand what you mean.

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 30, 2022, 06:36:02 pm

Code: [Select]

[quote author=avk link=topic=57934.msg432772#msg432772 date=1643558182]
sc_StrToQWord, score: 130
rejected:             30000000

[/quote]
Я изначально не стремился обрабатывать дополнительные символы, кроме ведущих для чисел. Моя библиотека принимает именно числа. А не входящую билеберду. Извиняюсь!
Именно по этой причине, вы видите лишь "ошибки". Если я вам в числа залью буквы, вы так же эти ошибки увидите в своих функциях.
Всю билеберду, можно убрать до вызова. Это не займёт много времени исполнения, а при работе с числами, пользователь как раз обрабатывает символы для чисел. И при правильной программе, не будет ни каких посторонних символов. Кроме ведущих нулей, и выделения системы счисления. Я изначально рассчитывал именно на скорость. Но не думал что настолько получу ускорение.
google translate:
I didn't initially aim to handle additional characters other than leading ones for numbers. My library accepts exactly numbers. And not the incoming billebird. I'm sorry!
It is for this reason that you only see "errors". If I put letters in numbers for you, you will also see these errors in your functions.
All billeberd can be removed before the call. This will not take much execution time, and when working with numbers, the user is just processing the characters for numbers. And with the right program, there will be no extraneous characters. In addition to leading zeros, and the selection of the number system. I originally counted on the speed. But I did not think that I would get such an acceleration.

Quote

So I still don't understand what you mean.

Когда я писал, у вас была выложена только функция function TryDecimals2Int(const a: array of char; out aValue: LongWord): Boolean; без String! Я обрадовался когда увидел, что ваша функция обошла мою по скорости! Но позже, я её переписал для String и вся скорость потерялась на этом этапе.
После этого я переписал свою под массив PChar. И увидел в чём реальное ускорение было. А ускорение было не в функции, а в том, чтоб не использовать String в функции.
Вероятно потому вы и не понимаете о чём я пишу. Функций function TryDecStr2Int(const s: string; out aValue: LongWord): Boolean; у вас тогда ещё не было.
google translate:
When I wrote, you only had function TryDecimals2Int(const a: array of char; out aValue: LongWord): Boolean; without String! I was delighted when I saw that your function bypassed mine in speed! But later, I rewrote it for String and all the speed was lost at this stage.
After that, I rewrote mine under the PChar array. And I saw what the real acceleration was. And the acceleration was not in the function, but in not using String in the function.
That's probably why you don't understand what I'm talking about. Functions function TryDecStr2Int(const s: string; out aValue: LongWord): Boolean; you did not yet have.

Но у меня возникает вопрос, а почему в вашем коде не идёт ускорение при использовании массива PChar? Функция TryChars2Int(a[1..Length(a)], v) работает явно быстрее. На каком моменте в коде начинает проявляться перевод String в массив PChar?
TryChars2Int(a[1..Length(a)], v) работает в два раза быстрее чем TryStr2Int(a, v). Почему в вашем коде это не работает быстрее?
google translate:
But I have a question, why is there no acceleration in your code when using the PChar array? The function TryChars2Int(a[1..Length(a)], v) is clearly faster. At what point in the code does the translation of String into a PChar array begin to appear?
TryChars2Int(a[1..Length(a)], v) is twice as fast as TryStr2Int(a, v) . Why doesn't it work faster in your code?

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 30, 2022, 07:17:41 pm

The library that is discussed in the current topic tries to be compatible with Val().
Other options are not interesting.

Title: Re: Alternative set of string-to-int conversion routines
Post by: BobDog on January 30, 2022, 08:10:10 pm

Val, of course, also covers float numbers.
Actually fp val is quite fast compared to C.
it is also much faster than freebasic val.

Code: Pascal [Select][+]

uses
sysutils;
function atof(p:pchar):double;cdecl  external 'msvcrt.dll' name 'atof';
function strtod (p:pchar; b :pointer):double cdecl  external 'msvcrt.dll' name 'strtod';
var
 t:int64;
 res:double;
 code:word;
 k,i:longword;
 number: ansistring='  -12345.6789';
 b:^pbyte=nil;
 begin
 
for k:=1 to 5 do
begin
t:=gettickcount64;
for i:=0 to 1000000 do
res:=strtod(pchar(number),b);
writeln(gettickcount64-t,' milliseconds strtod = ',res);
 
t:=gettickcount64;
for i:=0 to 1000000 do
 
val(number,res,code);
writeln(gettickcount64-t,' milliseconds  val =   ',res);
 
t:=gettickcount64;
for i:=0 to 1000000 do
res:=atof(pchar(number));
writeln(gettickcount64-t,' milliseconds   atof = ',res);
writeln;
end;
writeln('Press enter to end . . .');
readln;
end.
 
 

It shouldn't be too hard to include floats in pascal conversions, but searching for the decimal point will slow things down.

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 31, 2022, 07:26:54 am

Quote from: BobDog on January 30, 2022, 08:10:10 pm

Val, of course, also covers float numbers.
...

Does this have anything to do with "set of string-to-int conversion routines"?

Title: Re: Alternative set of string-to-int conversion routines
Post by: Seenkao on January 31, 2022, 08:21:03 am

Quote from: avk on January 30, 2022, 07:17:41 pm

The library that is discussed in the current topic tries to be compatible with Val().
Other options are not interesting.

Quote from: avk on January 31, 2022, 07:26:54 am

Quote from: BobDog on January 30, 2022, 08:10:10 pm

Val, of course, also covers float numbers.
...

Does this have anything to do with "set of string-to-int conversion routines"?

Мда... сами себе противоречите? )))
И ваши функции не совместимы с Val. Потому что Val - это одна функция, работающая с разнообразным множеством чисел. Ваши функции могут быть лишь совместимы с производными от Val. До тех пор, пока это не станет одной функцией.
Это так, для информации. Успехов!
google translate:
Hmm ... contradict yourself? )))
And your functions are not compatible with Val. Because Val is one function that works with a diverse set of numbers. Your functions can only be compatible with Val derivatives. Until it becomes one function.
This is so, for information. Good luck!

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on January 31, 2022, 08:53:27 am

Thank you, I would like to hope that there are still no contradictions here.

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on February 01, 2022, 12:38:03 pm

Quote from: MarkMLl on January 21, 2022, 04:49:00 pm

Quote from: PascalDragon on January 21, 2022, 03:51:40 pm
One potential issue might be (not tested): the generic code is compiled without debug information, but code that uses it is compiled with and thus the specialization will have debug information as well. Thus when stepping through the latter code one might suddenly land in the generic code which one assumed is without debug information.

On the other hand, I invariably use a locally-built FPC+Lazarus and looking at my build log for FPC 3.2.0 I see

make NO_GDB=1 CPU_TARGET=i386 OPT='-V3.0.4 -O- -gl -Xs- -vt' all |/usr/local/bin/fpc-filter-vt

Lazarus+LCL though is built with default options, so it could be there.

I'll try to keep an eye open for what I think I've seen, and will report back.

I've caught it in the act. If a method is declared virtual+abstract and then overridden in a subclass, stepping into the method's implementation takes you through fpc_check_object_ext() which is in generic.inc. So it gives the impression that generic stuff is being pulled in, even if that's not in fact happening.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: PascalDragon on February 01, 2022, 01:38:03 pm

Quote from: MarkMLl on February 01, 2022, 12:38:03 pm

Quote from: MarkMLl on January 21, 2022, 04:49:00 pm
Quote from: PascalDragon on January 21, 2022, 03:51:40 pm
One potential issue might be (not tested): the generic code is compiled without debug information, but code that uses it is compiled with and thus the specialization will have debug information as well. Thus when stepping through the latter code one might suddenly land in the generic code which one assumed is without debug information.

On the other hand, I invariably use a locally-built FPC+Lazarus and looking at my build log for FPC 3.2.0 I see

make NO_GDB=1 CPU_TARGET=i386 OPT='-V3.0.4 -O- -gl -Xs- -vt' all |/usr/local/bin/fpc-filter-vt

Lazarus+LCL though is built with default options, so it could be there.

I'll try to keep an eye open for what I think I've seen, and will report back.

I've caught it in the act. If a method is declared virtual+abstract and then overridden in a subclass, stepping into the method's implementation takes you through fpc_check_object_ext() which is in generic.inc. So it gives the impression that generic stuff is being pulled in, even if that's not in fact happening.

Well, not everything that's named “generic” does involve generics. ;)

Though I wonder why it steps through fpc_check_object_check? Do you have the RTL compiled with debug information?

Sidenote: This check is only active if either object checks or range checks are enabled

Title: Re: Alternative set of string-to-int conversion routines
Post by: MarkMLl on February 01, 2022, 02:12:25 pm

Quote from: PascalDragon on February 01, 2022, 01:38:03 pm

Well, not everything that's named “generic” does involve generics. ;)

Note that I'm not complaining- particularly while I can see what I've seen.

Quote
Though I wonder why it steps through fpc_check_object_check? Do you have the RTL compiled with debug information?

Compiled with -O- -gl -Xs- which I presume would be sufficient... that's a legacy of when I'd also got SPARC etc. running years ago and some things were a bit less than robust.

Quote
Sidenote: This check is only active if either object checks or range checks are enabled

Also enabled at the project level, although probably not for the LCL etc.

MarkMLl

Title: Re: Alternative set of string-to-int conversion routines
Post by: avk on February 02, 2022, 05:43:17 pm

This is of course so important for the current topic.

I don't even know how to thank you.