Recent

Author Topic: [SOLVED] Variable addresses in SRAM  (Read 6407 times)

dseligo

  • Hero Member
  • *****
  • Posts: 1194
Re: [SOLVED] Variable addresses in SRAM
« Reply #30 on: October 05, 2022, 10:09:54 am »
Eventually I discovered what consumed my SRAM: it was string concatenation.
After confirming that stack overwrites allocated data, with the compiler switch -at (together with -al) I easily found out how much SRAM was consumed for temporary variables in every unit. For each concatenation of strings, 256 bytes were consumed and in some units more than 1KB was reserved.
I solved it by making my own string concatenation procedures and now even AVRs with 2KB SRAM are enough for this project of mine.

Bottom line: be careful with strings in the embedded world.

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: [SOLVED] Variable addresses in SRAM
« Reply #31 on: October 05, 2022, 10:54:16 am »
It is better to stick to shortstring instead of ansistring or others. On most if not all platforms this reserves just 256 byte or even less if declared properly.
E.g. if you know before hand what the maximum string length is, you can declare a shortstring type with the appropriate size:
Code: Pascal  [Select][+][-]
  1. type Tmystring = string[16];
will occupy just 17 bytes on embedded targets. (one for size)
Specialize a type, not a var.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: [SOLVED] Variable addresses in SRAM
« Reply #32 on: October 05, 2022, 10:55:58 am »
It is better to stick to shortstring instead of ansistring or others. On most if not all platforms this reserves just 256 byte or even less if declared properly.
E.g. if you know before hand what the maximum string length is, you can declare a shortstring type with the appropriate size:
Code: Pascal  [Select][+][-]
  1. type Tmystring = string[16];
will occupy just 17 bytes on embedded targets. (one for size)

He'd already assured us he was (in response to my reply #3 in this thread). Something's not right...

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dseligo

  • Hero Member
  • *****
  • Posts: 1194
Re: [SOLVED] Variable addresses in SRAM
« Reply #33 on: October 05, 2022, 11:37:57 am »
It is better to stick to shortstring instead of ansistring or others. On most if not all platforms this reserves just 256 byte or even less if declared properly.
E.g. if you know before hand what the maximum string length is, you can declare a shortstring type with the appropriate size:
Code: Pascal  [Select][+][-]
  1. type Tmystring = string[16];
will occupy just 17 bytes on embedded targets. (one for size)

Yes, I use shortstring with appropriate size and it occupies size + 1. Problem is when strings are concatenated.

Consider this program:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. {$mode ObjFPC}
  4.  
  5. var s1: String[10];
  6.  
  7. begin
  8.   s1 := 'test';
  9.   s1 := s1 + '123';
  10. end.

In assembler listing you see:
Code: ASM  [Select][+][-]
  1. ...
  2. # Temps allocated between r28+2 and r28+258
  3. ...
  4. # [8] s1 := 'test';
  5.         ldi     r18,lo8(_sPROJECT1s_Ld1)
  6.         ldi     r21,hi8(_sPROJECT1s_Ld1)
  7.         ldi     r24,lo8(U_sPsPROJECT1_ss_S1)
  8.         ldi     r25,hi8(U_sPsPROJECT1_ss_S1)
  9.         ldi     r22,10
  10.         mov     r23,r1
  11.         mov     r20,r18
  12.         call    fpc_shortstr_to_shortstr
  13. # Temp 2,256 allocated
  14. # [9] s1 := s1 + '123';
  15.         ldi     r18,lo8(_sPROJECT1s_Ld2)
  16.         ldi     r19,hi8(_sPROJECT1s_Ld2)
  17.         ldi     r20,lo8(U_sPsPROJECT1_ss_S1)
  18.         ldi     r21,hi8(U_sPsPROJECT1_ss_S1)
  19.         ldi     r24,lo8(2)
  20.         ldi     r25,hi8(2)
  21.         add     r24,r28
  22.         adc     r25,r29
  23.         ldi     r22,-1
  24.         mov     r23,r1
  25.         call    fpc_shortstr_concat
  26.         ldi     r20,lo8(2)
  27.         ldi     r21,hi8(2)
  28.         add     r20,r28
  29.         adc     r21,r29
  30.         ldi     r24,lo8(U_sPsPROJECT1_ss_S1)
  31.         ldi     r25,hi8(U_sPsPROJECT1_ss_S1)
  32.         ldi     r22,10
  33.         mov     r23,r1
  34.         call    fpc_shortstr_to_shortstr
  35. # Temp 2,256 released
  36. # [10] end.
  37.  

My problem was with '# Temp 2,256 allocated' - in addition to space variable occupies, FP allocated 256 bytes more in this example. It was for concatenation.

I attached complete asm listing.

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: [SOLVED] Variable addresses in SRAM
« Reply #34 on: October 05, 2022, 12:09:54 pm »
But concatenations *must* assume  string[255] unless you declare an intermediate string type with the expected total length.
The compiler will respect that.
Specialize a type, not a var.

ccrause

  • Hero Member
  • *****
  • Posts: 845
Re: [SOLVED] Variable addresses in SRAM
« Reply #35 on: October 05, 2022, 01:14:41 pm »
The compiler uses the fpc_shortstr_concat helper function to copy the source strings into the destination string [in this context string means shortstring for brevity].  The destination string is passed to this function via a var parameter with a shortstring type, i.e. the compiler has to create a temporary variable of type shortstring (as Thaddy mentioned) which will take up 256 bytes of stack space.

To avoid this large temporary variable, consider using move to copy the string fragments directly into the destination.  This way there is no temporary variable at play, but the programmer has to handle a couple more responsibilities to ensure you don't unintentionally overwrite data, find the correct offset into the destination, ensure the destination is correctly sized, and update the string length after adding data.  Basically re-implement the functionality of fpc_shortstr_concat, but without the use of a temporary variable.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: [SOLVED] Variable addresses in SRAM
« Reply #36 on: October 05, 2022, 01:41:41 pm »
The compiler uses the fpc_shortstr_concat helper function to copy the source strings into the destination string [in this context string means shortstring for brevity].  The destination string is passed to this function via a var parameter with a shortstring type, i.e. the compiler has to create a temporary variable of type shortstring (as Thaddy mentioned) which will take up 256 bytes of stack space.

If that is the case, why is that code comparing the address of the destination with those of the source strings?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dseligo

  • Hero Member
  • *****
  • Posts: 1194
Re: [SOLVED] Variable addresses in SRAM
« Reply #37 on: October 05, 2022, 04:22:15 pm »
To avoid this large temporary variable, consider using move to copy the string fragments directly into the destination.  This way there is no temporary variable at play, but the programmer has to handle a couple more responsibilities to ensure you don't unintentionally overwrite data, find the correct offset into the destination, ensure the destination is correctly sized, and update the string length after adding data.  Basically re-implement the functionality of fpc_shortstr_concat, but without the use of a temporary variable.

I did that, although I didn't use built-in move (I am careful about string sizes, but since I use strings only for LCD display I just cut if it doesn't fit):

Code: Pascal  [Select][+][-]
  1. procedure MyStringAddLeft(constref s1: TLCDString; var s: TLCDString);
  2. const MaxStrSize = SizeOf(TLCDString) - 1;
  3. var bLengthLeft, bFinalLength, bCounter, bIndex: Byte;
  4. begin
  5.   // get left string length and calculate final length
  6.   bLengthLeft := Length(s1);
  7.   If bLengthLeft > MaxStrSize then bLengthLeft := MaxStrSize;
  8.   bFinalLength := bLengthLeft + Length(s);
  9.   If bFinalLength > MaxStrSize then bFinalLength := MaxStrSize;
  10.   s[0] := Chr(bFinalLength);
  11.  
  12.   // move original string to the right, cut right side if it doesn't fit
  13.   bIndex := bFinalLength - bLengthLeft;
  14.   For bCounter := bFinalLength downto bLengthLeft + 1 do begin
  15.     s[bCounter] := s[bIndex];
  16.     dec(bIndex);
  17.   end;
  18.  
  19.   // copy string to the left side
  20.   For bCounter := 1 to bLengthLeft do
  21.     s[bCounter] := s1[bCounter];
  22. end;
  23.  
  24. procedure MyStringAddRight(var s: TLCDString; constref s1: TLCDString);
  25. const MaxStrSize = SizeOf(TLCDString) - 1;
  26. var bLengthLeft, bFinalLength, bCounter, bIndex: Byte;
  27. begin
  28.   // get left string length and calculate final length
  29.   bLengthLeft := Length(s);
  30.   bFinalLength := bLengthLeft + Length(s1);
  31.   If bFinalLength > MaxStrSize then bFinalLength := MaxStrSize;
  32.   s[0] := Chr(bFinalLength);
  33.   bIndex := 1;
  34.  
  35.   // copy string to the right side
  36.   For bCounter := bLengthLeft + 1 to bFinalLength do begin
  37.     s[bCounter] := s1[bIndex];
  38.     inc(bIndex);
  39.   end;
  40. end;

ccrause

  • Hero Member
  • *****
  • Posts: 845
Re: [SOLVED] Variable addresses in SRAM
« Reply #38 on: October 05, 2022, 04:35:11 pm »
The compiler uses the fpc_shortstr_concat helper function to copy the source strings into the destination string [in this context string means shortstring for brevity].  The destination string is passed to this function via a var parameter with a shortstring type, i.e. the compiler has to create a temporary variable of type shortstring (as Thaddy mentioned) which will take up 256 bytes of stack space.

If that is the case, why is that code comparing the address of the destination with those of the source strings?

MarkMLl
This is a good question.  Since fpc_shortstr_concat can handle the situation where either of the two sources can also be the destination, the compiler can omit the temp in this case.  However, in the disassembly listing @dseligo showed, the compiler is using a temporary shortstring to pass to fpc_shortstr_concat as the dests parameter, followed by a fpc_shortstr_to_shortstr call to copy the temporary string into the final destination.  Seems like there is an opportunity for optimization here?

d.ioannidis

  • Full Member
  • ***
  • Posts: 221
    • Nephelae
Re: [SOLVED] Variable addresses in SRAM
« Reply #39 on: October 05, 2022, 04:43:01 pm »
Hi,


I did that, although I didn't use built-in move (I am careful about string sizes, but since I use strings only for LCD display I just cut if it doesn't fit):


if you use strings only for lcd display how about a global shortstring variable which you'll use it as a temp buffer ?

Concatenate a shortstring variable ( not a string[10] ) doesn't produce temp variables AFAIU . ( try the next example ; )

Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. {$mode ObjFPC}
  4.  
  5. var s1: ShortString;
  6.  
  7. begin
  8.   s1 := 'test';
  9.   s1 := s1 + '123';
  10. end.
  11.  

Also, you can use it with position index for storage for the lcd rows ( i.e. the rows of a 4x20 LCD will be at s1[10] upto s1[30] for row 1, s1[40] upto s1[60] for row 2, etc ) ...

No more stack problems ;)

regards,
« Last Edit: October 05, 2022, 04:45:30 pm by d.ioannidis »

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: [SOLVED] Variable addresses in SRAM
« Reply #40 on: October 05, 2022, 05:06:30 pm »
This is a good question.  Since fpc_shortstr_concat can handle the situation where either of the two sources can also be the destination, the compiler can omit the temp in this case.  However, in the disassembly listing @dseligo showed, the compiler is using a temporary shortstring to pass to fpc_shortstr_concat as the dests parameter, followed by a fpc_shortstr_to_shortstr call to copy the temporary string into the final destination.  Seems like there is an opportunity for optimization here?

Removal of pessimisation more like :-) My suspicion would be that the compiler is doing something odd to that it can get High() of the destination parameter... particularly since it's a var I certainly hope it's not doing spurious copies.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

alpine

  • Hero Member
  • *****
  • Posts: 1038
Re: [SOLVED] Variable addresses in SRAM
« Reply #41 on: October 05, 2022, 05:41:33 pm »
Eventually I discovered what consumed my SRAM: it was string concatenation.
After confirming that stack overwrites allocated data, with the compiler switch -at (together with -al) I easily found out how much SRAM was consumed for temporary variables in every unit. For each concatenation of strings, 256 bytes were consumed and in some units more than 1KB was reserved.
I solved it by making my own string concatenation procedures and now even AVRs with 2KB SRAM are enough for this project of mine.

Bottom line: be careful with strings in the embedded world.
I strongly recommend using PChar instead of ShortString in such a restricted environments.
Then use strlen(), strcopy(), strcat() from the strings unit. It will be a bit tedious at first, but you won't hit the curbs as you did now.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: [SOLVED] Variable addresses in SRAM
« Reply #42 on: October 05, 2022, 06:25:20 pm »
I strongly recommend using PChar instead of ShortString in such a restricted environments.
Then use strlen(), strcopy(), strcat() from the strings unit. It will be a bit tedious at first, but you won't hit the curbs as you did now.

What would you use for the base variables- array[something] of char?

It certainly sounds like a useful workaround, but the bottom line is that this shouldn't be happening: if OP really did define his strings as string[something] then they all have a fixed length and there should be no need of any temporaries... I can't see where shortstring has got into this.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

alpine

  • Hero Member
  • *****
  • Posts: 1038
Re: [SOLVED] Variable addresses in SRAM
« Reply #43 on: October 05, 2022, 07:01:47 pm »
I strongly recommend using PChar instead of ShortString in such a restricted environments.
Then use strlen(), strcopy(), strcat() from the strings unit. It will be a bit tedious at first, but you won't hit the curbs as you did now.

What would you use for the base variables- array[something] of char?
Yes.

It certainly sounds like a useful workaround, but the bottom line is that this shouldn't be happening: if OP really did define his strings as string[something] then they all have a fixed length and there should be no need of any temporaries... I can't see where shortstring has got into this.
It happens because you can use assignments ( := ) and expressions with ShortString. Actually only the ( + ) operator is defined and you would say it can be easily translated, but (I suspect) the compiler don't bother for the associativity, commutativity, etc. of each type + (e.g. string concatenation and numeric addition have different properties) and makes it the easy way - first computes the expression in temporary, then makes the assignment. In the common case, the infix notation for expressions always may need some temporaries for the evaluation.

When using PChars then all that you can do is to call a subroutines and to pass them as parameters. There won't be a hidden allocations.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

ccrause

  • Hero Member
  • *****
  • Posts: 845
Re: [SOLVED] Variable addresses in SRAM
« Reply #44 on: October 06, 2022, 08:07:01 am »
For the adventurous programmer, here is a proposed compiler enhancement that skips creating a temporary variable when assigning a shortstring concatenation expression to a shortstring variable: https://gitlab.com/ccrause/fpc-source/-/commits/shortstrconcat

The following test:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. var
  4.   s1: string[5] = '123';
  5.   s2: string[5] = '4567';
  6.  
  7. begin
  8.   s1 := s1 + s2;
  9.   writeln(s1);
  10. end.
now compiles to:
Code: [Select]
PASCALMAIN:
.Lc2:
.Ll1:
# [project1.lpr]
# [7] begin
call FPC_INIT_FUNC_TABLE
.Ll2:
# [8] s1 := s1 + s2;
ldi r18,lo8(TC_sPsPROJECT1_ss_S2)
ldi r19,hi8(TC_sPsPROJECT1_ss_S2)
ldi r20,lo8(TC_sPsPROJECT1_ss_S1)
ldi r21,hi8(TC_sPsPROJECT1_ss_S1)
ldi r24,lo8(TC_sPsPROJECT1_ss_S1)
ldi r25,hi8(TC_sPsPROJECT1_ss_S1)
ldi r22,5
mov r23,r1
call fpc_shortstr_concat
.Ll3:
# [9] writeln(s1);
call fpc_get_output
movw r2,r24
ldi r18,lo8(TC_sPsPROJECT1_ss_S1)
ldi r19,hi8(TC_sPsPROJECT1_ss_S1)
movw r20,r2
mov r22,r1
mov r23,r1
mov r24,r1
mov r25,r1
call fpc_write_text_shortstr
call fpc_iocheck
movw r24,r2
call fpc_writeln_end
call fpc_iocheck
.Ll4:
# [10] end.
call fpc_do_exit

This is only a start, since this patch only improves the case of s0 := s1 + s2 where all strings are of typeshortstring and s0 may the same variable as s1 or s2..

Note: there may be unintended side effects caused by this patch, so testing and feedback would help improve the quality...

 

TinyPortal © 2005-2018