[Solved] ReadBuffer(AnsiString[1], 1500); - AnsiString is empty

[rvk]

Gah. What thought reading into the AnsiString would do, is give me the raw bytes but as a string. Y'know, like this (sans the spaces):

Code: [Select]

ansistringBytes := 'D7 FC 05 F8 B7 07 4F 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 B3 09 DD FB D8 F9 C0 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0C 00 04 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 00 03 00 03 00 FF FF FF FF FF FF FF FF FF FF FF FF 02 00 00 00 02 00 01 00 FF FF FF FF FF FF FF FF FF FF FF FF';


O no, it doesn't. It returns it exactly like notepad would show you the strings. That's why string isn't a really suitable container for this kind of data. You could of course translate the bytes to a string-like string like you showed but that involves an extra step. Just searching an array for byte 255 is much more efficient.

After you found the position you're looking for you can't read it into a TStringList like you wanted. Because it's still all raw-data. Then you'll need to determine how you want to show the data and if it's like what you showed you can convert the raw-data with dec2hex to what you want.

[FiftyTifty]

O no, it doesn't. It returns it exactly like notepad would show you the strings. That's why string isn't a really suitable container for this kind of data. You could of course translate the bytes to a string-like string like you showed but that involves an extra step. Just searching an array for byte 255 is much more efficient.

After you found the position you're looking for you can't read it into a TStringList like you wanted. Because it's still all raw-data. Then you'll need to determine how you want to show the data and if it's like what you showed you can convert the raw-data with dec2hex to what you want.

For some reason, there's no Dec2Hex(), only Hex2Dec(). There is IntToHex() and HexStr(), but they don't accept individual bytes. Also tried experimenting with SetString, but the above two functions hate bytes.

This stuff's hard man.

[rvk]

For some reason, there's no Dec2Hex(), only Hex2Dec(). There is IntToHex() and HexStr(), but they don't accept individual bytes. Also tried experimenting with SetString, but the above two functions hate bytes.

Int2Hex is the one
It isn't that hard. Int2Hex can convert to $FF but also to $FFFF etc. So the second parameter is for the number of digits.

Here is a small example for converting a RAW byte array to a string like you wanted:

Code: Pascal  [Select][+][-]

1. const
2.   Arr: array[0..16 * 7 + 7] of byte =
3.     ($D7, $FC, $05, $F8, $B7, $07, $4F, $09, $00, $00, $00, $00, $00, $00, $00, $00,
4.     $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00,
5.     $00, $00, $00, $00, $00, $00, $00, $00, $B3, $09, $DD, $FB, $D8, $F9, $C0, $05,
6.     $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $0C,
7.     $00, $04, $00, $04, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00, $00,
8.     $00, $00, $06, $00, $03, $00, $03, $00, $FF, $FF, $FF, $FF, $FF, $FF, $FF, $FF,
9.     $FF, $FF, $FF, $FF, $02, $00, $00, $00, $02, $00, $01, $00, $FF, $FF, $FF, $FF,
10.     $FF, $FF, $FF, $FF, $FF, $FF, $FF, $FF);
11. var
12.   S: string;
13.   I: integer;
14. begin
15.   S := '';
16.   for I := 0 to Length(Arr) - 1 do
17.     S := S + IntToHex(Arr[I], 2) + ' ';
18.   ShowMessage(S);
19. end;

Will result in a string like this:

Code: [Select]

D7 FC 05 F8 B7 07 4F 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 B3 09 DD FB D8 F9 C0 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0C 00 04 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 00 03 00 03 00 FF FF FF FF FF FF FF FF FF FF FF FF 02 00 00 00 02 00 01 00 FF FF FF FF FF FF FF FF FF FF FF FF

[FiftyTifty]

Ah, I was using an AnsiString, to convert the whole byte array (1500) into one big ass string. The compiler kept giving me error messages.

Code: [Select]

function GetWholeNPCData(msMAPFile: TMemoryStream; dwordPointer: DWord): AnsiString;
var
  bufferNPCData: AnsiString;
  bytebufferNPCData: array [1..1500] of Byte;
  iCounter: Word;
begin

  SetLength(bufferNPCData, 1500);

  msMAPFile.Seek(dwordPointer, 0);

  msMAPFile.ReadBuffer(bytebufferNPCData, 1500);

  for iCounter := 1 to 1500 do begin
    bufferNPCData[iCounter] := IntToHex(bytebufferNPCData[iCounter], 2);
  end;

  Result := bufferNPCData;

end;

And the compiler throws:

maincode.pas(123,32) Error: Incompatible types: got "AnsiString" expected "Char"

Which, now that I think about it, is due to the string returning a char when using it as an array. Going to see if I can work my way around that...AppendStr() or Insert(), maybe?

Could just split up the stream into mini streams and chuck them into your function, but...Myeh. Kinda hacky?

[rvk]

This is wrong:

Code: Pascal  [Select][+][-]

1. for iCounter := 1 to 1500 do begin
2.   bufferNPCData[iCounter] := IntToHex(bytebufferNPCData[iCounter], 2);
3. end;

bufferNPCData is a string (or ansistring) so bufferNPCData[ x ] is a character. While IntToHex will give you TWO characters, namely FF or AE etc.

You need to do something like this:

Code: Pascal  [Select][+][-]

1. bufferNPCData := ''; // first empty the string
2. for iCounter := 1 to 1500 do begin
3.   // then just add IntToHex to that string for every byte
4.   bufferNPCData := bufferNPCData + ' ' + IntToHex(bytebufferNPCData[iCounter], 2);
5. end;

(and you don't need the SetLength(bufferNPCData, 1500); because we just append a string to a string and don't feed raw data directly in bufferNPCData.)

Do note that you get a very very large string.
I'm not sure what you want to do with that.
Like we said before, searching that string is very very inefficient.

It's better to return this array of 1500 and work with that from then on.

[FiftyTifty]

This is wrong:

Code: Pascal  [Select][+][-]

1. for iCounter := 1 to 1500 do begin
2.   bufferNPCData[iCounter] := IntToHex(bytebufferNPCData[iCounter], 2);
3. end;

bufferNPCData is a string (or ansistring) so bufferNPCData[ x ] is a character. While IntToHex will give you TWO characters, namely FF or AE etc.

You need to do something like this:

Code: Pascal  [Select][+][-]

1. bufferNPCData := ''; // first empty the string
2. for iCounter := 1 to 1500 do begin
3.   // then just add IntToHex to that string for every byte
4.   bufferNPCData := bufferNPCData + ' ' + IntToHex(bytebufferNPCData[iCounter], 2);
5. end;

(and you don't need the SetLength(bufferNPCData, 1500); because we just append a string to a string and don't feed raw data directly in bufferNPCData.)

Do note that you get a very very large string.
I'm not sure what you want to do with that.
Like we said before, searching that string is very very inefficient.

It's better to return this array of 1500 and work with that from then on.

Was just doing something exactly like that whilst you were posting. Heh. This worked:

Code: [Select]

iIndex := 1;
  for iCounter := 1 to 1500 do begin

    strTemp := IntToHex(bytebufferNPCData[iCounter], 2);

    bufferNPCData[iIndex] := strTemp[1];
    inc(iIndex);
    bufferNPCData[iIndex] := strTemp[2];
    inc(iIndex);

  end;

The reason why I wanted to use a huge string, is that the in-built functionality for finding data (pos(), mainly) only works with strings. Could be the sleep deprivation, but CompareByte looks like an utter pain in the ass to work with. Though, I should probably experiment with that function I found a few posts back.

The main issue with it, is that I'd have to keep resizing the buffer, by getting rid of the first occuring bytes, in order to find all the NPC structures. And I have to keep track of the position of all the found structures, in such a way that I can then write back the data to the TMemoryStream.

Just out of my depth here, is all, so I was aiming to stick with what was easy and familiar.

[rvk]

Wel, if you do it like you have now you might find the wrong string.

If you search for FF in 12 34 56 6F F9 33, as you did it now without the spaces it will find the FF in between the 4th and 5th hex number.
Searching for multiple FFs or a combination of other hex-numbers might get your error-rate down but it is still very dangerous to do it like this.

It's still better to do an array search, increasing your index and using comparebytes to see if the pattern you want is at index in that array.

And what's wrong with using that AOBScan() function you found? It does something quite similar as comparebyte and also works with PByte which is a pointer to an index in a array of bytes.

If you give an example I'm sure we can show you here how you would need to use comparebytes to find the index you want. Once you found one index you could use that index to find the next pattern that matches for your next block.

Trust me... learning about using byte-arrays now will save you a lot of work in the future. Using strings like you do now is really going to complex things a lot and just throw you in deeper and deeper.

The main issue with it, is that I'd have to keep resizing the buffer, by getting rid of the first occuring bytes, in order to find all the NPC structures. And I have to keep track of the position of all the found structures, in such a way that I can then write back the data to the TMemoryStream.

You weren't thinking about converting that hex-string back to the original data, were you? If you were.... yikes... you should definitely go with arrays and TMemoryStream. Saving the position of all NPC structures is really the way to go. Once you want to change a value you can do it directly in the TMemoryStream. B.T.W. a TMemoryStream has a Memory property which is basically an array of bytes. So you don't even need a separate array. Just search that memory and build a NPC record array. It's sooooooooooo much easier than converting to a hex-string, extracting data, changing data in that string and writing that back.

[rvk]

B.T.W. here is an example of searching a TMemoryStream for a pattern. It starts from .Position so you can do incremental searched for all your NPC records.

From http://forum.lazarus.freepascal.org/index.php/topic,30108.msg191187.html#msg191187

Code: Pascal  [Select][+][-]

1. type
2.   TPatternArray = Array of byte;
3.  
4. function DoSearch(Stream: TMemoryStream; Pattern: TPatternArray): Int64;
5. var
6.   idx: Integer;
7. begin
8.   Result := -1;
9.   for idx := Stream.Position to Stream.Size - Length(Pattern) do
10.   begin
11.     if CompareMem(Stream.Memory + Idx, @Pattern[0], Length(Pattern))
12.     then exit(idx);
13.   end;
14. end;

You can even use the HexStringToByteArray on that page to convert your hex-pattern to a TPatternArray to feed into this DoSearch function.

[FiftyTifty]

Wel, if you do it like you have now you might find the wrong string.

If you search for FF in 12 34 56 6F F9 33, as you did it now without the spaces it will find the FF in between the 4th and 5th hex number.
Searching for multiple FFs or a combination of other hex-numbers might get your error-rate down but it is still very dangerous to do it like this.

It's still better to do an array search, increasing your index and using comparebytes to see if the pattern you want is at index in that array.

And what's wrong with using that AOBScan() function you found? It does something quite similar as comparebyte and also works with PByte which is a pointer to an index in a array of bytes.

If you give an example I'm sure we can show you here how you would need to use comparebytes to find the index you want. Once you found one index you could use that index to find the next pattern that matches for your next block.

Trust me... learning about using byte-arrays now will save you a lot of work in the future. Using strings like you do now is really going to complex things a lot and just throw you in deeper and deeper.

You weren't thinking about converting that hex-string back to the original data, were you? If you were.... yikes... you should definitely go with arrays and TMemoryStream. Saving the position of all NPC structures is really the way to go. Once you want to change a value you can do it directly in the TMemoryStream. B.T.W. a TMemoryStream has a Memory property which is basically an array of bytes. So you don't even need a separate array. Just search that memory and build a NPC record array. It's sooooooooooo much easier than converting to a hex-string, extracting data, changing data in that string and writing that back.

Hear ya loud and clear. Guess it can't hurt to come up with some array equivalents to Pos() and such.

And ya, the plan was to split up the string, write some parts to the gui, and when the user saves the file, the string is written back to the memory stream. I guess this warrants a new thread, eh? I'll elaborate on the pitfalls 'n' such.

[rvk]

Guess it can't hurt to come up with some array equivalents to Pos() and such.

I just posted such function for TMemoryStream in my previous post (the DoSearch function).

It will do everything in one go. You don't have to read from stream to array and use a pos-like function. You can just directly search the TMemoryStream and build an array of all the needed positions of the NPC records.

That way, when you want to change the values, you can change them directly in the TMemoryStream (assuming they stay the same size) and you don't have to fiddle around with converting twice (with all the problems that come with it).