Lazarus

Programming => General => Topic started by: RHSRLT on December 07, 2019, 06:19:06 pm

Title: Convert pdf-file to hex
Post by: RHSRLT on December 07, 2019, 06:19:06 pm
Hello together,

i am working on a programm to store some statistics in a Postgresql-Database.

One of the points i want to do is to store a pdf-file in the Database. As they are small files i want to store them direct in the database with a bytea-column. To do so i need to convert the file to a hexadecimal-string.

Is there any way to convert a pdf-file to hex so i can store it inside the database? I read a few post about converting strings to hex, but I'm not sure how to convert a whole file.

I hope someone out here can help me with my problem.
Title: Re: Convert pdf-file to hex
Post by: zeljko on December 07, 2019, 06:38:12 pm
Use postgresql functions encode/decode for writing/reading https://www.postgresql.org/docs/10/functions-binarystring.html
Title: Re: Convert pdf-file to hex
Post by: lucamar on December 07, 2019, 06:38:37 pm
There are several ways to achieve it. For example, since you seem to know how, reading the file into a string with ReadFileToString() and converting that string.

More flexible is to use a TFileStream and convert each character/byte on the fly. A quick function to do char conversion can be as simple as:
Code: Pascal  [Select][+][-]
  1. function ToHex(AChar: Char): String;
  2. var
  3.   AByte: Byte;
  4. begin
  5.   AByte := Ord(AChar);
  6.   Result := AByte.ToHexString(2);
  7.   // or just: Result := IntToHex(Ord(AChar), 2);
  8. end;
Title: Re: Convert pdf-file to hex
Post by: MarkMLl on December 07, 2019, 08:54:04 pm
Quote from: RHSRLT on December 07, 2019, 06:19:06 pm
Is there any way to convert a pdf-file to hex so i can store it inside the database? I read a few post about converting strings to hex, but I'm not sure how to convert a whole file.

You don't need to. I forget the term but PostgreSQL is entirely happy storing a file verbatim, and I believe does so fairly efficiently by design.

MarkMLl
Title: Re: Convert pdf-file to hex
Post by: RHSRLT on December 08, 2019, 01:15:19 pm
Quote from: zeljko on December 07, 2019, 06:38:12 pm
Use postgresql functions encode/decode for writing/reading https://www.postgresql.org/docs/10/functions-binarystring.html

I tried this but i get an error which says that the '%' Symbol cant be decoded. I tried hex, base64 and escape but always i get this error.


Quote from: lucamar on December 07, 2019, 06:38:37 pm
There are several ways to achieve it. For example, since you seem to know how, reading the file into a string with ReadFileToString() and converting that string.

More flexible is to use a TFileStream and convert each character/byte on the fly. A quick function to do char conversion can be as simple as:

ReadFileToString() dows work but I dont get the convert to hex working. If i try to store the string as text and select it afterwards its only 644 chars long from originally 176282 chars. The columns type is text which should be unlimited.
I never worked with TFileStream before and don't know how to convert the Char on the fly since i dont fully understand the way TFileStream works.


Quote from: MarkMLl on December 07, 2019, 08:54:04 pm
You don't need to. I forget the term but PostgreSQL is entirely happy storing a file verbatim, and I believe does so fairly efficiently by design.

There is a possibility to store files as LargeObject but the documentation regarding LOs is quite confusing and the commands to work with are completly different to the normal postgresql-commands.


Thanks for your help so far, if i dont get it to work i think i will use the filesystem to store the data und put the path in the database. I experience that this is easier to achieve.
Title: Re: Convert pdf-file to hex
Post by: lucamar on December 08, 2019, 02:22:51 pm
Quote from: RHSRLT on December 08, 2019, 01:15:19 pm
I never worked with TFileStream before and don't know how to convert the Char on the fly since i dont fully understand the way TFileStream works.

It's as easy as this, reading bytes rather than chars:

Code: Pascal  [Select][+][-]
  1. function FileToHex(const AFileName: String): String;
  2. var
  3.   AStream: TFileStream;
  4.   AByte: Byte;
  5. begin
  6.   Result := '';
  7.   try
  8.     AStream := TFileStream.Create(AFileName, fmOpenRead);
  9.     try
  10.       while AStream.Read(AByte, 1) > 0 do
  11.         Result := Result + IntToHex(AByte, 2);
  12.     finally
  13.       AStream.Free;
  14.     end;
  15.   except
  16.     on e: EFOpenError do ShowMessage(e.Message);
  17.   end;
  18. end;
Title: Re: Convert pdf-file to hex
Post by: RHSRLT on December 11, 2019, 06:47:28 pm
Awesome, thank you. Storing Hex-Data (hopefully the pdf) in the database works.

But now i'm struggeling to get the data back.
I tried to reverse your conversion with a TStringStream and write the file with TFileStream but the created pdf-File shows "Format Error: No PDF-Format or damaged Document". I'm pretty sure the conversion back to the file went bad, but i cant figure out what the problem is.

Code: Pascal  [Select][+][-]
  1.  
  2. sveBerichte.FileName := cmbDownload.Text;
  3. If sveBerichte.execute Then
  4. begin
  5. filepath := sveBerichte.Filename;
  6. hexstring := '';
  7. quyBerichte.close;
  8. quyBerichte.SQL.Text := 'Select bericht from eberichte where nr = :nr';
  9. quyBerichte.Params.ParambyName('nr').AsString := cmbdownload.text;
  10. quyBerichte.open;
  11. hexstring := quyBerichte.Fields[0].AsString;
  12. quyBerichte.close;
  13. try
  14.   HStream := TStringStream.Create(hexstring);
  15.   try
  16.     while HStream.Read(CByte, 2) > 0 do
  17.       pdfstring := pdfstring + HexStr(CByte, 1);
  18.     finally
  19.      Stream.Free;
  20.     end;
  21.   finally
  22.   end;
  23.   begin
  24.   Stream := TFileStream.Create(filepath, fmCreate);
  25.   try
  26.     Stream.Write(pdfstring[1], Length(pdfstring));
  27.   finally
  28.     Stream.Free;
  29.   end;
  30.   end;
  31.   end;
Title: Re: Convert pdf-file to hex
Post by: lucamar on December 11, 2019, 09:16:55 pm
Untested and there're probably some errors but it should be something like this:

Code: Pascal  [Select][+][-]
  1.   sveBerichte.FileName := cmbDownload.Text;
  2.   If not sveBerichte.execute then
  3.     Exit;
  4.   filepath := sveBerichte.Filename;
  5.   hexstring := '';
  6.   {I'll assume al this works as it should}
  7.   quyBerichte.close;
  8.   quyBerichte.SQL.Text := 'Select bericht from eberichte where nr = :nr';
  9.   quyBerichte.Params.ParambyName('nr').AsString := cmbdownload.text;
  10.   quyBerichte.open;
  11.   hexstring := quyBerichte.Fields[0].AsString;
  12.   quyBerichte.close;
  13.  
  14.   HStream := TStringStream.Create(hexstring);
  15.   try
  16.     Stream := TFileStream.Create(filepath, fmCreate);
  17.     try
  18.       HStream.Seek(0, soFromBeginning); {Murphy-guard!}
  19.       {CByte is declared as String[2], isn't it?}
  20.       while HStream.Read(CByte[1], 2) > 0 do
  21.         Stream.Write(chr(StrToInt('0x' + CByte)));
  22.     finally
  23.       Stream.Free;
  24.     end;
  25.   finally
  26.     HStream.Free;
  27.   end;
  28. end;

Your hexstring contains hexadecimal characters which you must convert back to integer; this can be done with (among others) StrToInt(). Then you can use Chr() to convert the byte to a character which, finally, is saved to the file. Note than I'm here assuming: 1) reading the database works as it should; and 2) CByte is declared as a String[2]. If it's not then this won't work at all.

Quite frankly, I wouldn't implement this feature as this, needing so may conversion back and forth. I would use a BLOB field (or similar) and read/write the file as is. But to each their own, I guess ;)
TinyPortal © 2005-2018