Recent

Author Topic: Encrypting data for TCSVDataSet  (Read 1542 times)

Tony Stone

  • Sr. Member
  • ****
  • Posts: 280
Encrypting data for TCSVDataSet
« on: December 26, 2023, 12:30:13 am »
I am using CryptoLib4Pascal to encrypt a CSV text file.  I think I have the encryption/decryption worked out and am understanding it so far to the best of my ability.  I want to use the TCSVDataSet in my password manager because I am hoping to use the data aware controls on my forms.  However the way I am encrypting this data is probably quite "hackish" to real programmers.  Can somone with some experience tell me if my method is at least reasonably secure?  Basically I am reading the encrypted CSV file, then decrypting it and copying it to a stringlist(which should stay in memory  I think) and then i use LoadFromStream to load a TStringStream into the CSVDataSet.  It sort of works in my tests but it seems very hackish to me... also I think I am having an issue where when I save the stringlist again I am having all edits appended to the top.  Anyway, please review my code below and tell me if I should be going a complete different direction for this.  Thanks for any input.



Code: Pascal  [Select][+][-]
  1. uses
  2.  
  3.   ClpISecureRandom,
  4.   ClpIKeyParameter, ClpIBufferedCipher,
  5.   ClpSecureRandom, ClpDigestUtilities, ClpCipherUtilities, ClpCryptoLibTypes,
  6.   ClpConverters, ClpEncoders, ClpParametersWithIV, ClpKeyParameter;
  7.  
  8.  
  9. var
  10.  
  11.   strListtest: TStringList;
  12.   csvtext: TStringStream;
  13.    
  14.  
  15.  
  16.  
  17. procedure TfrmMain.btnSaveAsEncryptedClick(Sender: TObject);
  18. begin
  19.   // Update strListtest with the current content of the dataset
  20.   csvtext := TStringStream.Create();
  21.   try
  22.     CSVDataset1.SaveToStream(csvtext, dfDefault);
  23.     strListtest.Text := csvtext.DataString;
  24.   finally
  25.     csvtext.Free;
  26.   end;
  27.  
  28.  
  29.   // Now encrypt and save the updated strListtest
  30.   EncryptStringListAndSave(strListtest, '3C6DB4EFEEBA0CB8CB1E8B39385FDE25E1DA5446F4918DF75538CFCF5A95C1C9', 'encryptedfile.txt');
  31. end;
  32.  
  33.  
  34. procedure TfrmMain.btnLoadEncryptedFileClick(Sender: TObject);
  35. begin
  36.   strListtest:=TStringList.Create;
  37.  
  38.  
  39.   DecryptFileAndLoadToStringList('encryptedfile.txt','3C6DB4EFEEBA0CB8CB1E8B39385FDE25E1DA5446F4918DF75538CFCF5A95C1C9',strListtest);
  40.  
  41.  
  42.   csvtext:= TStringStream.Create(strListtest.Text);
  43.   CSVDataset1.LoadFromStream(csvtext,dfDefault);
  44.   CSVDataset1.Active:=True;
  45.  
  46.  
  47. end;
  48.  
  49.  
  50. // this all feels so wrong and not secure!!
  51.  
  52.  
  53. procedure TfrmMain.EncryptStringListAndSave(const InputList: TStringList; const Key: String; const FileName: String);
  54. var
  55.   secureRandom: ISecureRandom;
  56.   Cipher: IBufferedCipher;
  57.   KeyParam: IKeyParameter;
  58.   IV: TBytes;
  59.   InputBytes, CipherText: TBytes;
  60.   FileStream: TFileStream;
  61. begin
  62.   // Generate a random "initialization vector" so if the same text is encrypted more than once
  63.   // it will still produce different encrypted data every time.  (as i understand it)
  64.   secureRandom := TSecureRandom.Create();
  65.   SetLength(IV, 16);
  66.   SecureRandom.NextBytes(IV); // Use a secure random number generator
  67.  
  68.  
  69.   KeyParam := TKeyParameter.Create(THex.Decode(Key));
  70.  
  71.  
  72.   Cipher := TCipherUtilities.GetCipher('AES/CBC/PKCS7PADDING'); // i think this is right
  73.   Cipher.Init(True, TParametersWithIV.Create(KeyParam, IV)); // true for encryption
  74.  
  75.  
  76.   InputBytes := TConverters.ConvertStringToBytes(InputList.Text, TEncoding.UTF8);
  77.   CipherText := Cipher.DoFinal(InputBytes);
  78.  
  79.  
  80.   FileStream := TFileStream.Create(FileName, fmCreate);
  81.   try
  82.     FileStream.WriteBuffer(IV[0], Length(IV));// save the iv first so we can decrypt later
  83.     FileStream.WriteBuffer(CipherText[0], Length(CipherText));
  84.   finally
  85.     FileStream.Free;
  86.   end;
  87. end;
  88.  
  89.  
  90. procedure TfrmMain.DecryptFileAndLoadToStringList(const FileName: String; const Key: String; OutputList: TStringList);
  91. var
  92.   Cipher: IBufferedCipher;
  93.   KeyParam: IKeyParameter;
  94.   IV, CipherText, DecryptedBytes: TBytes;
  95.   FileStream: TFileStream;
  96. begin
  97.   FileStream := TFileStream.Create(FileName, fmOpenRead);
  98.   try
  99.     // Read the IV from the file (first 16 bytes for AES)
  100.     SetLength(IV, 16);
  101.     FileStream.ReadBuffer(IV[0], Length(IV));
  102.  
  103.  
  104.     // Read the rest of the file as encrypted data
  105.     SetLength(CipherText, FileStream.Size - FileStream.Position);
  106.     FileStream.ReadBuffer(CipherText[0], Length(CipherText));
  107.   finally
  108.     FileStream.Free;
  109.   end;
  110.  
  111.  
  112.   // Prepare the key
  113.   KeyParam := TKeyParameter.Create(THex.Decode(Key));
  114.  
  115.  
  116.   // Initialize the AES Cipher in CBC mode with PKCS7 Padding
  117.   Cipher := TCipherUtilities.GetCipher('AES/CBC/PKCS7PADDING');
  118.   Cipher.Init(False, TParametersWithIV.Create(KeyParam, IV)); // false for decryption
  119.  
  120.  
  121.   DecryptedBytes := Cipher.DoFinal(CipherText); // doing actual decryption
  122.  
  123.  
  124.   // Load decrypted data into the TStringList
  125.   OutputList.Text := TConverters.ConvertBytesToString(DecryptedBytes, TEncoding.UTF8);
  126.   // now the decrypted data stays in memory so i can assign to a CSVdataset... test ok so far
  127. end;
  128.  
  129.  
  130.  

Tony Stone

  • Sr. Member
  • ****
  • Posts: 280
Re: Encrypting data for TCSVDataSet
« Reply #1 on: December 26, 2023, 02:48:47 pm »
So as far as my duplicate record issue with TCSVDataSet i think it is a known issue which is discussed here and there is a bug report but no progress on it: https://forum.lazarus.freepascal.org/index.php?topic=60714.0

As far as this being a reasonable method to secure a simple database I think I am getting my own answers.  I believe the Crypto library is solid but the more i played around last night it occurred to me that there is a possibility that another malicious program could crash my password manager and potentially gain all of the decrypted data in a memory dump of sorts.  So I am coming to my own conclusion that this is not a reasonably secure approach for a password manager.  I will avoid the data aware controls and take another path... probably doing a simple CSVDocument still but encrypting each individual sensitive field and only decrypt when necessary.  And then encrypting the entire dataset before writing to disk.  So all sensitive parts of the data will end up being double encrypted.  And as I said before if there is a better approach I am not considering I would love to hear it.  Thanks.

Fibonacci

  • Hero Member
  • *****
  • Posts: 788
  • Internal Error Hunter
Re: Encrypting data for TCSVDataSet
« Reply #2 on: December 26, 2023, 10:30:30 pm »
Is the key embedded and this is for your personal use only? If not, you should show how the key is derived from a password/keyfile/something else.

Is it secure? Depends on your level of paranoia. Can your adversary run software on your PC? Does he have physical access to your PC? Can he break in and gain physical access to your PC while powered on?

Plaintext passwords and the key will stay in memory and can be read by other software or directly from RAM (if PC is on and locked).

At least the key + key derive information (password, key file location etc) should be kept in one location in the process memory, and after they are no longer needed this part of memory should be erased.

Tony Stone

  • Sr. Member
  • ****
  • Posts: 280
Re: Encrypting data for TCSVDataSet
« Reply #3 on: December 26, 2023, 10:53:11 pm »
This is personal use as of now, yes.  The key I am currently using is simply for testing reasons.  The level of paranoia... hmmm... hard to calculate lol.  I suppose my only concern would be if some one were to use this password manager and they already have a malicious program installed that snoops on other programs.  I mean this could be an issue with any password manager or software for that matter.  I just feel like trying to keep it simple and using the Data aware controls and a TCSVDataSet is not in any way adding any layers of additional security.  If anything it may be making it difficult to add another layer of security.  A password manager should at the very least avoid security issues due to laziness in the programming(how i feel but in practice as a hobbyist it makes it easier for me to just drop controls on a form)...  I am currently playing around and experimenting with some other ideas.  Like use string arrays to keep data in memory and keeping things encrypted like usernames and passwords.  only decrypt them when they need to be accessed by user.

Fibonacci

  • Hero Member
  • *****
  • Posts: 788
  • Internal Error Hunter
Re: Encrypting data for TCSVDataSet
« Reply #4 on: December 27, 2023, 12:33:54 am »
they already have a malicious program installed that snoops on other programs

If that happens, you can only unplug the internet cable and do full reinstall of the OS. Even then you cant be 100% sure anymore. This PC is burned. Sophisticated adversary may be even able to eg reprogram Intel ME which is mini-OS installed inside CPU or on the motherboard, and has direct access to all devices including network adapters, can listen your keystrokes, can see your screen, read your RAM and can use your network adapter to send this data without you noticing anything.

So it depends on how much you want to protect your data and who the adversary might be.

Your solution may be enough for storing encrypted passwords file in the cloud, assuming you are sure nobody has physical access to your PC.

Tony Stone

  • Sr. Member
  • ****
  • Posts: 280
Re: Encrypting data for TCSVDataSet
« Reply #5 on: December 27, 2023, 12:52:11 am »
Thanks.  Just smashed my desktop into a million pieces.  lol not really.  I think you are right on all points... and this has not stopped anyone from making password managers in the past.  So I suppose my goal needs to be more along the lines of reasonable security I can obtain with the tools I have.  Which my goal is to all be strictly written in Pascal.  I suppose I just want to be as reasonably secure as possible and not create vulnerabilities by being too lazy and using the data aware controls.  I am somewhat moved on to a new idea as of now.  I am also trying to study up on key derivation so thanks for that hint, I knew my example code is still far from anything you would want to use in production.

 

TinyPortal © 2005-2018