(SOLVED) Encrypt DBF File

[InfoMan]

Using: Lazarus 1.1 (14/06/2012), Windows 7 Ultimate 32 Bits.

Good evening to all!

Is there any way to encrypt a database in DBF in Lazarus so that it can only be read by my program? If, as I do that?

Many thanks to all who respond.

[BigChimp]

There are several ways you could think about:

• Patch the FPC dbf code to read/write encrypted data
• Use things like TrueCrypt containers to encrypt (part of the) the file system where the dbf is located
• Encrypt the contents of the DBF file in your application - which will leave open the structure though.

As far as I know, there is no built-in facility for encrypting data for DBF.

Also, the usual caveats apply: if your program is run in an untrusted environment, the encryption keys and/or plaintext data can be retrieved (e.g. from the source code, the compiled code on disk/in memory). The amount of effort needed for this varies - encryption may very well be a sensible way to protect confidential data.

[ludob]

If you enter "dbf encrypt" in the search box on the left side of this screen you get http://www.lazarus.freepascal.org/index.php/topic,15422.msg83016.htm . That thread explains how to configure TDBF to use a stream for loading and storing and how to combine that with encryption.

[BigChimp]

Ah yes, searching the forum, the wiki and the FPC and Lazarus mailing iist.. the forgotten art that turns up hidden gems....

You could probably easily adapt that code to use the FPC Blowfish units.

However, you are writing the dbf unencrypted to temporary storage before encrypting it, which is a tradeoff your security analysis may or may not deem acceptable.

[taazz]

However, you are writing the dbf unencrypted to temporary storage before encrypting it, which is a tradeoff your security analysis may or may not deem acceptable.

the code snips do not write to any temporary storage before encryption unless you mean the memory stream when you say temporary storage.
This method has average security in the sense  that a memory dump of the program will reveal the unencrypted data  no need to go fish for keys in the exe which is an other attack vector.
what would you suggest one should do in such conditions do you know any database that correctly supports encryption and even if the database file is stolen can't be opened with out your program?

In this case must be embedded also.

[BigChimp]

Taking the search idea one step further by using a search engine to search on terms like "dbase encryption", "DBase IV encryption":

This file format document:
http://dbase.com/Knowledgebase/INT/db7_file_fmt.htm
indicates there is a
dBASE IV encryption flag
No idea how it works or how strong it is, but it might be possible to implement that in the dbf units, which would be an elegant solution.

Hmmm, this post:
http://www.antionline.com/archive/index.php/t-236215.html
indicates "the encryption key can be seen in its "reserved" space in the header"... so it presumably isn't a very strong mechanism

This file structure document
http://ulisse.elettra.trieste.it/services/doc/dbase/DBFstruct.htm#T3
indicates there is also a Dbase 5.0 for DOS as well as a Dbase 5.0 for Windows
Encryption flag.
... which may indicate the same or... different encryption for that file format.

Looking at the source code for open source DBF (derivative) environments like xharbour might be handy, see e.g.
http://www.mail-archive.com/xharbour-developers@lists.sourceforge.net/msg02403.html
where at least the encryption flag is mentioned.

However, if I may guess, I suspect built in DBase encryption is weak...

Hoping people with experience with DBase will pipe up... (but I suspect chances of that are low)

[ludob]

the code snips do not write to any temporary storage before encryption unless you mean the memory stream when you say temporary storage.
This method has average security in the sense  that a memory dump of the program will reveal the unencrypted data  no need to go fish for keys in the exe which is an other attack vector.

Right. There is no temporary disk storage. Datasets are clear text memory based in any case. Dumping memory with or without the intermediate memory stream will give you the database contents, even if your db backend and wire protocol use encryption correctly.
A better security is obtained by encrypting the field contents. You can still use the dbcontrols when you implement the OnGetText and OnSetText events for the encrypted fields. Using (a hash of) a user password to encrypt the data avoids storing the key in the program.

[BigChimp]

the code snips do not write to any temporary storage before encryption unless you mean the memory stream when you say temporary storage.

Oops, you're right.

This method has average security in the sense  that a memory dump of the program will reveal the unencrypted data  no need to go fish for keys in the exe which is an other attack vector.

Agreed. As you say, it's a different attack vector but if you have the program running, it's not the easiest (therefore most important/likely)

what would you suggest one should do in such conditions do you know any database that correctly supports encryption and even if the database file is stolen can't be opened with out your program?

In this case must be embedded also.

I think that's the fundamental problem: whenever your data leaves a physically trusted environment with good logical access security, you cannot completely secure the data.
Depending on the importance of the data and the measures you have taken (e.g. encryption), the residual risk may be almost theoretical.
If needed, things like hardware tokens (RSA keys etc) that you use for getting passwords for decryption can be used, etc.

So, there is nothing totally secure, but depending on your security analysis, encrypting the data/database may well be a very good step to increase security.

Edit: clarification

[InfoMan]

Thank you for all.

[BigChimp]

FYI... Updated the TDBF tutorial wiki page...

[Elmug]

A way to encrypt can be to enter text data in a separate application that can encrypt/decript, then copy-paste the encrypted data into the database.

To read it, the opposite is done, always using the encocder/decoder application, which is temporary pass-word based.

[taazz]

Right. There is no temporary disk storage. Datasets are clear text memory based in any case. Dumping memory with or without the intermediate memory stream will give you the database contents, even if your db backend and wire protocol use encryption correctly.
A better security is obtained by encrypting the field contents. You can still use the dbcontrols when you implement the OnGetText and OnSetText events for the encrypted fields. Using (a hash of) a user password to encrypt the data avoids storing the key in the program.

I agree there are methods to protect the data including not saving the password used but ask the user each time is needed.
Those methods are only applied to the data though, somehow I get the impression that the topic starter is more concerned with the schema in which case this is an average solution that would stop most people from accessing it but not a good programmer.

Agreed. As you say, it's a different attack vector but if you have the program running, it's not the easiest (therefore most important/likely)

Well its not easy to scan the exe for passwords either but its not considered secure to have a password hard coded in your application.
I would say its on the same difficulty with scanning an exe for passwords.

I think that's the fundamental problem: whenever your data leaves a physically trusted environment with good logical access security, you cannot completely secure the data.
Depending on the importance of the data and the measures you have taken (e.g. encryption), the residual risk may be almost theoretical.
If needed, things like hardware tokens (RSA keys etc) that you use for getting passwords for decryption can be used, etc.

So, there is nothing totally secure, but depending on your security analysis, encrypting the data/database may well be a very good step to increase security.

Edit: clarification

Agreed there is nothing totaly secure I was only wandering if you know any secure embeddable database with strong encryption that I might have missed. The techniques and algorithms are well known.

[BigChimp]

I think that's the fundamental problem: whenever your data leaves a physically trusted environment with good logical access security, you cannot completely secure the data.
Depending on the importance of the data and the measures you have taken (e.g. encryption), the residual risk may be almost theoretical.
If needed, things like hardware tokens (RSA keys etc) that you use for getting passwords for decryption can be used, etc.

So, there is nothing totally secure, but depending on your security analysis, encrypting the data/database may well be a very good step to increase security.

Edit: clarification

Agreed there is nothing totaly secure I was only wandering if you know any secure embeddable database with strong encryption that I might have missed. The techniques and algorithms are well known.

Nope, don't know of any...

[ezlage]

I do it this way:
(using DCP Crypt)

To decode a DBF file:

Code: [Select]

function DecodTbu(i:integer):boolean; //"i" is a number that identify what DBF file will be loaded
var
  motor:TDCP_rc6; //I use RC6 to encode
begin
  {...}
  msu[i]:=TMemoryStream.Create; //msu is a array of TMemoryStream
  fsu[i]:=TFileStream.Create(dbfpath,fmOpenRead); //fsu is a array of TFileStream
  motor:=TDCP_rc6.Create(nil);
  motor.InitStr(mypasswd,TDCP_sha512); //Here RC6 uses a SHA512 hash of my passphrase
  motor.DecryptStream(fsu[i],msu[i],fsu[i].Size); //Decode file to memory
  motor.Burn;
  motor.Free;
  fsu[i].Free;
  {...}
end;
To encode DBF file:

Code: [Select]

function CodTbu(i:integer):boolean; //Encode file to disk
var
  motor:TDCP_rc6;
begin
  {...}
  msu[i].Position:=0; //Rewinds the TMemoryStream before encode to disk
  fsu[i]:=TFileStream.Create(dbfpath,fmCreate);
  motor:=TDCP_rc6.Create(nil);
  motor.InitStr(mypasswd,TDCP_sha512);
  motor.EncryptStream(msu[i],fsu[i],msu[i].Size);
  motor.Burn;
  motor.Free;
  fsu[i].Free;
  {...}
end;
To use DBF file in memory (without decode it to disk):

Code: [Select]

  {..}
  with TDBF1 do begin
    UserStream:=msu[i];
    Storage:=stoMemory;
    Open;
  end;
  {..}

I hope that helps.
Sorry by my poor english.

[BigChimp]

I know this is an old thread but I have some new info.

If you are using tdbf with stream storage and memo fields, then closing and opening the dataset will delete the memo contents (but not the contents of the dbf memorystream/other fields).

@ezlage & everybody: did anybody encounter this problem?

I'm currently working on a fix for this in FPC trunk, but it's not quite ready yet.