TIniFile doesn't read a file with BOM

[alexxis]

Hi, I'm experiencing a strange problem with the TINIFile component : it doesn't parse files that have BOM (Byte Order Mark).

If I try to import a file with BOM it doesn't load any sections, Why ?
If I remove the BOM (e.g. with vim ---> :set nobomb ), the file is parsed correctly.

How can I solve the problem ?


I'm using FPC 2.6.2 and Lazarus 1.0.12

[howardpc]

It's an unsophisticated hack cobbled together to give you an idea (not the sort of thing you want in production code) but you can do something like the following; creating a descendant of TIniFile (or better of TCustomIniFile) to get you over one or two problematic .inis which have BOMs. This assumes the file is utf8 encoded:

Code: [Select]

unit noBOMIniFile;

{$mode objfpc}{$H+}

interface

uses IniFiles, FileUtil;

type

  { TNoBOMIniFile }

  TNoBOMIniFile=class(TIniFile)
    constructor Create(const AFileName: string; AEscapeLineFeeds: Boolean=False); override;
  end;

implementation

{ TNoBOMIniFile }

constructor TNoBOMIniFile.Create(const AFileName: string;
  AEscapeLineFeeds: Boolean);
var
  tex, tex2: TextFile;
  s, fn: string;
begin
  if FileExistsUTF8(AFileName) then begin
    AssignFile(tex, AFileName);
    try
      Reset(tex);
      readln(tex, s);
      if (s[1]=#$EF) and (s[2]=#$BB) and (s[3]=#$BF) then // probably a utf8 BOM
        begin
          fn:=AFileName+'noBOM.ini';
          AssignFile(tex2, fn);
          try
            Rewrite(tex2);
            WriteLn(tex2, copy(s, 4, Length(s)-3));
            while not EOF(tex) do begin
              readln(tex, s);
              writeln(tex2, s);
            end;
          finally
            CloseFile(tex2);
          end;
        end
      else fn:=AFileName;
    finally
      CloseFile(tex);
    end;
  end;
  inherited Create(fn, AEscapeLineFeeds);
end;

end. 


[engkin]

TIniFile has a constructor that takes a TStream. Simply use TFileStream on your file. Move the stream beyond the BOM mark then pass it to this constructor:

Code: [Select]

constructor TIniFile.Create(AStream: TStream; AEscapeLineFeeds : Boolean = False);


[Ocye]

That's my workaround:

Code: [Select]

procedure RemoveBOMFromFile(aFileName:string);
const
  UTF8BOM:string=#$EF#$BB#$BF;
var
  Content:string;
begin
  if FileExistsUTF8(aFileName) then
  with TFileStream.Create(aFileName,fmOpenReadWrite) do
  try
    if Size>3 then
    begin
      SetLength(Content,Size);
      Read(Content[1],Size);
      if Copy(Content,1,3)=UTF8BOM then
      begin
        Delete(Content,1,3);
        Seek(0,soBeginning);
        Write(Content[1],length(Content));
      end;
      SetLength(Content,0);
    end;
  finally
    Free;
  end;
end;


[engkin]

Usually you would use something like this:

Code: [Select]

  IniFile := TIniFile.Create(FileName);

Which does not work if the file starts with a BOM mark. My suggestion was to use something like this instead:

Code: [Select]

  FileStream := TFileStream.Create(FileName, fmOpenReadWrite);
  MarkHolder := FileStream.ReadDWord;
  if (MarkHolder and $00BFBBEF) = $00BFBBEF then
    FileStream.Position := 3
  else
    FileStream.Position := 0;

  IniFile := TIniFile.Create(FileStream);

Notice that all of these solutions do not convert your file to ANSI, and they expect the text in the file to include nothing over $7F. Otherwise IIRC UTF8 encoding uses 2 or more bytes for anything over $7F and the file needs to be converted to ANSI encoding.

Ocye's workaround deletes the BOM mark off the beginning of the file and overwrites the original file.

howardpc's generates another file with noBOM.ini added to the original file name.

Complete code:

Code: [Select]

uses
  ...Classes, IniFiles;
...
var
  IniFile: TIniFile;
  FileStream: TFileStream;
  FileName: String;
  MarkHolder: Cardinal;
...
begin
...
  FileName := ...;
  FileStream := TFileStream.Create(FileName, fmOpenReadWrite);//Use suitable mode for your usage, maybe fmOpenRead
  MarkHolder := FileStream.ReadDWord;
  if (MarkHolder and $00FFFFFF) = $00BFBBEF then
    FileStream.Position := 3
  else
    FileStream.Position := 0;

  IniFile := TIniFile.Create(FileStream);
//Use it as you would normally
...
  IniFile.Free;
  FileStream.Free;
end;

Edit:
Corrected the mask for MarkHolder. Thanks to circular

[typo]

You could take the opportunity to subclass the TIniFile, maybe with something like a TBOMIniFile or TXXICenturyIniFile.

TInifile class is lagged, but Delphi compatibility prohibits changes on TIniFile class directly, so subclass is the only way.

[CM630]

TInifile class is lagged, but Delphi compatibility prohibits changes on TIniFile class directly, so subclass is the only way.

Are you sure that it is compatible with Dephi? I doubt that older Delphis support unicode, while the Lazarus TIniFile does not support ASCII.
Also, I see no reason why this should stop the development of  TIniFile, simply a few working modes could be implemented.


I just started wondering- did anyone check if Delphi can read files with BOM?

[typo]

Anyway, if you want to make changes on TIniFile class, you should use the BugTracker.