Author Topic: [SOLVED] Determine line endings of the text file  (Read 719 times)

artem101

  • Full Member
  • ***
  • Posts: 149
[SOLVED] Determine line endings of the text file
« on: November 04, 2026, 06:37:02 pm »
Is there any built-in function in Lazarus to detect which line endings used in given file - CR, LF or CR+LF?


Code: Pascal  [Select][+][-]
  1. // something like this
  2. s := DetectLineEndings('/home/user/somefile.txt');
  3. writeln(s); // #10#13
« Last Edit: November 07, 2026, 01:12:16 pm by artem101 »

dsiders

  • Hero Member
  • *****
  • Posts: 1742
Re: Determine line endings of the text file
« Reply #1 on: November 04, 2026, 08:30:48 pm »
Is there any built-in function in Lazarus to detect which line endings used in given file - CR, LF or CR+LF?


Code: Pascal  [Select][+][-]
  1. // something like this
  2. s := DetectLineEndings('/home/user/somefile.txt');
  3. writeln(s); // #10#13

I can add this to my "Things I didn't know I needed" list. Apparently neither Lazarus nor FPC already include this specific capability. I'd go with an enumeration for the line ending styles though... instead of a string, like:

Code: Pascal  [Select][+][-]
  1. TLineEndingStyles = (lesUnknown, lesCRLF {windows}, lesCR {classic mac}, lesLFCR {serial,LABView}, lesLF{UNIX,Linux,modern macOS});
  2. function FileDetectLineEnding(AFileName; String): TLineEndingStyle;
  3. function StringDetectLineEnding(AString: string): TLineEndingStyle;

creaothceann

  • Sr. Member
  • ****
  • Posts: 481
Re: Determine line endings of the text file
« Reply #2 on: November 05, 2026, 01:54:51 am »
Not tested since it's late, but this should show one way to do it:

Code: Pascal  [Select][+][-]
  1. function DetectLineEndings(const s : AnsiString;  out LineEndings : AnsiString) : boolean;  // returns False if string is empty or ambiguous
  2. const
  3.         CR   = #13;
  4.         LF   = #10;
  5.         CRLF = CR + LF;
  6. var
  7.         i, Len : QWord;
  8.         Pair   : array[0..1] of AnsiChar;
  9. begin
  10.         Len := Length(s);  if (Len = 0) then exit(False);
  11.         LineEndings := '';
  12.         {$push}  {$RangeChecks off}
  13.         for i := 1 to Len do begin
  14.                 Move(s[i], Pair, 2);  // AnsiString is terminated by a #0, so this won't result in access violations
  15.                 if (Pair[0] >= #32) then continue;
  16.                 if (Pair = CRLF) then begin
  17.                         if (LineEndings = CRLF) then continue            else
  18.                         if (LineEndings = ''  ) then LineEndings := CRLF else exit(False);
  19.                 end else
  20.                 if (Pair[0] = CR) then begin
  21.                         if (LineEndings = CR  ) then continue            else
  22.                         if (LineEndings = ''  ) then LineEndings := CR   else exit(False);
  23.                 end else
  24.                 if (Pair[0] = LF) then begin
  25.                         if (LineEndings =   LF) then continue            else
  26.                         if (LineEndings =   '') then LineEndings :=   LF else exit(False);
  27.                 end;
  28.         end;
  29.         {$pop}
  30.         Result := True;
  31. end;
« Last Edit: November 05, 2026, 01:57:44 am by creaothceann »

Thausand

  • Hero Member
  • *****
  • Posts: 629
Re: Determine line endings of the text file
« Reply #3 on: November 05, 2026, 08:34:08 am »
if is need for determine lineending then is wrong implement. all example wrong because not take care /all/ platform lineneding. if not care all platform then not important and have know how detect lineending  :)
A docile goblin always follow HERMES.md

MarkMLl

  • Hero Member
  • *****
  • Posts: 8606
Re: Determine line endings of the text file
« Reply #4 on: November 05, 2026, 09:05:22 am »
I use this

Code: Pascal  [Select][+][-]
  1. var
  2.   lineBreak: string= '';
  3.  
  4.  
  5. (* Read up to 1024 bytes from the input file, and if there is an acceptable
  6.   line break set the global DefaultTextLineBreakStyle to match it. This will be
  7.   used not only to decide what will be written to the output files, but- more
  8.   importantly- what must be appended to each input and output line on-the-fly
  9.   when they are being hashed for MAC generation.
  10.  
  11.    If this fails nothing will be changed, i.e. DefaultTextLineBreakStyle will
  12.   depend entirely on the target platform.
  13. *)
  14. procedure InitLineBreakStyle(iFileName: string);
  15.  
  16. type
  17.   TiBlock= array[0..383] of byte;       (* I'd prefer this to be at least 1024  *)
  18.   TiFile= file of byte;
  19.  
  20. var
  21.   iblock: TiBlock;
  22.   iFile: TiFile;
  23.   i: integer;
  24.  
  25. begin
  26.   AssignFile(iFile, iFileName);
  27. {$push }{$I- }
  28.   Reset(iFile);
  29.   if IOResult <> 0 then
  30.     exit;
  31.   try
  32.     FillByte(iBlock{%H-}, SizeOf(iBlock), 0);
  33.     BlockRead(iFile, iBlock, SizeOf(iBlock));
  34.     if IOResult <> 0 then begin
  35.       case DefaultTextLineBreakStyle of
  36.         tlbsCRLF: lineBreak := #$0d#$0a;
  37.         tlbsCR:   lineBreak := #$0d
  38.       otherwise
  39.         lineBreak := #$0a;
  40.       end;
  41.       exit                              (* Via finally block                    *)
  42.     end;
  43.     for i := 0 to SizeOf(iBlock) - 2 do
  44.       case iBlock[i] of
  45.         $00: exit;                      (* Via finally block                    *)
  46.         $0a: begin
  47.                DefaultTextLineBreakStyle := tlbsLF;
  48.                lineBreak := #$0a;
  49.                break
  50.              end;
  51.         $0d: case iBlock[i + 1] of
  52.                $00: exit;               (* Via finally block                    *)
  53.                $0a: begin
  54.                       DefaultTextLineBreakStyle := tlbsCRLF;
  55.                       lineBreak := #$0d#$0a;
  56.                       break
  57.                     end
  58.              otherwise
  59.                DefaultTextLineBreakStyle := tlbsCR;
  60.                lineBreak := #$0d;
  61.                break
  62.              end
  63.       otherwise
  64.       end
  65.   finally
  66.     CloseFile(iFile)
  67.   end
  68. {$pop }
  69. end { InitLineBreakStyle } ;
  70.  

https://github.com/MarkMLl/dynamod/blob/main/hashlines.pas

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Logitech, TopSpeed & FTL Modula-2 on bare metal (Z80, '286 protected mode).
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Zvoni

  • Hero Member
  • *****
  • Posts: 3605
Re: Determine line endings of the text file
« Reply #5 on: November 05, 2026, 09:09:18 am »
Not tested since it's late, but this should show one way to do it:
Not bad, but it doesn't really answer op's question, since he's explicitly said "....from a file": Not a String, which might contain embedded LineEndings

I'd still keep your your "Pair", but expand your Consts to ALL possible LineEndings to get a real "generic" way
Then it would depend how i read the file: Line by line or in one go

In any case i would just check the LAST "Pair" against the possible LineEndings.
No Sense running through the whole text
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

creaothceann

  • Sr. Member
  • ****
  • Posts: 481
Re: Determine line endings of the text file
« Reply #6 on: November 05, 2026, 01:28:13 pm »
if is need for determine lineending then is wrong implement. all example wrong because not take care /all/ platform lineneding. if not care all platform then not important and have know how detect lineending  :)

artem101 was interested in CR, LF and CR+LF.


Not bad, but it doesn't really answer op's question, since he's explicitly said "....from a file": Not a String, which might contain embedded LineEndings

It's trivial to load a file into a string. If that takes too much RAM, would have to be handled via OS calls that map the file into the program's address space.


In any case i would just check the LAST "Pair" against the possible LineEndings.
No Sense running through the whole text

Yes, if the file is known to contain only one style or if the program isn't supposed to handle malformed files.


EDIT: Advancing a pointer through the string and casting that as needed to an array[0..1] of AnsiChar or AnsiChar might be faster than calling Move.
« Last Edit: November 05, 2026, 02:16:26 pm by creaothceann »

Zvoni

  • Hero Member
  • *****
  • Posts: 3605
Re: Determine line endings of the text file
« Reply #7 on: November 05, 2026, 03:13:21 pm »
Yes, if the file is known to contain only one style or if the program isn't supposed to handle malformed files.

Correct.
But think about it:
You create a new textfile on e.g. Winbloze (CRLF).
Windows itself "inserts" the LineEndings at the end of a string each time you hit return. There will be no single CR or single LF in that File, except if you manipulate those strings from some own code
Bottom Line: You have a 99.9999% Chance there will be only one style of LineEndings in a Textfile, but this depends entirely on the OS the TextFile is created on

and as i said: Your code isn't that bad.
It just bugged me, that you run FORWARD through the whole text without breaking after first HIT

If it's a really, really, really big textfile:
If you have enough RAM: Load the whole Textfile in one go
If not, read the first XX KB/MB

In both cases: Start at the end and look for the first hit on the LineEndings.
You have a 99.99999% Chance it's the LineEnding the File was created with

One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

creaothceann

  • Sr. Member
  • ****
  • Posts: 481
Re: Determine line endings of the text file
« Reply #8 on: November 06, 2026, 12:38:22 am »
It just bugged me, that you run FORWARD through the whole text without breaking after first HIT

That is an option, perhaps for a FastDetectLineEndings function. It wasn't clear if the purpose was a fast guess or a complete examinaton.


Start at the end and look for the first hit on the LineEndings.
You have a 99.99999% Chance it's the LineEnding the File was created with

Some weeks ago I actually came across some old files with mixed endings in the Wipeout source code, for example WIPESRC\PSX26\INCLUDE\R3000.H.

I used Notepad3 because of its fast startup time, and it showed a dialog about the line endings.
Notepad++ simply detects the same file as having Macintosh line endings (CR), and regular Win11 Notepad detects it as CRLF and UTF-8.

artem101

  • Full Member
  • ***
  • Posts: 149
Re: Determine line endings of the text file
« Reply #9 on: November 06, 2026, 06:53:14 pm »
I tried this

Code: Pascal  [Select][+][-]
  1. function DetectLineEndingsInFile(const AFileName: string): string;
  2. {var
  3.   s:string;
  4.   sl:TStringList;
  5. begin
  6.   sl:=TStringList.Create;
  7.   try
  8.     sl.LoadFromFile(AFileName);
  9.     s:=sl.Text;
  10.  
  11.     if not DetectLineEndings(s, result) then
  12.       result := '';
  13.   finally
  14.     sl.free;
  15.   end;
  16. end;       }
  17. var
  18.   f:file;
  19.   s:string;
  20.   a: array[0..1023] of byte;
  21.   cnt:integer;
  22.   len:integer;
  23.   i:integer;
  24. begin
  25.   AssignFile(f, AFileName);
  26.   reset(f);
  27.   len:=  {min(SizeOf(a), FileSize(f))}1024;
  28.   BlockRead(f, a, len, cnt);
  29.   {s := UTF8Decode(StringOf(a));
  30.   if not DetectLineEndings(s, result) then
  31.     result := '';  }
  32.   result :='';
  33.   for i := 0 to cnt-1 do
  34.   begin
  35.     if (a[i] = 13) and ( a[i+1] = 10) then
  36.       Result := #13#10
  37.     else if (a[i] = 10) and ( a[i+1] = 13) then
  38.       Result := #10#13
  39.     else if (a[i] = 10) then
  40.       Result := #10
  41.     else if (a[i] = 13)then
  42.       Result := #13;
  43.  
  44.     if result <> '' then
  45.       break;
  46.   end;
  47.   CloseFile(f);
  48. end;      

but it fails with sigsegv

creaothceann

  • Sr. Member
  • ****
  • Posts: 481
Re: Determine line endings of the text file
« Reply #10 on: November 07, 2026, 12:07:27 am »
Use Reset(f, 1); to set the block size used by BlockRead to 1, or use file of byte.

Btw. when i is 1023, a[i + 1] will either result in a range error (if those are enabled) or access the memory after a.

Bart

  • Hero Member
  • *****
  • Posts: 5800
    • Bart en Mariska's Webstek
Re: Determine line endings of the text file
« Reply #11 on: November 07, 2026, 11:30:35 am »
Some observations:
I've never encountered #10#13 to be a line-ending (code from artem101).
In practice you can encounter files that have more than 1 style of line-endings (mixed Windows/Unix).
W.r.t. files: you might consider checking first if the file is indeed single byter encoded or UTF8 (and not LE/BE 16-bit). LaUtils has code for that somewhere b.t.w.

What would be the prurpose of detecting the LineEnding style in a text file?
In situations where text files may have any style (Mac, Win, Unix) I tend to just call AdjustLineBreaks() so the text (in memory) has a Line-Ending style which suits the OS i'm currently working in.

Bart

Thausand

  • Hero Member
  • *****
  • Posts: 629
Re: Determine line endings of the text file
« Reply #12 on: November 07, 2026, 12:40:44 pm »
artem101 was interested in CR, LF and CR+LF.
Yes, I read message artem :)

when not know then not know.

I've never encountered #10#13 to be a line-ending (code from artem101).
This is small example for when not know then not know.

newline wiki

And not have forget modern tekst and have unicode, that more fun  :)
« Last Edit: November 07, 2026, 12:43:02 pm by Thausand »
A docile goblin always follow HERMES.md

artem101

  • Full Member
  • ***
  • Posts: 149
Re: Determine line endings of the text file
« Reply #13 on: November 07, 2026, 01:03:10 pm »
Btw. when i is 1023, a[i + 1] will either result in a range error (if those are enabled) or access the memory after a.

Thanks for remark. This should solve possible range issue:
Code: Pascal  [Select][+][-]
  1. for i := 0 to cnt-2 do
  2. ...

artem101

  • Full Member
  • ***
  • Posts: 149
Re: Determine line endings of the text file
« Reply #14 on: November 07, 2026, 01:12:01 pm »
Use Reset(f, 1);

With this change my code now correctly works. Problem solved. Thanks!
« Last Edit: November 07, 2026, 05:14:44 pm by artem101 »

 

TinyPortal © 2005-2018