A Tale of Two File Systems

[stephanos]

Greetings all

I have what I hope is the last technical issue due to my move from Windows to Linux.  I used Lazarus (Windows, 32 bit) to write a playlist writer for my Sandisk mp3 player.  The programme and the playlist file sit in the music folder along with my sub folders of ripped dvds. on the player. The programme opens the m3u file, allows me to select multiple files on the player and their path\filenames are stored in a string and the string is written to a memo, where editing might be done, and then written to the m3u file.

As all my mp3 files are stored on the player I wanted to be able to rewrite the programme to work in Linux.  This rewrite presented me with multiple issues which this forum helped me with. 

The file format of the mp3 player is FAT32.  The m3u file I am using was created by me using Notepad.  When I use the Linux programme and connect the player to a Linux computer, the programme works well.

When using the Linux version of the programme on a Linux computer, I can open the m3u file and load its contents into the memo, append paths\filenames to the memo, save the memo content to the m3u file, overwriting the existing content.  However, when the player is disconnected from the Linux computer and the playlist is selected, nothing happens.  Using the player I can look inside the m3u file and it is empty.  I can reconnect the play and use kate to read the contents of the m3u file.  It is not empty.

Writing to the playlist file in a Linux environment works, but the content is not read by the mp3 player.

Kate can read the m3u file.  Notepad can read the m3u file.  Both can write to the m3u file.  But when the Linux version of the programme is used to write to the m3u file, the mp3 player cannot read the m3u file.

This is how I determined the file system
Connected the mp3 player to the Linux computer, and ran:

• lsusb
  Bus 001 Device 026: ID 0781:75a2 SanDisk Corp. Clip Sport Go

to determine that the player was seen and connected, then

• sudo dmesg
  [86537.283159] usb 1-3.1.2: Product: Clip Sport Go
  [193389.731436]  sdd: sdd1

to determine the device name, then

• sudo udevadm info /dev/sdd1
  E: ID_FS_VERSION=FAT32

to confirm the file system as fat32

File systems are my weakest area.  I am sure I am going to ask lots of questions.

Meanwhile, much help needed, any help appreciated

Stephanos

[Nimbus]

Could be as simple as Unix vs Windows line endings in the m3u file?
See if setting your memo's Lines.LineBreak property to Windows style endings changes anything.

Code: Pascal  [Select][+][-]

1. Memo1.Lines.LineBreak := #13#10;

[stephanos]

Thanks

Does it matter where I place this line of code.  I put it in the FormCreate procedure.  And it did not solve the problem.

I have been thinking about the problem.  I tried this:
I made a new Text file and a string:

Code: Pascal  [Select][+][-]

1. myLogFile : TextFile;
2. listofsongs : string;

I copied the memo text to the string

Code: Pascal  [Select][+][-]

1. listofsongs := Memo1.Lines.Text;

I replaced #10 with #13#10

Code: Pascal  [Select][+][-]

1. StringReplace(listofsongs, '#10', '#13#10', [rfIgnoreCase, rfReplaceAll]);

I made a link to the m3u file, opened the file,
                                   // The path/filename.m3u

Code: Pascal  [Select][+][-]

1. AssignFile(myLogFile, Localm3uPathFile); ReWrite(myLogFile);

and wrote the string to the file

Code: Pascal  [Select][+][-]

1. WriteLn(myLogFile, listofsongs); CloseFile(myLogFile);

The correct content is written to the m3u file, but the content is still not readable via the mp3 player

Well I enjoyed experimenting.

Any ideas welcome.

[Thaddy]

Is the fat32 partition mounted on linux, i.e. can you do ls on the directory?
Does ls /mnt list the drive letter(s)?
Did you specify the Linux path correctly? e.g. /mnt/c/mp3dir or something similar.
/mnt may also be /media on some systems.

[AlexTP]

- You get the ready playlist for the player.
- You have the path to the player.

so why not to try to run from terminal: path_to_player path_to_playlist?
What error do you see? Can you fix it here?

[Thausand]

I replaced #10 with #13#10

and wrote the string to the file

Code: Pascal  [Select][+][-]

1. WriteLn(myLogFile, listofsongs); CloseFile(myLogFile);

The correct content is written to the m3u file, but the content is still not readable via the mp3 player

That no work. Memo and #13#10 and writeln all make change to LineEnding. I read sandisk forum message user mac that have problem lineending sandisk m3u work only when use CRLF.

Make forget memo and use stringlist then make special option stringlist for linebreak as write nimbus.

Code: Pascal  [Select][+][-]

1. program test;
2.  
3. {$mode objfpc}{$h+}
4.  
5. uses
6.   classes, sysutils;
7.  
8. procedure WriteM3U(FileName:string);
9. var
10.   List:TStringList;
11.   Index:integer;
12. begin
13.   List:=TStringList.Create;
14.   List.Add('#EXTM3U');
15.   List.Add('#EXTINF:419,Alice in Chains - Rotten Apple'); // no require but may be sandisk need ?
16.   List.Add('Alice in Chains_Jar of Flies_01_Rotten Apple.mp3');
17.  
18.   List.SkipLastLineBreak:=True;
19.   List.LineBreak:=#13#10;
20.   List.SaveToFile(Filename);
21.   List.Free;
22. end;
23.  
24. begin
25.   WriteM3U('test.m3u');
26. end.
27.  

This simple fpc command test but can make work with Lazarus if want.

Then if test.m3u is work (make change example mp3 name with one or two mp3 name you have) then can make fancy Lazarus memo.

If not work then make playlist m3u manual at sandisk and have share with forum so can see and make fix.

[CM630]

You can try:
1. Make a copy of the working file.
2. Edit the file with the tool that damages it.
3. Compare both files with a hex viewer.
4. Post the files here if you still cannot handle it yourself.

Linux and Android cannot handle properly non-Latin filenames in FAT32, but I doubt that this is related.

[hcoenen]

You might want to try to create a playlist on your player first and work backwards from there,

I took a peek at wikipedia and character encoding might come into play as well.

[stephanos]

Thanks all

A tiny bit of background: Because the Windows path is fixed I have specified a file name for the m3u, PlayListStephen.m3u.  It can only be in one place/path
?:\Sport Go\Music\PlatListStephen.m3u
However, in the Linux version, and so it can be used on any Linux computer the path/ is not known, so that file has to be selected.

Summary of problem: Because the file system of the mp3 player is FAT 32 the Linux version of the programme opens and writes to the m3u file, but the player cannot read the file content.
The Windows version works properly

I experimented with TstringLists:

procedure TForm1.BitBtn5Click(Sender: TObject);  // Save 
declared a TstringList

Code: Pascal  [Select][+][-]

1. TListofsongs : TstringList;

Begin

Code: Pascal  [Select][+][-]

1. TListofsongs := TStringList.Create;

Copied the global string, that contains the m3u path/filename.m3u, plus a #10, into a local string. 
There is also a #10 at the end.  It stops the programme writing to the m3u file.
                       Local string :=  global string

Code: Pascal  [Select][+][-]

1.  Localm3uPathFile := m3uPathFile;

Removed the #10

Code: Pascal  [Select][+][-]

1. Tag := Pos(#10, Localm3uPathFile, 1);
2. Localm3uPathFile := copy(Localm3uPathFile, 1, (Tag-1));

Copied the content of the Memo to the TString. 

Code: Pascal  [Select][+][-]

1. TListofsongs.Add(Memo1.Lines.Text);

Amended the line endings

Code: Pascal  [Select][+][-]

1. TListofsongs.LineBreak := '#13#10';

Write the Tstring to the m3u file

Code: Pascal  [Select][+][-]

1. Tlistofsongs.SaveToFile(Localm3uPathFile);

I was surprised that no writing to the file took place.  The file remained empty.

I was able to check that TListofsongs had content by copying it to a local string and using the local string for a caption

Code: Pascal  [Select][+][-]

1. Listofsongs := TListofsongs.Text;
2. Label3.Caption := Listofsongs;

So what was to be written to the file was also output as a caption to a label

Now the two files idea
In Windows I used a new created notepad file and called it PlayListStephen.m3u.  It was blank
I ran the Windows programme and opened the file and appended 3 songs.  I exited the programme, removed the player and l used the playlist function to select and look inside the file.  The list was complete.
I did the same procedure in Linux and called the file Lplist.m3u.  After removing the player Lplist.m3u was visible as a playlist on the player but was empty.  So I re-attached the player and used Kate to view Lplist.m3u.   The content was correct. 

I have attached both

Blimey this is a mammoth effort and thanks to all

[Thausand]

File lplist.m3u not have $0d end and tell why not work.

Example my show how make list have #13#10.

Not able write file may be when not have access right for directory (or file). If create playlist sandisk then may be owner or access right change.

[stephanos]

Dear All

I regret I cannot understand most of the last post.  Sorry.  What seems evident is that the m3u file is accessed by the Linux version, data is written to it, but the player does not read the file.  Kate and Notepad can read the file.

However, there has been two developments.  I tried another search and found references to other programmes that convert a text file from Linux to Windows.  Mostly replacing the #10 with #13#10.
The first was a text editor called gedit.  I was able to save it with a Windows file endings – It did not work.  Next was installation of a simple command line programme unix2dos.  I installed it
     "sudo apt-get install unix2dos"
and ran this command
     "unix2dos /media/stephanos/"Sport Go"/Music/PlayListStephen.m3u "
It worked.  The content of PlayListStephen.m3u is readable by the player.
I consider this to be half a solution.  Because at the command line I will need to know the path to the file. I am not using the string that has the path/filename for that computer.  I want to be able to run this programme from any Linux computer, and the path will be different.

I progressed.  I tied this code

Code: Pascal  [Select][+][-]

1. var  Localm3uPathFile, Listofsongs : string;
2.                                     TListofsongs : TStringList;
3.                                        myLogFile : TextFile;  
4. Begin
5. Localm3uPathFile := m3uPathFile; // copy global string to local string
6. Tag := Pos(#10, Localm3uPathFile, 1); // find the position of #10 at the end of path/filename
7. Localm3uPathFile := copy(Localm3uPathFile, 1, (Tag-1)); // remove #10
8. listofsongs := Memo1.Lines.Text;  // copy memo text to a string
9. listofsongs := StringReplace(listofsongs, #10, #13#10, [rfIgnoreCase, rfReplaceAll]);
10. AssignFile(myLogFile, Localm3uPathFile);
11. ReWrite(myLogFile); WriteLn(myLogFile, listofsongs);
12. CloseFile(myLogFile);

A 75% success.  The m3u file is readable by the player.  However, when the m3u file is viewed in Kate, each line of a path/song.mp3 has a blank line between them.

     "Andy Irvine\Way Out Yonder\Gladiators.mp3

      Andy Irvine\Way Out Yonder\Theyll Never Believe Its True_froggys Jig.mp3

      Andy Irvine\Way Out Yonder\Way Out Yonder.mp3

      Baggyrinkle\Old Swansea Town\A Long Time Ago.mp3


      "
It works.  However, I want to improve it, if I can.  When I run the Linux programme again, and I assume when I run the Windows programme again, there are these blank lines in the Memo window.  Often I will have over 20 lines of path/filename.mp3.  So I will have to delete these lines.  I would prefer they were not present.

Alternatively, am I able to run the command

Code: Pascal  [Select][+][-]

1. unix2dos /media/stephanos/"Sport Go"/Music/PlayListStephen.m3u

as

Code: Pascal  [Select][+][-]

1. unix2dos Localm3uPathFile;

within the Linux/Lazarus programme?

Wait to hear, and thanks

[Lutz Mändle]

Your file Lplist.m3u in the archive has a blank char (20hex) at the first character of the filename, maybe from there comes the problem reading the file.

The appended file is the result of ls -tr *.m3u > ls.txt in the directory containing the files of the archive.

Otherwise the only difference I can see is the line ending, the windows version has crlf (#13#10) and the linux version has only lf (#10), as expected.

[Bart]

The AdjustLineBreaks() procedure can probably do what you want (no need to re-invent the wheel).
(And Trim() to remove leading and trailing spaces)

Bart

[Thausand]

Hello stephanos,

I sorry delay. Many problem forum, not possible login and have cookie error.

I regret I cannot understand most of the last post.  Sorry.

No need sorry (I make sorry for bad English).

Better write what not understand

However, when the m3u file is viewed in Kate, each line of a path/song.mp3 has a blank line between them.

#13 decimal = $0d hexadecimal
#10 decimal = $0a hexadecimal

Is same number and is LineEnd.

Windows have LineEnd #13#10 and is small name CRLF (Carriage Return + Line Feed), Linux have LineEnd #10 (Line Feed) and is small name LF

Wiki write better english: https://wiki.freepascal.org/Multiplatform_Programming_Guide#Working_with_files_and_folders

Alternatively, am I able to run the command unix2dos /media/stephanos/"Sport Go"/Music/PlayListStephen.m3u as unix2dos Localm3uPathFile within the Linux/Lazarus programme?

Yes, can but do not. Too complicate and not help solve problem kate.

I firm write: This problem for Kate not you. Text view have windows and linux LineEnd automatic or can have convert fast option. If not have then editor no good.

TStringList have same ancestor TMemo.Lines and work same for LineEnd. That is why I write not use Memo because TMemo confuse with obscufate. You example try do many thing same time. First make work simple then make improve.

Code: Pascal  [Select][+][-]

1. procedure ExampleM3U(FileName:string);
2. var
3.   List:TStringList;
4.   Index:integer;
5. begin
6.   List:=TStringList.Create;
7.  
8.   // Line last not have empty line
9.   List.SkipLastLineBreak:=True;
10.  
11.   // make item and write. No complicate path and make path later.
12.   List.Add('Gladiators.mp3');
13.   List.Add('Theyll Never Believe Its True_froggys Jig.mp3');
14.   List.Add('Way Out Yonder.mp3');
15.   List.Add('A Long Time Ago.mp3');
16.  
17.   List.LineBreak:=#13#10;
18.   List.SaveToFile(Filename); // CRLF
19.  
20.   // Have list empty
21.   List.Clear;
22.  
23.   // read M3u
24.   List.LoadFromFile(Filename); // CRLF
25.  
26.  
27.   // write m3u.LF
28.   List.LineBreak:=#10;
29.   List.SaveToFile(Filename+'.LF.txt'); // LF
30.  
31.   // write m3u.CRLF
32.   List.LineBreak:=#13#10;
33.   List.SaveToFile(Filename+'.CRLF.txt'); // CRLF
34.  
35.   List.Free;
36. end;
37.  
38. ..
39. ExampleM3U('test.m3u');
40. ..
41.  

No need make more difficult and have 1:1 translate when use TMemo.Lines.

[CM630]

@Stephanos, I did not understand if you have solved the issue.
Since you wrote that you have switched to Linux, I would recommend you to install a decent file manager first, something like Double Commander (actually, I do not know if there is another decent FM for Linux).

Here is what it shows: both files have different new line separators. If one of the files works, and the other does not, you should use the same separator as the working one.