Print Page - Trying to get My Documents folder

Programming => Operating Systems => Windows => Topic started by: Josh on July 18, 2017, 11:29:26 pm

Title: Trying to get My Documents folder
Post by: Josh on July 18, 2017, 11:29:26 pm

Hi

I am using latest trunk to test some old units out and have come across a problem, with my routine that get user's Documents Folder.

The routine is working fine, when user has standard characters; but when user has character like áé etc, then it fails and raises the exception in the code.

Code: Pascal [Select][+]

function GetMyDocuments: string;
var
  r: Bool;
  path: array[0..Max_Path] of Char;
begin
  r := ShGetSpecialFolderPath(0, path, CSIDL_Personal, False) ;
  if not r then raise Exception.Create('Could not find MyDocuments folder location.') ;
  path:=SysToUTF8(path);
  Result := Path;
end;
 

Title: Re: Trying to get My Documents folder
Post by: Handoko on July 19, 2017, 02:58:42 am

Maybe this can help:
http://forum.lazarus.freepascal.org/index.php?topic=3109.0

Title: Re: Trying to get My Documents folder
Post by: PatBayford on July 19, 2017, 05:25:04 am

The problem you are having josh, is that most versions of Windows DO NOT "speak" UTF8/16, or even necessarily Unicode correctly. You will need to ensure that any strings passed to Windows API functions are converted to old-fashioned ASCII code strings.
PS - don't forget that Windows strings need to be enclosed in quotes if they contain spaces.

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 06:18:17 am

@Josh

Quote

I am using latest trunk to test some old units out and have come across a problem, with my routine that get user's Documents Folder.
The routine is working fine, when user has standard characters; but when user has character like áé etc, then it fails and raises the exception in the code.

Try something like this:

Code: Pascal [Select][+]

uses shlobj;
 
function GetSpecialFolder(const CSIDL: Integer): WideString;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := SpecialPath;
  finally
    StrDispose(SpecialPath);
  end;
end;
 
procedure TForm1.Button1Click(Sender: TObject);
begin
  ShowMessage(String(GetSpecialFolder(CSIDL_PERSONAL)));
  //ShowMessage(UTF16ToUTF8(GetSpecialFolder(CSIDL_PERSONAL))); //for FPC 2.6.4 --> unit LazUTF8
end; 

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 08:22:53 am

Hi Getmem,

Thanks for your code, if I use this on my old lazarus installs it indeed works, as does my code.

If I use this code on Lazarus Trunk, It does not work and produces no result, I have modified the code to show an 'Oops' message which is triggered if the call to SHGetSpecialFolderPathW fails. Which it does on Trunk.

Is it possible I have found a bug in Lazarus Trunk?

Code: Pascal [Select][+]

function GetSpecialFolder(const CSIDL: Integer): WideString;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := SysToUTF8(SpecialPath)
    else showmessage('Oops');   
  finally
    StrDispose(SpecialPath);
  end;
end;          
 

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 08:31:31 am

Quote

If I use this code on Lazarus Trunk, It does not work and produces no result, I have modified the code to show an 'Oops' message which is triggered if the call to SHGetSpecialFolderPathW fails. Which it does on Trunk.
Is it possible I have found a bug in Lazarus Trunk?

I also use Lazarus trunk and the code works fine. It's more likely a windows related issue(privilege, elevation, etc...). Please try:

Code: Pascal [Select][+]

ShowMessage(SysErrorMessage(GetLastOSError))

instead of:

Code: Pascal [Select][+]

ShowMessage('Oops');

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 08:36:38 am

Hi the error displayed adding that in produces the following error on trunk only.
The System Cannot find the path specified

Please note this is on the same computer, running both lazarus versions at the same time.

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 08:51:15 am

I have no idea why SHGetSpecialFolderPathW api fails on your computer. You can also try:

Code: Pascal [Select][+]

uses windirs;
begin
  ShowMessage(GetWindowsSpecialDir(CSIDL_PERSONAL));
end;

Code: Pascal [Select][+]

uses LazFileUtils;
begin
  ShowMessage(AppendPathDelim(GetUserDir + 'Documents'));
end;

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 09:03:23 am

Hi

ShowMessage(GetWindowsSpecialDir(CSIDL_PERSONAL));

also on trunk is displaying a blank, but on old old lazarus is displaying ok.

I do not think it is related to this machine, as I have tried on other machines and windows versions that I have , as soon as the user name has a utf8 character in it then it is unable to find the Documents folder.
I have also sent both compiled binaries with same code to a friend in Finland, and he has just reported back that only the old version is showing the correct location and the second version is displaying the error.

My test SVN version is 55492, I will download the very latest version, in case something has changed in the last couple of days?

Addition
GetUserDir is displaying the correct value. Unfortunately this is not usable as 'Documents' is not guaranteed to be correct across windows os's etc.

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 09:55:21 am

Quote from: josh on July 19, 2017, 08:22:53 am

Is it possible I have found a bug in Lazarus Trunk?
Code: Pascal [Select][+][-]
function GetSpecialFolder(const CSIDL: Integer): WideString;
...
Result := SysToUTF8(SpecialPath)

No, you added the bug all by yourself. Instead of using the working code provided by GetMem, you added a conversion that is doomed to fail. Why?
You should know the W(ide) versions of WinAPI, like SHGetSpecialFolderPathW, work with Unicode, UTF-16.
That is why the parameter SpecialPath is PWideChar.
GetMem and everybody: I think the return type of your function should be UnicodeString instead of WideString. WideString should be used only with OLE programming.

Quote from: PatBayford on July 19, 2017, 05:25:04 am

The problem you are having josh, is that most versions of Windows DO NOT "speak" UTF8/16, or even necessarily Unicode correctly. You will need to ensure that any strings passed to Windows API functions are converted to old-fashioned ASCII code strings.

That is not correct. The WinAPI functions that work with old-fashioned ANSI strings do not work with Lazarus Unicode system. You must use the W versions that work with Unicode.
Please don't spread false information.
Unicode should be used everywhere in any case because ANSI <-> Unicode string conversions are lossy. The old ANSI string support can be seen as a historical remain, retained only for backwards compatibility.
Unicode in Lazarus + WinAPI are explained here:
http://wiki.freepascal.org/Unicode_Support_in_Lazarus#Calling_Windows_API

Title: Re: Trying to get My Documents folder
Post by: wp on July 19, 2017, 10:09:19 am

The original function of the first post is working perfectly on my Win7 VM with a special user "UmlautUser-äöüÄ". The only thing that needs to be changed is that SysToUTF8 must be replaced by WinCPToUTF8 on fpc 3+.

So, I guess the issue has something to do with missing rights or similar, but not with utf8/unicode.

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 10:16:01 am

Quote from: wp on July 19, 2017, 10:09:19 am

The original function of the first post is working perfectly on my Win7 VM with a special user "UmlautUser-äöüÄ". The only thing that needs to be changed is that SysToUTF8 must be replaced by WinCPToUTF8 on fpc 3+.

It works if all the characters belong to the current codepage. Isn't it so?
There is no reason to gamble with codepages any more. Just use Unicode and you are good.

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 10:16:23 am

Quote

I think the return type of your function should be UnicodeString instead of WideString. WideString should be used only with OLE programming.

You can use both. UnicodeString is reference counted on the other hand WideString basically is the same good old COM BSTR type windows always used, without ref. count. WideString is slower though, since windows makes a copy on winapi calls. Unless you don't call a winapi 1000x times, you should be fine. The funny thing is UnicodeChar points back to WideChar. :)

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 10:20:16 am

Hi

The problem is that it is not even getting to the systoutf8 function, the function SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) is returning FALSE, and the error being reported is 'The System Cannot find the path specified'.

I have tried using WinCPToUTF8 but this has no change in that the error is prior to this.

I can compile the small test app on old laz and new laz, place both files in same directory and one will work and one will not displaying the message 'System cannot find the path'

code now using

Code: Pascal [Select][+]

function GetSpecialFolder(const CSIDL: Integer): UnicodeString ;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := WinCPToUTF8 (SpecialPath)
    else ShowMessage(SysErrorMessage(GetLastOSError));
  finally
    StrDispose(SpecialPath);
  end;
end;       
 

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 10:20:37 am

Should SHGetSpecialFolderPath actually still be used ??
https://msdn.microsoft.com/en-us/library/windows/desktop/bb762204(v=vs.85).aspx

Quote

SHGetSpecialFolderPath is not supported. Instead, use ShGetFolderPath

And ShGetFolderPath is deprecated in favor of SHGetKnownFolderPath.
https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181(v=vs.85).aspx

FPC also uses that one in windirs.pp

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 10:23:35 am

@rvk
According to @Josh SHGetKnownFolderPathW also fails, see a few post above.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 10:25:12 am

Quote from: GetMem on July 19, 2017, 10:23:35 am

@rvk
According to @Josh SHGetKnownFolderPathW also fails, see a few post above.

~~Really? Can you quote that part? I can't find it.~~
Edit: Woops, I found it.
GetWindowsSpecialDir doesn't work either.

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 10:42:27 am

Quote from: josh on July 19, 2017, 10:20:16 am

The problem is that it is not even getting to the systoutf8 function, the function SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) is returning FALSE, and the error being reported is 'The System Cannot find the path specified'.

Why don't you just test with the working function provided by GetMem? Why must you modify it first?
[Edit] Ok, I saw you had tried it as-is first. Why the WinAPI function returns FALSE is a mystery then.

Just to be sure: You use default settings in Lazarus, right?
It is possible to disable the default Unicode support by -dDisableUTF8RTL as explained here:
http://wiki.freepascal.org/Lazarus_with_FPC3.0_without_UTF-8_mode
... although even it should not affect the value returned from WinAPI.

Quote

I have tried using WinCPToUTF8 but this has no change in that the error is prior to this.

Uhhh, no! For some reason you fail to understand the fundamentals.
With the current Unicode support in Lazarus you practically never need explicit conversion functions.
No SysToUTF8, no WinCPToUTF8, no AnsiToUTF8, nothing.
FPC 3.0+ has dynamic string encodings. Data is converted automatically when it is assigned.
The only exception is when you really must use Windows system codepages, but in your case it is not necessary.
When you use only Unicode, things go smoothly.

wp and others, please don't recommend system codepages to him or anybody else.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 10:52:53 am

Hi Guys,

Something strange is happening.......Some progress

using the code

Code: Pascal [Select][+]

function GetSpecialFolder(const CSIDL: Integer): WideString;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := SysToUTF8(SpecialPath)
    else ShowMessage(SysErrorMessage(GetLastOSError));
  finally
    StrDispose(SpecialPath);
  end;
end;       
 

This works from ide and if app is launched on old laz
On trunk, it does not work from ide, but does work if I launch the compiled application.

Even though this is progress, in that my application may work when run directly, I am unable to use ide to develop my apps.

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 10:56:09 am

@Juha
According to @Josh the function "SHGetSpecialFolderPathW" fails. The api returns a BOOL type which in his case is false. It has nothing to do with Lazarus and any conversion below the function SHGetSpecialFolderPathW. The last OS error is: "The System Cannot find the path specified". The question is why? It could be a privilege or elevation issue maybe an Antivirus?

@Josh
Try to run lazarus trunk as admin(see image), disable AV and remove SysToUTF8 it's not needed as @Juha explained earlier.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 11:00:02 am

Almost got trunk installed in a VM. So I'll try it there.
(edit: Works fine for me in Windows 7, Lazarus 1.6.4 and trunk in and out of the IDE)

But if you are using trunk, I don't think you should use WideString and SysToUTF8.
(Aren't you getting a lot of warnings about string conversions??)

Try this in trunk:

Code: Pascal [Select][+]

uses windirs, ShlObj;
 
function GetSpecialFolder(const CSIDL: Integer): String;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := SpecialPath
    else ShowMessage(SysErrorMessage(GetLastOSError));
  finally
    StrDispose(SpecialPath);
  end;
end;
 
procedure TForm1.Button1Click(Sender: TObject);
begin
  Memo1.Lines.Add(GetWindowsSpecialDir(CSIDL_PERSONAL));
  Memo1.Lines.Add(GetSpecialFolder(CSIDL_PERSONAL));
end;
 

B.T.W. What was wrong with using GetWindowsSpecialDir from windirs? Or does that also not work in your latest tests?

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 11:00:13 am

Hi

I have tried without systoutf8, but the error is before this, the oddity which you probably missed is that the error is occurring when running in ide.

I have created a video, which I will upload somewhere in minute, so you can see the oddity ( note the vid has the systoutf8 in it, but I have tested with and without and the result is the same).

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 11:04:03 am

Quote from: GetMem on July 19, 2017, 10:16:23 am

You can use both. UnicodeString is reference counted on the other hand WideString basically is the same good old COM BSTR type windows always used, without ref. count. WideString is slower though, since windows makes a copy on winapi calls.

That's why UnicodeString should be used instead. WideString is a wrong type here. Why to use it without any benefit?
FPC developer Marco has repeated the same thing: use UnicodeString.

Quote

The funny thing is UnicodeChar points back to WideChar. :)

Yes, there is no memory management involved. The types are identical.

Title: Re: Trying to get My Documents folder
Post by: balazsszekely on July 19, 2017, 11:10:52 am

@Juha

Quote

Why to use it without any benefit?

Widestring is available from delphi4 up, is like a good old friend. Besides I have a few, old projects in delphi 2007 where UnicodeString is not yet available. I'm using for historical reasons, no real benefit.

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 11:15:50 am

Quote from: GetMem on July 19, 2017, 11:10:52 am

Widestring is available from delphi4 up, is like a good old friend. Besides I have a few, old projects in delphi 2007 where UnicodeString is not yet available. I'm using for historical reasons, no real benefit.

Ok, that is a good reason. :)

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 11:22:58 am

Hi

uploaded a video showing my problem, I also changed the code to not use systoutf8

https://youtu.be/V9dGD1k3XLA

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 11:23:09 am

Quote from: josh on July 19, 2017, 11:00:13 am

I have created a video, which I will upload somewhere in minute, so you can see the oddity ( note the vid has the systoutf8 in it, but I have tested with and without and the result is the same).

We believe the problem is real. Did you test privileges as GetMem suggested? It sounds like the most probable cause now.

Sorry for nagging but if you still test with a seriously wrong SysToUTF8() conversion, it means you just don't understand the string types used by WinAPI "W" functions, FPC 3.0+ and Lazarus. Please study them!
This is about Lazarus:
http://wiki.freepascal.org/Unicode_Support_in_Lazarus

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 11:24:59 am

Quote from: josh on July 19, 2017, 11:22:58 am

https://youtu.be/V9dGD1k3XLA

This video is unavailable.

Could you also post a complete test-project?

And please specify what OS and what version of Lazarus work and doesn't work.
(just "old laz" doesn't really cover it)

And did you also test the GetWindowsSpecialDir from windirs?

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 11:28:18 am

Hi JuhaManninen
I am now not using systoutf8, and the error is still occuring in the IDE, the first 20 seconds of the video, show the simple app running in the IDE, you can see the code of the function. the second half of video, show the application that was created 20 seconds earlier running correctly when launched outside of the ide.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 11:42:22 am

@rvk

New Project, with tbutton on it and onclick event.

The code below on mine displays error when ran in IDE, but operates correctly when launched directly.

The old laz lazarus 1.3 fpc 2.71, hense the use of the strtoutf8 used previously, this has now been removed.

Hopefully the video will start to appear soon...

Code: Pascal [Select][+]

unit Unit1;
 
{$mode objfpc}{$H+}
 
interface
 
uses
  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, StdCtrls,shlobj;//,LazUTF8 ;
 
type
 
  { TForm1 }
 
  TForm1 = class(TForm)
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
 
  public
 
  end;
 
var
  Form1: TForm1;
 
implementation
 
{$R *.lfm}
 
{ TForm1 }
 
function GetSpecialFolder(const CSIDL: Integer): WideString;
var
  SpecialPath: PWideChar;
begin
  Result := '';
  SpecialPath := WideStrAlloc(MAX_PATH);
  try
    FillChar(SpecialPath^, MAX_PATH, 0);
    if SHGetSpecialFolderPathW(0, SpecialPath, CSIDL, False) then
      Result := SpecialPath
    else ShowMessage(SysErrorMessage(GetLastOSError));
  finally
    StrDispose(SpecialPath);
  end;
end;
 
procedure TForm1.Button1Click(Sender: TObject);
begin
  ShowMessage(String(GetSpecialFolder(CSIDL_PERSONAL)));
 
 
 // ShowMessage(GetWindowsSpecialDir(CSIDL_PERSONAL));
  //ShowMessage(UTF16ToUTF8(GetSpecialFolder(CSIDL_PERSONAL))); //for FPC 2.6.4 --> unit LazUTF8
end;
 
 
end.
 

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 11:42:58 am

Quote from: josh on July 19, 2017, 11:28:18 am

I am now not using systoutf8, and the error is still occuring in the IDE, ...

True, the error is not related to string conversions. It is likely a privileges issue.
However you need to understand the strings issues for your future programming. That's why I nagged. :)

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 12:07:04 pm

@Tried running lazarus as admin, but this is not working as doing so the Documents folder is the administrator's Documents folder, not the Current Users Document Folder.

Title: Re: Trying to get My Documents folder
Post by: Thaddy on July 19, 2017, 12:22:24 pm

There's a difference between the "My Documents" and "Documents" folder. Only one of them is current. The other one is a left-over from a re-install of Windows, e.g. after a crash or an update to Windows 10, but choosing "keep my data". The "Documents" folder is usually the real one. "My documents" can sometimes still be accessed with admin rights. Note in any case they are virtual folders, the actual path is different and under users (or under windows.old!). So don't mix up the two. The example code above gives you access to *only* the current documents folder, the correct one.
Note,that since it is a virtual folder, the "My Documents" symlink *may* still exist, but can point to nowhere. The data is not lost, it is still there and accessible under windows.old

It may seem complex but it is easy. If you still do not understand it I will try to re-phrase it.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 12:29:41 pm

I tried your code with a freshly compiled trunk in Windows 7 64 bit Virtualbox with username UmlautUser-äöüÄ.
It gave me the correct directory: C:\Users\UmlautUser-äöüÄ\Documents\

Did you try the GetWindowsSpecialDir from windirs yet? Or does it also return an empty string in the IDE?

Code: Pascal [Select][+]

uses windirs;
 
procedure TForm1.Button1Click(Sender: TObject);
begin
  ShowMessage(GetWindowsSpecialDir(CSIDL_PERSONAL));
end;

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 12:31:46 pm

Hi Thaddy,

I can verify the location of the Documents folder in windows by clicking it, choosing properties and it gives their correct locations. and Each user documents folder is created and has their own My Pic, My Movies etc folders.

With the routine on mine, it displaying path not found only when application is run in the IDE, which is causing a problem.

Title: Re: Trying to get My Documents folder
Post by: Thaddy on July 19, 2017, 12:34:52 pm

I adapted my answer with more info. Can you read it and tell me if you understand it and if my answer covers the case I suspect is your issue?
Also note rvk's answer, that will return only the current one...the current symlink...for the current Windows installation...But that is the only correct one...

You need to find out the actual path to the data of the old installation and can not usually use the virtual folder anymore. And you can also not treat the old one as a special folder, you must treat it as a full path and find that out first.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 12:44:49 pm

@Thaddy, thanks for your explanation, I do understand what you are saying, but this does not cover the issue that running the app in the IDE fails, and running directly it works ok.

@rvk
ShowMessage(GetWindowsSpecialDir(CSIDL_PERSONAL));
Return no information when run in the IDE, but shows correctly when launched directly.

I have just created a new user with administrator rights, and this also is doing the same, test system win 7 64 ultimate, lazarus 32 bit, creating 32 bit application.

Title: Re: Trying to get My Documents folder
Post by: Thaddy on July 19, 2017, 12:49:47 pm

Quote from: josh on July 19, 2017, 12:44:49 pm

@Thaddy, thanks for your explanation, I do understand what you are saying, but this does not cover the issue that running the app in the IDE fails, and running directly it works ok.

That means that the IDE might not handle virtual folders very well under Windows. I suspect a low-level IO is used instead of the shell api routines. The latter api is the only one that can handle virtual folders under windows transparently. Today I am stuck on arm, but I think this can be easily confirmed by someone else. Worst case scenario is that some idiot hooked the IO API in the Lazarus IDE. In that case: that's an idiot.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 12:57:41 pm

Quote from: Thaddy on July 19, 2017, 12:49:47 pm

That means that the IDE might not handle virtual folders very well under Windows. I suspect a low-level IO is used instead of the shell api routines.

Thaddy, the call is already directly to the WinApi with SHGetSpecialFolderPathW. And it seems SHGetFolderPathW (which is called from windirs) also fails in that situation. No virtual directories are used or called.

I have freshly compiled trunk and can't reproduce this issue.
So it's not something from default trunk installation.

Maybe some component might catch or hook it.
So my suggestion would be to checkout a fresh trunk version and try it again without any extra components.

I also don't see why it would fail in the IDE and not outside it.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 12:59:38 pm

@Thaddy, maybe? What is odd is this only happens when user has utf8 characters in it, I have just created a new user with standard ascii characters only; and I have no such problem with the IDE.
So it may be just a odd combination of situations that are causing the problem.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 01:00:29 pm

Quote from: josh on July 19, 2017, 12:59:38 pm

So it may be just a odd combination of situations that are causing the problem.

PS. also disable ANY antivirusscanner you might have running !!!

That's the only cause I think sits between the WinApi and Lazarus.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 01:06:11 pm

@rvk, I have just tried -deactivating my AV completely and still suffering the same issue.
I will try a complete separate installation of trunk in separate folder, it will take some time so will be an hour or so before I can report back on results of that.

Title: Re: Trying to get My Documents folder
Post by: Thaddy on July 19, 2017, 01:11:57 pm

Quote from: josh on July 19, 2017, 01:06:11 pm

@rvk, I have just tried -deactivating my AV completely and still suffering the same issue.
I will try a complete separate installation of trunk in separate folder, it will take some time so will be an hour or so before I can report back on results of that.

Well I can't test on windows today before 20:00 ECT, but what about declaring all strings in the routine as unicodestring? FPC will convert a UTF8 to UTF16 automatically..
Usually (although not rare, rvk may be right) Virus scanners are not *that* dumb anymore...

Title: Re: Trying to get My Documents folder
Post by: JuhaManninen on July 19, 2017, 01:25:57 pm

Quote from: josh on July 19, 2017, 12:59:38 pm

What is odd is this only happens when user has utf8 characters in it, ...

Actually on Windows they are UTF-16. Anyway, the user name has Unicode characters outside the 7-bit ASCII range.
Still a mystery...

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 01:35:56 pm

Quote from: josh on July 19, 2017, 01:06:11 pm

I have just tried -deactivating my AV completely and still suffering the same issue.

Maybe deactivating isn't enough in this case. It might still be sitting in between.
(although it's a longshot but I can't think of anything else at the moment)
(and it would also be weird it worked in an earlier version of the IDE)

Which virusscanner is it?

Title: Re: Trying to get My Documents folder
Post by: Thaddy on July 19, 2017, 01:43:35 pm

Quote from: JuhaManninen on July 19, 2017, 01:25:57 pm

Anyway, the user name has Unicode characters outside the 7-bit ASCII range.
Still a mystery...

That's not a mystery, but fully allowed.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 01:46:33 pm

Quote from: Thaddy on July 19, 2017, 01:43:35 pm

Quote from: JuhaManninen on July 19, 2017, 01:25:57 pm
Anyway, the user name has Unicode characters outside the 7-bit ASCII range.
Still a mystery...
That's not a mystery, but fully allowed.

It's a mystery why the API call fails (in case of OP's computer). And only on extended characters at that.

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 02:05:46 pm

Update:
Just completed install of fresh lazarus trunk, and this exhibits same behavior.
So I have done some further testing, if I set target as i386, then I get same odd behavior ie Path not found when run in IDE.
If I change target to x86-64, re-build application and now the IDE is displaying the correct folder.

video showing behavior in i386 and x86-64, at end of video I call up laza help so you can see the version info.

https://youtu.be/f5jEfNeVyuM

maybe its narrowing down the issue, I hope so.

Title: Re: Trying to get My Documents folder
Post by: ahiggins on July 19, 2017, 02:12:27 pm

Might not be related but I seem to remember having a similar issue running an external application (file not found error), turned out to be that my 32bit application do not like accessing the "Program Files" directory on a 64bit system, recompiled as 64bit, worked fine. There is post on here somewhere that explains it fully. Sorry if I'm completely of the mark.

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 02:13:25 pm

So 64 bit application works fine inside the IDE?
(but actually the 64 bit app doesn't run inside the IDE.)

It could also be a GDB.exe issue.
Did you try the 32bit version with Shift+Ctrl+F9 (or Run > Run without debugger) ???

Could you check all places where you have shell32.dll on your computer?

cmd.exe
cd \
dir shell32.dll /s

Code: [Select]

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\UmlautUser-äöüÄ>cd \

C:\>dir shell32.dll /s
 Volume in drive C has no label.
 Volume Serial Number is 9C53-3EF9

 Directory of C:\Windows\System32

11/21/2010  05:23 AM        14,174,208 shell32.dll
               1 File(s)     14,174,208 bytes

 Directory of C:\Windows\SysWOW64

11/21/2010  05:24 AM        12,872,192 shell32.dll
               1 File(s)     12,872,192 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.17514_none_ca4f304d289b7800

11/21/2010  05:23 AM        14,174,208 shell32.dll
               1 File(s)     14,174,208 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.17514_none_d4a3da9f5cfc39fb

11/21/2010  05:24 AM        12,872,192 shell32.dll
               1 File(s)     12,872,192 bytes

     Total Files Listed:
               4 File(s)     54,092,800 bytes
               0 Dir(s)   6,162,755,584 bytes free

C:\>

Title: Re: Trying to get My Documents folder
Post by: Josh on July 19, 2017, 02:43:01 pm

location of shell32.dll

Code: [Select]

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\joáh>cd\

C:\>dir shell32.dll /s
 Volume in drive C has no label.
 Volume Serial Number is 40EB-6EBC

 Directory of C:\Windows\System32

20/06/2017  13:57        14,183,936 shell32.dll
               1 File(s)     14,183,936 bytes

 Directory of C:\Windows\SysWOW64

20/06/2017  13:57        12,880,896 shell32.dll
               1 File(s)     12,880,896 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.17514_none_ca4f304d289b7800

21/11/2010  04:23        14,174,208 shell32.dll
               1 File(s)     14,174,208 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.18222_none_ca42435328a5836f

26/07/2013  03:24        14,172,672 shell32.dll
               1 File(s)     14,172,672 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.18952_none_ca21e0d928bdc353

02/11/2015  02:00        14,176,768 shell32.dll
               1 File(s)     14,176,768 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.22403_none_cae2822641b201d5

26/07/2013  03:24        14,176,256 shell32.dll
               1 File(s)     14,176,256 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.23155_none_caae56a441d8e264

02/11/2015  02:00        14,182,912 shell32.dll
               1 File(s)     14,182,912 bytes

 Directory of C:\Windows\winsxs\amd64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.23806_none_cae573d441af5c17

20/06/2017  13:57        14,183,936 shell32.dll
               1 File(s)     14,183,936 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.17514_none_d4a3da9f5cfc39fb

21/11/2010  04:24        12,872,192 shell32.dll
               1 File(s)     12,872,192 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.18222_none_d496eda55d06456a

26/07/2013  02:55        12,872,704 shell32.dll
               1 File(s)     12,872,704 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.18952_none_d4768b2b5d1e854e

02/11/2015  02:00        12,875,776 shell32.dll
               1 File(s)     12,875,776 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.22403_none_d5372c787612c3d0

26/07/2013  02:56        12,874,752 shell32.dll
               1 File(s)     12,874,752 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.23155_none_d50300f67639a45f

02/11/2015  02:00        12,878,848 shell32.dll
               1 File(s)     12,878,848 bytes

 Directory of C:\Windows\winsxs\wow64_microsoft-windows-shell32_31bf3856ad364e35
_6.1.7601.23806_none_d53a1e2676101e12

20/06/2017  13:57        12,880,896 shell32.dll
               1 File(s)     12,880,896 bytes

     Total Files Listed:
              14 File(s)    189,386,752 bytes
               0 Dir(s)  28,598,775,808 bytes free

C:\>

If I run without debugger it works correctly, if I create debug and release mode then neither of these work correctly,
So it looks like maybe gdb is messing with something.

Code: [Select]

GNU gdb (GDB) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb)

Title: Re: Trying to get My Documents folder
Post by: rvk on July 19, 2017, 02:56:39 pm

Strange because I also use GDB 7.2 with my standard install of trunk.
I do download GDB 7.7.1 in my script but don't actively use it.

Here is 7.7.1
http://svn.freepascal.org/svn/lazarus/binaries/i386-win32/gdb/bin/
and here is 7.2.1
https://sourceforge.net/projects/lazarus/files/Lazarus%20Windows%2032%20bits/Alternative%20GDB/GDB%207.2.1/
(but I think the 7.2.1 is the same one I use with trunk because it's the one from the binutils)

You could try one of these.