Synapse ahd XOAUTH2 (gmail)?

[rvk]

WARNING!!!
do NOT try to recompile Project1 (the DLL) b/c you would LOOSE Lazarus after doing it: you will not be able to compile any porgram and will have to re-install Lazarus ..... hopefully not until hell freezes over, like in my case!

oe... I will definitely try that (when I wake up). recompiling a project should never lead to corruption of you Lazarus installation so I'm very curious of what's going wrong. Reinstalling trunk is an automated process here so that won't be a problem.

[epergola]

I admire your courage RVK!
But ...caution, it happened to me 3 times
(twice I did not know what I did to cause it, the 3rd time I found it).
I belong to the ones that a trunk refers to tree (the real one, with green leaves).
the build / clean amd make -vtkyxz is chinese.
If I download the installation EXE i rely on it exclusively to re-install clean.
Why would I download the installation EXE if I did understand the trunk and other chinese stuff?.
However, compiling a DLL and loosing Lazarus is not acceptable I think, trunk or
not trunk.
I am going to sleep to.

[rvk]

Some remarks while I'm testing:

In your unitoauth2.pas you still have aGetXOAuth2Base64 define as a function while in setxauth2 you call it as a procedure. I think it returns the code64 correctly but your unitoauth2.pas is out of date. I suppose you did changed it in your original xoauth2.dll because I do get a code64 back.

At first the geTAUHT2 didn't create a token file but that was because I didn't have libeay32.dll and ssleay32.dll in my project directory (or anywhere else in my path). After copying them in the geTAUHT2 worked and produced the tokens.dat and a correct code64.

I do have the feeling you might end up in doing the GetAccess 2 times. One time in the CheckTokenFile if there is a tokens.dat and then the second time in GettheAccess in geTAUHT2.

You're missing something vital in your TForm1.Button1Click. You receive a code64 (access_token) from your geTAUHT2 but you don't use it anywhere. If you look in my frmmain.pas you'll see I added a helper for TSMTPSend to be able to send the 'AUTH XOAUTH2 code64' right after the smtp.StartTLS. If you don't do that Google won't accept the connection.

Then... because I couldn't trace a DLL I made a small geTAUHT3 procedure to not use the DLL but use the functions in unitoauth2 directly. Just to test if something goes wrong there too. But it didn't.

Code: [Select]

function geTAUHT3(var EMAIL, FULLNAME, code64: string; tokenfile_name: string): boolean;
type
  TCreateit = procedure(client_id, client_secret: string); stdcall;
  TGettheAccess = procedure(var email, fullname: string; UseTokenFile: boolean = False);
    stdcall;
  TDestroyit = procedure; stdcall;
  TCheckTokenFile = function: boolean; stdcall;
  TGetXOAuth2Base64=procedure (var val: string);stdcall;
  //TGetXOAuth2Base64 = function: string; stdcall;
  Tgettoken_filename = procedure(Value: string); stdcall;
var
  myObj: TObject;
  _EC: function: Pointer;
  Handle: cardinal;
  _Createit: TCreateit;
  _GettheAccess: TGettheAccess;
  _Destroyit: TDestroyit;
  _CheckTokenFile: TCheckTokenFile;
  _GetXOAuth2Base64: TGetXOAuth2Base64;
  _gettoken_filename: Tgettoken_filename;
  msg: string;
const
  client_id = '896304839415-nnl5e0smrtakhr9r2l3bno0tes2mrtgk.apps.googleusercontent.com';
  client_secret = 'dUahHDn3IMyhCIk3qD4tf8E_';
begin
  Result := False;
  email := '';
  _Createit := @unitoauth2.Createit;
  if Assigned(_Createit) then
  begin
    _Createit(client_id, client_secret);
    _gettoken_filename := @unitoauth2.gettoken_filename;
    if Assigned(_gettoken_filename) then
      _gettoken_filename(tokenfile_name);
    _CheckTokenFile := @unitoauth2.CheckTokenFile;
    if Assigned(_CheckTokenFile) then
    begin
      if boolean(_CheckTokenFile) <> True then
        msg := 'Could not get Authentication';
    end
    else
      msg := 'CheckTokenFile not found';
    _GettheAccess := @unitoauth2.GettheAccess;
    if Assigned(_GettheAccess) then
      _GettheAccess(email, fullname, True);
    _GetXOAuth2Base64 := @unitoauth2.aGetXOAuth2Base64;
    if Assigned(_GetXOAuth2Base64) then
      _GetXOAuth2Base64(code64);

    _Destroyit := @unitoauth2.Destroyit;
    if Assigned (_Destroyit) then
      _Destroyit;

  end;
  Result := email <> '';
end;
Everything went great (getting access and getting the code64) and the testmail is send.

When I change it back to geTAUHT2 everything went great until StartTLS. So the correct code64 is received so the xoauth2.dll should work correctly. I get this error:

Code: [Select]

SMTP ERROR: StartTLS:Success-Other undefined Status
After that the program crashes with a SIGSEGV. So the cleanup of the dll is not done correctly.

But I'm puzzled as to why that would mess up the TSMTPSend because that is not even done in the dll.


Edit: I think it has something to do with the fact that the DLL loads the openssl dll's and after it unloads, it also unloads the openssl dll's while your program still needs those to send the mail. Maybe because your DLL is still part of the same process the DLL's can get unloaded (which is wrong).

For instance if you uncheck the FreeLibrary of your own dll it all works:

Code: [Select]

    end;
    // FreeLibrary(Handle);
  end;

So as a workaround, you could free the DLL at the end of your program and not right after geTAUHT2. Or you need to find a way to "reload" the openssl libraries again for your own .exe. Or you need to find a way for your DLL not to unload the openssl libraries.

B.T.W. I had no problems compiling the DLL. In Laz1.4.2 or trunk (=Lazarus SVN development version)
So no need for hell to freez over for me

[rvk]

Okay... (also please read my previous post above)

It's the initialization of openssl that gets lost after you unload your DLL.

If you reinitialize openssl after unloading your dll it works fine:
(at the end of geTAUHT2)

Code: [Select]

    end;
    FreeLibrary(Handle);
  end;

  // reinitialize openssl
  ssl_openssl_lib.SslLibraryInit;

  Result := email <> '';
end;
(I'm still not sure why exactly this is necessary)

The only thing I still get is the SIGSEGV but that's something to do with the cleanup of your DLL but the mail get send successfully.

[epergola]

Hello Rvk
That is a bit strange. My dll (basically unitoauth2.pas) does not use opensssl, hence I do not see how the loading/unloading of "xoauth2.dll"  would affect the lib*.dll.
I will investigate further.
Uhm... either you have the luck of the devil, or I am cursed then, since not once but 3 times, simply compiling Project 1 sent Lazarus to the bananas field.
I will keep you posted (not on this last topic though, as I will not touch it!)

[rvk]

My dll (basically unitoauth2.pas) does not use opensssl, hence I do not see how the loading/unloading of "xoauth2.dll"  would affect the lib*.dll.

O but it does
unitoauth2 does some communicating via HTTPS so openssl is needed to set up a connection to https://accounts.google.com/. So it does use the openssl dll's. You just don't explicitly call them but in HttpPostURL and HttpGetText (from Synapse) they do get used. In ssl_openssl_lib there is a finalization part which calls DestroySSLInterface. It tries to unload the library-dll's but because of the reference counting of the DLL, it stays in memory. My guess is that it somehow gets deactivated (the methods get unloaded but the DLL itself stays in memory). That's why calling ssl_openssl_lib.SslLibraryInit again reinitializes the openssl-dll's and you can call them again (see my last post).

[epergola]

Ok, I will use the reinit stuff.
But I have a doubt.
Per se if a module is loaded in a process, unloading it should not affect a module loaded in another  (Hmodule is an address relative to the process it opens it).
I suspect i did somethin stupid in my DLL
(Project1).
In the test program, setxoauth2.pas) I had this (jyst before FreeLibrary):
{
    @Destroyit := GetProcAddress(Handle, 'Destroyit') ;
    if Assigned (Destroyit) then
      Destroyit;
}

and in the Project1.dll
procedure Destroyit;stdcall;
begin
   GoogleOAuth2.destroy;
end;
         
I commented it out b/c it gave me an error (AV I think), and now I realize that I had
 var
    GoogleOAuth2: TGoogleOAuth2; 
in the implementation part (and thinks this was the cause of the error in Destroyit).

So I ask you if you would be so kind to move that line to  the interface (unitoauth2.pas), recompile  and send the output to me?
(Unless you think all of this is meaningless)

[rvk]

But I have a doubt.
Per se if a module is loaded in a process, unloading it should not affect a module loaded in another  (Hmodule is an address relative to the process it opens it).
I suspect i did somethin stupid in my DLL
...
(Unless you think all of this is meaningless)

I think, because both your .exe and .dll are part of the same process the openssl dlls share something.

Your .exe initializes the openssl.dll (i.e. the methods get loaded) and wait for it to be called by your .exe.

After that, in the same process but in your .dll, this is done again by your .dll. It is needed for the https.

Then your dll gets unloaded and something in the finalization of the openssl.dll gets called deactivating something, is my guess.

After that, when you call the openssl-functions, the methods are not loaded and you get the error.

I thought it was strange too that in finalization something from your dll could mess up the loaded part of the .exe but it's the only thing I could come up with and is supported by the fact that reinitialization works.

Then the other problem. When exiting the program I get that SIGSEGV. That is due to the DLL and needs to be taken care of.

I'll try to make some adjustment and see if that helps. (but I don't think it is because GoogleOAuth2 is in the interface part).

[rvk]

Could you post your last project1 (the DLL source) again. You have made some changes (for instance the function to procedure call of GetXOAuth2Base64) and while I can compile the project1.dll and it gets loaded by the demo, I can't seem to be able to call GettheAccess. (It justs exits the complete program).

My guess is that something similar also happens to you because just compiling (and only compiler) should never be able to screw up the Lazarus installation.

(or maybe your adjusted source does it for me too )

[epergola]

Here it goes

[rvk]

Your project1 source is still not valid.

It complains that synapse is not found (not a requirement in your options) and the function aGetXOAuth2Base64 declaration is wrong in the interface part and the implementation part.

(you have it as a function without a result while in your demo it is a procedure.)

I fixed those things here... but the strange thing is the dll compiles ok in a separate folder but when I try to put it all in one folder the output dll is different and crashes. Weird...

[rvk]

Also {$DEFINE USE_BIN_STR} should be added above the unit in the DLL. You're working with strings as parameters. According to this it should be added.

When you add the ssl_openssl_lib.SslLibraryInit does sending the mail work for you?

(i.e. do you also only get a SIGSEGV at the end of your procedure?)

[epergola]

No, neither ssl_openssl_lib.SslLibraryInit nor commenting out Freelibrary make any difference.
(Sorry for the mismatched units I sent).
My D5 program gets an error on Start Tls
(returns false).
Definitely something is wrong in the dll.
If i comment out
if not geTAUHT2(EMAIL,FULLNAME,code64,'tokens.dat')
the email does not work but get no other error.
If not, not only the email does not work, but the line
msg_lines.free;
gives me an Invalid pointer operation, which seems to indicate a screwup in the memory caused by the function in the dll.
Because I cannot touch Project1, could you please send me the latest dll ?
P.S. there is no SIGSEGV  in D5. What is it?

[rvk]

P.S. there is no SIGSEGV  in D5. What is it?

No, I get the SIGSEGV (access violation) in the Lazarus demo program.

That's the one I'm testing at the moment and when adding the ssl_openssl_lib.SslLibraryInit line (or commenting out Freelibrary everything runs fine until the Button1Click ends and you get the access violation.

With the changes of that line (or omitting FreeLibrary) the StartTls-command works correctly for me.

The problem of the SIGSEGV is in the procedure GetXOAuth2Base64. When I only comment out that line the procedure runs fine (of course without sending a mail because there is no valid code64 returned).

That's the procedure you had a function for and turned into a procedure. So there something goed wrong.

You get the error on freeing msg_lines.free because it is the last line of the procedure. Try switching them with the SMTP.free and you see the error turning up there. Just comment out GetXOAuth2Base64(code64); and you see it runs without error.


(B.T.W. with all this it would have been easier to adjust unitoauth2.pas with a json-reader for Delphi, SuperObject for instance. Then there would no need for a DLL. But I would love to know why this is not working first

[epergola]

You are right in your last point, but i tried Superobject but found several things that I could not overcome with D5.
Does {$DEFINE USE_BIN_STR} in the dll means all string are widestring?
According to the note below, if we pass widestrings between dll & Exe should be the solution. So if you could send me the
xoauth2.dll corrected when you finish I will try it.

I found this on the web
http://stackoverflow.com/questions/2638652/exchanging-strings-pchar-between-a-freepascal-compiled-dll-and-a-delphi-compil

Depending on what data you're passing, you may be able to use WideStrings instead. They're allocated on the Windows heap, so if you allocate one in the DLL and free it in the EXE they'll both go through the same memory manager.

You should be able to use a function declaration like this:

procedure GetAString(var Result: WideString); stdcall;
And Delphi and FreePascal will both handle the allocation and freeing automatically. WideString is the same in the Unicode-enabled Delphis as the pre-Unicode ones, so you won't need to change things for that either.