Lazarus

Programming => Operating Systems => Mac OS X => Topic started by: furious programming on April 17, 2019, 02:02:05 am

Title: [SOLVED] Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 02:02:05 am
I need to get the time with nanosecond precision from the system to be able to measure the duration of the set of operations being performed (multiplatform project). I found this site in documentation — Using Kernel Time Abstractions (https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/services/services.html) — in which the set of appropriate functions is given.

The clock_get_system_nanotime function will be suitable for my needs. Unfortunately, I have not found in the web a single example of its use in Free Pascal. I do not know if it is imported in any unit for the FPC or not. I do not have a computer with macOS at hand and I do not know this system, so I can not check it myself.

Can any of the experts of this system show me an example of using the function clock_get_system_nanotime?

The only thing I would like to know is how to get the current time and convert it to the total number of nenoseconds in the form of Int64. And also what units must be included to access this function (or an example of an import declaration) and the required data types.

Thank you for all the help.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: dbannon on April 17, 2019, 02:09:26 am
I think you will struggle to find a cross platform nanosecond measurement model.

Most people use GetTickCount64() to measure at the millisecond level. Beyond that the overhead of your measurement comes into play.

Davo
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 02:16:43 am
I think you will struggle to find a cross platform nanosecond measurement model.

Yes, that's why I have to use different functions on different platforms. Windows and Linux is done, macOS remained.

Quote
Most people use GetTickCount64() to measure at the millisecond level.

Millisecond precision is not enough. The more so that the precision of the system clock is too low (and depends on the platform/OS) for the millisecond result to be worth anything. I need to measure the duration of the operation being performed, which may be less than one millisecond.

Generally, I need the equivalent of the QueryPerformanceCounter function from Windows or clock_gettime (for example with CLOCK_MONOTONIC) from Linux. All solutions working with milliseconds precision are impossible to use.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: 440bx on April 17, 2019, 02:38:11 am
Upfront: I don't have an answer for your question.

I have another question instead:

Millisecond precision is not enough.
If your program is running on a multitasking O/S, it is subject to whatever the scheduler decides to do.  In such cases, even when the measurement is very precise, it is hard to say that it is representative of anything.  Of course, you can take averages, drop large variations but, no matter what you do, the measurements, whatever they may be, will vary a great deal making them of questionable use.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: Thausand on April 17, 2019, 03:42:23 am
Yes, that's why I have to use different functions on different platforms. Windows and Linux is done, macOS remained.
Look epiktimer http://wiki.lazarus.freepascal.org/EpikTimer

I no think Epiktimer have support MacOS. But MacOS have precision time read https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/services/services.html

I not have MacOS and maybe some can try and have macOS ?
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: dbannon on April 17, 2019, 05:56:17 am
Well spotted Thausand !

Its in the OnlinePackageManager, so easy install. I'll try it out on my battered old Mac and see what I get.

Davo
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: dbannon on April 17, 2019, 06:49:34 am

OK, works fine on the Mac, see -

Code: Pascal  [Select]
  1. procedure TForm1.SpeedButton1Click(Sender: TObject);
  2. var
  3.   ET: TEpikTimer;   // Need to add etpackage to project requirements
  4. begin
  5.   ET := TEpikTimer.Create(Application);
  6.   ET.Clear();
  7.   ET.Start;
  8.   Sleep(1);
  9.   ET.Stop;
  10.   showmessage('1ms = ' + FloatToStr(ET.Elapsed));
  11. end;    

Tells me that 1mS=0.001166 (seconds)

That is, of course, 1,166 microseconds. Subsequent runs - 0.001155; 0.001085; 0.001147; 0.001221; 0.001133

Replace the sleep with a single debugln() line, we get - 0.000042; 0.000036; 0.000034; 0.000046; 0.000022

The noise at microsecond levels would seem to make nanosecond measurements beyond reasonable expectations. But as a tool for measuring microseconds, it looks pretty good ! 

The number returned is a float, so you can, if you wish, extract any resolution you want from it, but don't confuse that with precision !

This, on a 2012 Macbook pro, Lazarus 2.0.2

Davo
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: engkin on April 17, 2019, 09:09:52 am
[DELETED]
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: User137 on April 17, 2019, 12:51:40 pm
Is the task something you can repeat? Perform the same thing 10000 times and then the millisecond time will be meaningful enough. Single iteration then actually takes total time divided by 10000.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 06:09:48 pm
If your program is running on a multitasking O/S, it is subject to whatever the scheduler decides to do. […]

This is obvious, I know that and no, it does not bother me at all. I do not use these results for statistics.


Look epiktimer http://wiki.lazarus.freepascal.org/EpikTimer

I've seen the code of this timer many of times. For macOS uses GetTickCount64, and I do not want to use this function because its result is not a predetermined time unit and depends on platform/OS/hardware/whatever.

Quote
But MacOS have precision time read https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/services/services.html

Just to this documentation site I've provided a link in the first post of this thread...


Have you read what I asked for at all? Come on guys... I asked a specific question. I do not need advice on what to choose, I do not need an assembler (probably), and I certainly do not need to attach to the project a thousand lines of someone else's code just to use its 10 lines.

I know exactly what function I need to use, I know that it is available and that it meets all assumptions. So I will ask again — is there anyone here who can give a specific, short and simple example of using the function clock_get_system_nanotime?
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: 440bx on April 17, 2019, 06:26:37 pm
If your program is running on a multitasking O/S, it is subject to whatever the scheduler decides to do. […]
This is obvious, I know that and no, it does not bother me at all.

I agree that is, or at least should be, obvious. 

The problem is, it won't accomplish the goal stated in your original post:
I need to get the time with nanosecond precision from the system to be able to measure the duration of the set of operations being performed (multiplatform project).
It is obvious you will not get nanosecond precision - because the O/S scheduler is (obviously) going to get in the way.

Just for the record, in spite of the above, if I had an example of what you requested, I'd provide it to you.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: Thaddy on April 17, 2019, 06:51:50 pm
exact nano or pico resolution is usually done with dedicated hardware. RTOS systems. FPC can do that with embedded.
For multi-tasking systems: don't get your hopes up.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 07:05:14 pm
It is obvious you will not get nanosecond precision - because the O/S scheduler is (obviously) going to get in the way.

Precision will be nanosecond, because the result will be the number of nanoseconds (which, after all, provides the mentioned function). At most, it will deviate from reality to some extent — I know that.

In my case, if the deviation from the actual measurement will not be greater than, let's assume, 0.1ms, it will not cause noticeably malfunction of the program. Noticeably, because it is about the perception of the human eye, not about the possibility of errors in the program. And I suspect that in typical conditions such large distortions do not occur (more than assumed 100000ns). But if so, the worst thing that can happen is a normal lag, that will not destabilize the program in any way.

I care more specifically about the function clock_get_system_nanotime, because it returns the result in nanoseconds (in a known unit of time), which then I can use in the function FpNanoSleep, without any complex calculations, rounding and similar. Just one multiplication and done. The final code will be short and easy to understand for everyone (including me), and this is importontant (at least for me).

Quote
Just for the record, in spite of the above, if I had an example of what you requested, I'd provide it to you.

I appreciate. In the meantime, I will try to write it myself, but I still will not be able to check it out in practice, so I will ask for a test.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 08:29:04 pm
This should be enough:

Code: Pascal  [Select]
  1. unit CustomTimeUtils;
  2.  
  3. {$MODE OBJFPC}{$LONGSTRINGS ON}
  4.  
  5. interface
  6.  
  7.   procedure clock_get_system_nanotime(secs, nanosecs: puint32); cdecl; external 'libname'{?};
  8.  
  9.   function TimeInNanoseconds(): Int64;
  10.  
  11. implementation
  12.  
  13. function TimeInNanoseconds(): Int64;
  14. var
  15.   Seconds: UInt32 = 0;
  16.   Nanoseconds: UInt32 = 0;
  17. begin
  18.   clock_get_system_nanotime(@Seconds, @Nanoseconds);
  19.   Result := Int64(Seconds) * 1000000000 + Int64(Nanoseconds);
  20. end;
  21.  
  22. end.

But I do not know the library name (highlighted line) — someone knows it? And simple program to test:

Code: Pascal  [Select]
  1. program TimeTest;
  2.  
  3. {$MODE OBJFPC}{$LONGSTRINGS ON}
  4.  
  5. uses
  6.   SysUtils, CustomTimeUtils;
  7. var
  8.   I: Integer;
  9. begin
  10.   WriteLn(TimeInNanoseconds());
  11.  
  12.   for I in [0 .. 9] do
  13.   begin
  14.     Sleep(1000);
  15.     WriteLn(TimeInNanoseconds());
  16.   end;
  17.  
  18.   ReadLn();
  19. end.

The results given in individual loop iterations should more or less differ by 1 billion.

A test project in the attachment. Can anyone set the proper name of the library, check if it compile and works, and show the console output? I will be grateful.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: sstvmaster on April 17, 2019, 09:55:58 pm
clock_get_system_nanotime is a Kernel Function.

- https://developer.apple.com/documentation/kernel/1416671-clock_get_system_nanotime?language=objc
- http://newosxbook.com/src.jl?tree=xnu-1228.15.4&file=/osfmk/ppc/rtclock.c#absolutetime_to_nanotime

But i dont if this is right, i only search. :D
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 17, 2019, 11:17:54 pm
Yep, it is. Is this means, that it can not be imported and used in sofware layer, e.g. in a regular application?
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: sstvmaster on April 18, 2019, 12:25:04 am
https://stackoverflow.com/questions/51555194/what-does-nanotime-do-on-macos

or libsystem_kernel.dylib
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 18, 2019, 12:57:43 am
@sstvmaster: thank you, I have improved the source of the test project. Now it should compile and run, so I added the complete sources to the attachment. If someone has time, I would ask for a check.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: lainz on April 18, 2019, 01:10:42 am
What I get
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 18, 2019, 01:22:40 am
Is it possible to use this function at all? Even indirectly?
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: Thausand on April 18, 2019, 04:41:39 am
Is it possible to use this function at all? Even indirectly?
Example stackoverflow go language wrapper. Is no system function.

Can see is wrapper darwin http://newosxbook.com/src.jl?tree=xnu-1228.15.4&file=/osfmk/ppc/rtclock.c#absolutetime_to_nanotime
Code: C  [Select]
  1. void
  2. clock_get_system_nanotime(
  3.         uint32_t                        *secs,
  4.         uint32_t                        *nanosecs)
  5. {
  6.         uint64_t        now, t64;
  7.         uint32_t        divisor;
  8.  
  9.         now = mach_absolute_time();
  10.  
  11.         *secs = t64 = now / (divisor = rtclock_sec_divisor);
  12.         now -= (t64 * divisor);
  13.         *nanosecs = (now * NSEC_PER_SEC) / divisor;
  14. }
  15.  

Important read "mach_absolute_time()"

Stackoverflow writes go use import https://github.com/golang/go/blob/fe68ab3bcde97e3a325e4aa3c70f5f9172540453/src/runtime/sys_darwin.go#L351
Code: Go  [Select]
  1. 351 go:cgo_import_dynamic libc_mach_absolute_time mach_absolute_time "/usr/lib/libSystem.B.dylib"
  2.  

I not know /usr/lib/libSystem.B.dylib ... or is simple libc

If is dylib then no static and is dynamic load.

I not sure help you furious programmer... i sorry if not help.


Have more read topic time measure:
http://nadeausoftware.com/articles/2012/04/c_c_tip_how_measure_elapsed_real_time_benchmarking
https://developer.apple.com/library/archive/qa/qa1398/_index.html
https://opensource.apple.com/source/xnu/xnu-3789.41.3/libsyscall/wrappers/mach_absolute_time.s.auto.html
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 18, 2019, 02:37:34 pm
Yesterday I was tired of thinking about the solution and writing stupid. Today I have a fresh mind and I can look at the problem again. it should be better. 8)

I not sure help you furious programmer... i sorry if not help.

You probably solved the riddle.

The first thing is mach_absolute_time, which is the right time retrieval function. It can be used to get the current time and then, calculate the nanoseconds count.

But the second thing is this article:

Quote
http://nadeausoftware.com/articles/2012/04/c_c_tip_how_measure_elapsed_real_time_benchmarking

It turns out that there is a better solution than mach_absolute_time, that is, the gettimeofday function. Although the result has microsecond precision, but it is available on many POSIX systems, including Linux, BSD, OSX, Solaris and so on. And it is declared in Unix unit with the name fpgettimeofday.

Thanks to this, I can simplify the project code and use QPC and QPF for Windows, and fpgettimeofday on other systems (with predefined frequency value). Get the time, calculate nanoseconds count and use the result for fpnanosleep.


@Thausand: huge thanks! It should do the job.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: Basile-B on April 18, 2019, 03:13:44 pm
If the problem is still not solved You can try to see how this is done in D stand library since its StopWatch is nano-second accurate on all platforms.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 18, 2019, 07:17:10 pm
I hope that the problem is solved due to the "discovery" of the FPGetTimeOfDay function. :D

But I need confirmation, that's why I prepared a new test application that checks the correctness of the simple function that takes time and returns it in the form of a number of nanoseconds. This simple function looks like this:

Code: Pascal  [Select]
  1. function TimeInNanoseconds(): Int64;
  2. var
  3.   Time: TTimeVal;
  4. begin
  5.   FPGetTimeOfDay(@Time, nil);
  6.   Result := Int64(Time.tv_sec) * 1000000000 + Int64(Time.tv_usec) * 1000;
  7. end;

Should work on different non-Windows systems, including macOS, Linux, BSD and Solaris. Test application is in the attachment, so please check if it compiles and works properly. The output of the console will be appreciated.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 21, 2019, 08:59:53 pm
No one has the opportunity to check this test program on desktop macOS (or on any other Unix-like system) and write whether it works and what is the output of the console?

Source code is in the attachment in the previous post. It will help me a lot.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: lainz on April 22, 2019, 03:23:57 am
Attached console. MacOS.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: Thausand on April 22, 2019, 04:15:23 am
attach raspbian
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 22, 2019, 01:38:51 pm
@Lainz and @Thausand: thank you for tests.

It looks like the calculations on both macOS and Linux are correct, so the problem is solved.
Title: Re: Getting the current time in form of nanoseconds count as Int64
Post by: ASBzone on April 22, 2019, 09:55:13 pm
I hope that the problem is solved due to the "discovery" of the FPGetTimeOfDay function.

I would love to see how you call the QPC on Windows in your routine.
Title: Re: [SOLVED] Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 22, 2019, 10:00:37 pm
Sure, here you have:

Code: Pascal  [Select]
  1. function TClock.GetHardwareCounterValue(): Int64;
  2. {$IFDEF UNIX}
  3. var
  4.   Counter: TTimeVal;
  5. {$ENDIF}
  6. begin
  7.   {$IFDEF WINDOWS}
  8.   Result := 0;
  9.   QueryPerformanceCounter(Result);
  10.   {$ELSE}
  11.   FPGetTimeOfDay(@Counter, nil);
  12.   Result := Int64(Counter.tv_sec) * 1000000000 + Int64(Counter.tv_usec) * 1000;
  13.   {$ENDIF}
  14. end;
  15.  
  16. procedure TClock.WaitForNMI();
  17. var
  18.   {$IFDEF WINDOWS}
  19.   NextFrameCounts, CurrentCounts: Int64;
  20.   {$ELSE}
  21.   WaitTime: Int64;
  22.   RequestedTime, RemainingTime: TTimeSpec;
  23.   {$ENDIF}
  24. begin
  25.   {$IFDEF WINDOWS}
  26.   NextFrameCounts := FFrameCountsBegin + FCountsPerFrame;
  27.  
  28.   repeat
  29.     CurrentCounts := GetHardwareCounterValue();
  30.   until CurrentCounts >= NextFrameCounts;
  31.   {$ELSE}
  32.   WaitTime := FFrameCountsBegin + FCountsPerFrame - FFrameCountsEnd;
  33.  
  34.   if WaitTime > 0 then
  35.   begin
  36.     RemainingTime.tv_sec := WaitTime div 1000000000;
  37.     RemainingTime.tv_nsec := WaitTime mod 1000000000;
  38.  
  39.     repeat
  40.       RequestedTime := RemainingTime;
  41.  
  42.       if FPNanoSleep(@RequestedTime, @RemainingTime) = 0 then Exit;
  43.       if FPGetErrNo() <> ESysEINTR then Exit;
  44.     until False;
  45.   end;
  46.   {$ENDIF}
  47. end;
Title: Re: [SOLVED] Getting the current time in form of nanoseconds count as Int64
Post by: ASBzone on April 22, 2019, 11:06:22 pm
Sure, here you have:   ...

Thanks!  This thread caused me to read some seriously interesting articles today.   Thank you for this.
Title: Re: [SOLVED] Getting the current time in form of nanoseconds count as Int64
Post by: furious programming on April 23, 2019, 12:09:24 am
Your welcome.

But my game is not a good example about how to use QPC, because it is used to build custom mechanism which work similar to FPNanoSleep but, instead of wait, it consumes all available CPU power. On Windows there is no solution to freeze the program for microseconds or nanoseconds — system Sleep function does not allow for this.

For clarity, the piece of code given in my previous post comes from the Platformer.Time.pp unit, and the sources of the entire project are available in this post (https://forum.lazarus.freepascal.org/index.php/topic,44908.msg318476.html#msg318476). If you want, download it and see what the whole timer code looks like.
Title: Re: [SOLVED] Getting the current time in form of nanoseconds count as Int64
Post by: ASBzone on April 23, 2019, 04:48:24 pm
Your welcome.

But my game is not a good example about haw to use QPC, because it is used to build custom mechanism which work similar to FPNanoSleep but, instead of wait, it consumes all available CPU power. On Windows there is no solution to freeze the program for microseconds or nanoseconds — system Sleep function does not allow for this.

For clarity, the piece of code given in my previous post comes from the Platformer.Time.pp unit, and the sources of the entire project are available in this post (https://forum.lazarus.freepascal.org/index.php/topic,44908.msg318476.html#msg318476). If you want, download it and see what the whole timer code looks like.

Thanks again.  I read several articles and pages, and was able to adapt it to my needs so that I could replace the following, which I had been using for tracking the passing of time.

Here's the source of what I had been using for some functions:  https://stackoverflow.com/questions/17109814/why-datetimetomilliseconds-in-dateutils-pas-is-marked-as-internal (https://stackoverflow.com/questions/17109814/why-datetimetomilliseconds-in-dateutils-pas-is-marked-as-internal)

For now, it is a Windows-only solution, but I will deal with cross-platform in due time.