Recent

Author Topic: Different date and time for files, Windows versus Lazarus  (Read 1118 times)

ejohnstone

  • New member
  • *
  • Posts: 5
Different date and time for files, Windows versus Lazarus
« on: July 17, 2020, 12:11:52 am »
   I've been working a little with the date and time in Windows files, using the Time field of type TSearchRec (obtained with calls to FindFirst and FindNext, of course). I've noticed a few oddities that I don't understand. I'm using the library routines DateTimeToStr(FileDateToDateTime(Time)) from the SysUtils library.

  1. Outside of Lazarus, when you bring up the properties of a file in Windows, the 'Modified' date and time are not the same as the Lazarus time of TSearchRec.Time. Furthermore, these timestamps are not always different by the same amount. When working with timestamps from a couple of days ago the difference is one second (Lazarus is always one second later). But when looking at files that were saved 12/19/2019, the difference is 1:00:01 or 1:00:02 (for example Windows says Thursday, ‎December ‎19, ‎2019, ‏‎10:25:59 PM but Lazarus says 12/19/2019 23:26:00). It appears to me that Lazarus is decoding the Time value correctly. Maybe the difference of an hour has something to do with Daylight Saving Time, but what's going on here with the one or two second difference?

  2. Clearly a bug? Saved a file at 23:59:59. Windows properties says July 14, 2020. 11:59:59 PM. Lazarus says 7/15/2020. The time is omitted completely, which is the bug. Shouldn't the time always be included when you call DateTimeToStr(FileDateToDateTime(Time))?  (The time is actually 0:00:00.)

ASBzone

  • Sr. Member
  • ****
  • Posts: 461
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Different date and time for files, Windows versus Lazarus
« Reply #1 on: July 17, 2020, 05:52:02 am »
For your second question, please see the following:

https://www.freepascal.org/docs-html/rtl/sysutils/datetimetostr.html

Where it says "Note that if the time part is 0 (i.e. midnight), no time is appended."

You can control the formatting in a few ways,  including:  https://www.freepascal.org/docs-html/rtl/sysutils/formatdatetime.html
« Last Edit: July 17, 2020, 05:53:53 am by ASBzone »
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.508)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

GAN

  • Sr. Member
  • ****
  • Posts: 294
Lazarus 2.0.8 FPC 3.0.4 Linux Mint Mate 19.3
Zeos 7.2.6 - Sqlite 3.32.3

https://gitlab.com/users/GAN__/projects

PascalDragon

  • Hero Member
  • *****
  • Posts: 2116
  • Compiler Developer
Re: Different date and time for files, Windows versus Lazarus
« Reply #3 on: July 17, 2020, 09:26:10 am »
   I've been working a little with the date and time in Windows files, using the Time field of type TSearchRec (obtained with calls to FindFirst and FindNext, of course). I've noticed a few oddities that I don't understand. I'm using the library routines DateTimeToStr(FileDateToDateTime(Time)) from the SysUtils library.

  1. Outside of Lazarus, when you bring up the properties of a file in Windows, the 'Modified' date and time are not the same as the Lazarus time of TSearchRec.Time. Furthermore, these timestamps are not always different by the same amount. When working with timestamps from a couple of days ago the difference is one second (Lazarus is always one second later). But when looking at files that were saved 12/19/2019, the difference is 1:00:01 or 1:00:02 (for example Windows says Thursday, ‎December ‎19, ‎2019, ‏‎10:25:59 PM but Lazarus says 12/19/2019 23:26:00). It appears to me that Lazarus is decoding the Time value correctly. Maybe the difference of an hour has something to do with Daylight Saving Time, but what's going on here with the one or two second difference?

  2. Clearly a bug? Saved a file at 23:59:59. Windows properties says July 14, 2020. 11:59:59 PM. Lazarus says 7/15/2020. The time is omitted completely, which is the bug. Shouldn't the time always be included when you call DateTimeToStr(FileDateToDateTime(Time))?  (The time is actually 0:00:00.)

Regarding the difference in seconds: the TSearchRec.Time field is for historic reasons a 32-bit value, it's accuracy is 2 seconds, thus you'll either get :00 or :02, but not :01. FPC 3.2.0 introduced TSearchRec.Timestamp which is a TDateTime and in trunk it will use a more accurate timestamp if the platform supports it (e.g. on DOS it will still be restricted to 2 second accuracy, but on Windows and *nix it will be more accurate) - we forgot to merge the use of the more accurate timestamp into 3.2. :-[

Regarding the difference in an hour: the TSearchRec.Time is not local time, but system time. You'll have to adjust that for your timezone. This is also something that TSearchRec.Timestamp handles in trunk.

ejohnstone

  • New member
  • *
  • Posts: 5
Re: Different date and time for files, Windows versus Lazarus
« Reply #4 on: July 17, 2020, 05:33:40 pm »
Thank you PascalDragon. What seems weird to me is that even when Windows shows an even number of seconds the value in SearchRec.Time is two seconds later. Whatever, not important, but curious to me nevertheless. What's also curious but not so important is whether the value is adjusted by Windows or adjusted by Lazarus as part of the calls to FindFirst and FindNext.

   I don't understand the significance of local time vs. system time. I'm just comparing two values. I'm looking at the date and time that's shown in Windows when you display the file properties and comparing that with the value of TSearchRec.Time. Where does local time come into play? What do you mean about adjusting the time for my timezone? Pardon me for being obtuse.

   I'm not familiar with the meaning of "trunk." I've seen it in other posts but I'm ignorant of the meaning. Can you explain?

PascalDragon

  • Hero Member
  • *****
  • Posts: 2116
  • Compiler Developer
Re: Different date and time for files, Windows versus Lazarus
« Reply #5 on: July 18, 2020, 02:14:58 pm »
Thank you PascalDragon. What seems weird to me is that even when Windows shows an even number of seconds the value in SearchRec.Time is two seconds later. Whatever, not important, but curious to me nevertheless. What's also curious but not so important is whether the value is adjusted by Windows or adjusted by Lazarus as part of the calls to FindFirst and FindNext.

It all depends on rounding. The date on a NTFS has a higher precision than 1 second. So if the time is 01:01:00.123 then Windows might display it as 01:01:00 while the mechanism of FPC to calculate the 2-second precision time might be 01:01:02. Don't be bothered too much by that 2 second difference.

   I don't understand the significance of local time vs. system time. I'm just comparing two values. I'm looking at the date and time that's shown in Windows when you display the file properties and comparing that with the value of TSearchRec.Time. Where does local time come into play? What do you mean about adjusting the time for my timezone? Pardon me for being obtuse.

NTFS stores the time as UTC, but Windows displays them in your local time. Take 7:42:04 (UTC) for example. In my case (European Central Time - Summertime) it will be displayed as 9:42:04. The operating system itself internally also operates with UTC (thus "system time"). For display purposes however this time is translated into "local time", meaning the time of your timezone. The FindFirst and FindNext functions retrieve (from the operating system) the time in "system time" aka UTC. Now you need to translate that into "local time" so that you get the same time (plus/minus the 2 seconds) that the Explorer displays. In theory the new Timestamp property should do that, but that's only the case in trunk (see below for an explanation). So you'll have to adjust that yourself.

   I'm not familiar with the meaning of "trunk." I've seen it in other posts but I'm ignorant of the meaning. Can you explain?

There are the release versions (e.g. 3.0.4, 3.2.0), then there are the fixes branches (e.g. 3.0.5, 3.2.1) which might become new fix releases (e.g. 3.0.6 and 3.2.2) and the development branch (currently 3.3.1) which will become 3.4 in the future, the later of which is also called "trunk" due to the name in our Subversion repository. That's also true for many other projects that use Subversion (e.g. Lazarus). In Git that main development branch is usually called "master" (no, let's not go into that discussion right now).

ASBzone

  • Sr. Member
  • ****
  • Posts: 461
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Different date and time for files, Windows versus Lazarus
« Reply #6 on: July 20, 2020, 12:50:20 am »
Regarding the difference in seconds: the TSearchRec.Time field is for historic reasons a 32-bit value, it's accuracy is 2 seconds, thus you'll either get :00 or :02, but not :01. FPC 3.2.0 introduced TSearchRec.Timestamp which is a TDateTime and in trunk it will use a more accurate timestamp if the platform supports it (e.g. on DOS it will still be restricted to 2 second accuracy, but on Windows and *nix it will be more accurate) - we forgot to merge the use of the more accurate timestamp into 3.2. :-[

Regarding the difference in an hour: the TSearchRec.Time is not local time, but system time. You'll have to adjust that for your timezone. This is also something that TSearchRec.Timestamp handles in trunk.


Any chance it will make it to the Fixes branch?  (Or should I assume "no" because this is really new/improved functionality?)
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.508)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

ASBzone

  • Sr. Member
  • ****
  • Posts: 461
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Different date and time for files, Windows versus Lazarus
« Reply #7 on: July 20, 2020, 12:51:02 am »
Windows issue

https://en.lmgtfy.com/?q=wrong+date+time+on+windows+10&pp=1&s=d&iie=1

Based on the description of the issue, why do you perceive this to be a Windows problem?
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.508)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

ASBzone

  • Sr. Member
  • ****
  • Posts: 461
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Different date and time for files, Windows versus Lazarus
« Reply #8 on: July 20, 2020, 12:53:12 am »
In Git that main development branch is usually called "master" (no, let's not go into that discussion right now).


 :D
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.508)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

ejohnstone

  • New member
  • *
  • Posts: 5
Re: Different date and time for files, Windows versus Lazarus
« Reply #9 on: July 20, 2020, 01:35:44 am »
I'd like to know how to include previous posts in those nifty boxes with the posting details at the top but I'm such a darn newbie.

On July 18 PascalDragon wrote:
NTFS stores the time as UTC, but Windows displays them in your local time. Take 7:42:04 (UTC) for example. In my case (European Central Time - Summertime) it will be displayed as 9:42:04. The operating system itself internally also operates with UTC (thus "system time"). For display purposes however this time is translated into "local time", meaning the time of your timezone. The FindFirst and FindNext functions retrieve (from the operating system) the time in "system time" aka UTC. Now you need to translate that into "local time" so that you get the same time (plus/minus the 2 seconds) that the Explorer displays. In theory the new Timestamp property should do that, but that's only the case in trunk (see below for an explanation). So you'll have to adjust that yourself.

ejohnstone's response:
I believe a difference of one hour occurs when
   a) a file was saved during daylight saving time but it's standard (not daylight) time currently, and "Adjust for daylight saving time automatically" is set to "On,"   or
   b) a file was saved during standard time but it's daylight time currently, and "Adjust for daylight saving time automatically" is set to "On,".  Is this an FPC bug?

   I tested the 8 combinations of those 3 booleans. See the table below.

   My computer's Time zone is set to "(UTC-05:00) Eastern Time (US & Canada)." The switch for "Adjust for daylight saving time automatically" is set to "On." Both files were saved with those settings on the dates shown; i.e. I did not change the date manually and then save the files.
   
       FileSaved        Daylight     Daylight         Windows                Lazarus         UTC
        during          Time Now?   Adjust
     Daylight Time?
     ----------------------------------------------------------------------------------------
           y                  y               on     July 14, 2020  9:49:41     9:49:42       13:49:41
           y                  y               off                          8:49:41     8:49:42
           y                  n               on                          9:49:41     8:49:42 <--
           y                  n               off                          8:49:41     8:49:42
           n                  y               on      Dec 19, 2019 22:25:59    23:26:00 <--
           n                  y               off                          22:25:59    22:26:00
           n                  n               on                          22:25:59    22:26:00        3:25:59
           n                  n               off                          22:25:59    22:26:00

   It seems like there's logic somewhere that says "If the Daylight Adjustment switch is set to 'On' then
adjust the Timestamp according to whether or not it's Daylight Time currently. That apparent and incorrect logic might be the result of which Windows function or API is called to perform the adjustment from system to local time.

https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times

In particular it says:

"To convert a file time to local time, use the FileTimeToLocalFileTime function. However, FileTimeToLocalFileTime uses the current settings for the time zone and daylight saving time. Therefore, if it is daylight saving time, it takes daylight saving time into account, even if the file time you are converting is in standard time."

"The NTFS file system records times on disk in UTC. To account for daylight saving time when converting a file time to a local time, use the following sequence of functions instead of using FileTimeToLocalFileTime:

FileTimeToSystemTime
SystemTimeToTzSpecificLocalTime
SystemTimeToFileTime"


jamie

  • Hero Member
  • *****
  • Posts: 3523
Re: Different date and time for files, Windows versus Lazarus
« Reply #10 on: July 20, 2020, 01:49:26 am »
its just rounding the msec in the filetime ..

that would be my assumption.. Maybe windows is truncating  it or Fpc is always going up one.

The only true wisdom is knowing you know nothing

ejohnstone

  • New member
  • *
  • Posts: 5
Re: Different date and time for files, Windows versus Lazarus
« Reply #11 on: July 20, 2020, 02:12:52 am »

Jamie I assume you're referring to the one or two second difference and not the one hour difference in my most recent post. With respect to the one second difference I'm satisfied that it's because the resolution in TSearchRec.Time is 2 seconds.

jamie

  • Hero Member
  • *****
  • Posts: 3523
Re: Different date and time for files, Windows versus Lazarus
« Reply #12 on: July 20, 2020, 02:22:07 am »

Jamie I assume you're referring to the one or two second difference and not the one hour difference in my most recent post. With respect to the one second difference I'm satisfied that it's because the resolution in TSearchRec.Time is 2 seconds.

 Yes I was referring to the round up at the end ..

 The one hour difference most likely is your DST value.

 But the msec issue is where many coders just set that to 0 and do not include that in the seconds.

 If memory serves I believe the msec field isn't rolled over at 60 which can cause issues which is why most just set it to 0 and don't include it in the seconds.

 FPC/LAZ most likely treating it differently..

The only true wisdom is knowing you know nothing

ASBzone

  • Sr. Member
  • ****
  • Posts: 461
  • Automation leads to relaxation...
    • Free BrainWaveCC Console Utilities
Re: Different date and time for files, Windows versus Lazarus
« Reply #13 on: July 25, 2020, 12:33:44 am »
I'd like to know how to include previous posts in those nifty boxes with the posting details at the top but I'm such a darn newbie.

Select the QUOTE button of the message you want to so quote.
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.11 r63516 / FPC v3.2.1-r46879 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 2004 (Build 19041.508)
Other Systems: Windows 10 Pro x64, Version 2004 or greater

ejohnstone

  • New member
  • *
  • Posts: 5
Re: Different date and time for files, Windows versus Lazarus
« Reply #14 on: July 28, 2020, 03:37:33 pm »
NTFS stores the time as UTC, but Windows displays them in your local time. Take 7:42:04 (UTC) for example. In my case (European Central Time - Summertime) it will be displayed as 9:42:04. The operating system itself internally also operates with UTC (thus "system time"). For display purposes however this time is translated into "local time", meaning the time of your timezone. The FindFirst and FindNext functions retrieve (from the operating system) the time in "system time" aka UTC. Now you need to translate that into "local time" so that you get the same time (plus/minus the 2 seconds) that the Explorer displays. In theory the new Timestamp property should do that, but that's only the case in trunk (see below for an explanation). So you'll have to adjust that yourself.

PascalDragon, any comment on my post of July 20, 1:35:44 regarding the Automatic Daylight Adjustment flag?

 

TinyPortal © 2005-2018