Recent

Author Topic: Faster CopyFile ?  (Read 1113 times)

zxandris

  • Jr. Member
  • **
  • Posts: 67
Faster CopyFile ?
« on: February 12, 2024, 10:57:34 am »
Is there a faster way to copy files, preferably preserving the modified and creation dates?  copyFile seems quite slow, it is low resource for the most part but wouldn't mind something faster if there is a way, only thing I don't want to show any dialogs or anything, something that can be run in the background.

Any help would be most appreciated.

CJ

cdbc

  • Hero Member
  • *****
  • Posts: 970
    • http://www.cdbc.dk
Re: Faster CopyFile ?
« Reply #1 on: February 12, 2024, 11:06:42 am »
Hi
I guess the native OS-Api-functions are as optimized as you can get...
You could try to roll your own and then increase the buffersize to some sensible 'chunk' depending on your available RAM and current OS  %)
But, if it will help, I dunno...
Regards Benny
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

jamie

  • Hero Member
  • *****
  • Posts: 6032
Re: Faster CopyFile ?
« Reply #2 on: February 12, 2024, 03:41:31 pm »
Did u try shfileoperation ?
The only true wisdom is knowing you know nothing

rvk

  • Hero Member
  • *****
  • Posts: 5978
Re: Faster CopyFile ?
« Reply #3 on: February 12, 2024, 04:10:52 pm »
Did u try shfileoperation ?
I wonder why this was never used in FileUtil.CopyFile()  :D

All about cross-platform (with the least effort), not about efficiency  ;)

ASerge

  • Hero Member
  • *****
  • Posts: 2199
Re: Faster CopyFile ?
« Reply #4 on: February 12, 2024, 08:23:33 pm »
In my opinion, in Windows, the CopyFile/CopyFileEx API function is fast enough. The built-in robocopy utility, as far as I know, uses it.
For very large files, you can try disable caching in CopyFileEx.

Handoko

  • Hero Member
  • *****
  • Posts: 5106
  • My goal: build my own game engine using Lazarus
Re: Faster CopyFile ?
« Reply #5 on: February 12, 2024, 08:34:18 pm »
I haven't tried but I heard FastCopy can copy files fast because:

Diff HDD:
Quote
Reading and writing are processed respectively in parallel by separate threads.

Same HDD:
Quote
Reading is processed until the big buffer fills. When the big buffer is filled, writing is started and processed in bulk.

Quote
Reading/Writing are processed with no OS cache, so other applications do not become slow.
 It can achieve Reading/Writing performance that is close to the device limit.
...
 It runs fast and does not hog resources, because MFC is not used. (Designed using Win32 API and C Runtime only)

Source: https://fastcopy.jp/help/fastcopy_eng.htm

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11310
  • FPC developer.
Re: Faster CopyFile ?
« Reply #6 on: February 13, 2024, 09:50:06 am »
Did u try shfileoperation ?
I wonder why this was never used in FileUtil.CopyFile()  :D

All about cross-platform (with the least effort), not about efficiency  ;)

Afaik shfileoperation varies in behavour and efficiency with Windows versions, and Windows 2000 misses it.

Kays

  • Hero Member
  • *****
  • Posts: 568
  • Whasup!?
    • KaiBurghardt.de
Re: Faster CopyFile ?
« Reply #7 on: February 13, 2024, 03:18:52 pm »
Is there a faster way to copy files, preferably preserving the modified and creation dates? […]
Yes, system calls like copy_file_range in combination with certain file systems should be faster. For example OpenZFS since version 2.2.0 has a “block cloning” feature: It does not actually copy the payload, so it is faster than duplicating (i. e. reading and writing) data.

You can’t necessarily influence the choice of file system (e. g. a network file system is potentially slower) and operating system (i. e. the set of available system calls) though, so the answer yes, there are faster ways to copy files is probably of theoretical nature.
Yours Sincerely
Kai Burghardt

zxandris

  • Jr. Member
  • **
  • Posts: 67
Re: Faster CopyFile ?
« Reply #8 on: February 13, 2024, 04:21:12 pm »
Thanks guys,

I'll research these options and see what I can do about speeding my copies up :)

CJ

 

TinyPortal © 2005-2018