I have no idea if time increases linear or not, if the file-count increases.
I can answer that question. The answer is a definite _yes_ without exceptions.
In addition to the number of files increasing the time it takes to get a list of files, the time _also_ increases with the folder nesting level.
Listing the same number of files in a folder at level 0 (root) will be faster than listing the files in a folder at level 3 (for instance) and the time required also increases as the nesting level increases.
Regarding
Marcov's suggestion, the way the files are listed might make a difference (I don't know if it does or not, I always use "details" for all folders) but, even just asking for the "plain" (details - a simple file list) still results in long wait times when the number of files gets around 10,000
HTH.