Win 10 64bit is one part of the story. The other one is Lazarus, it must be 64 bit as well.
I only have an old Laz 1.44 installation with 64 bit, because I normally work with 32 bit. But with this 64-bit system my demo runs without a crash even with 25 million rows. It's no fun working with it, though, because scrolling is very slow. Part of it is certainly due to the fact that these 150 millions of double values bring reach the limits of my system memory. Buf even an empty drawgrid with that many cells has already halfe the memory consumptions and does not scroll smoothly either.
I don't know about the performance of VirtualTreeview, but I would not expect it to be substantially faster. We are at the limits of today's PC's performance here.
And no: writing to a file and loading into any visual control will not be faster.
But to be honest: why do you want to show that many data? Nobody will look at them, not even you. You must write the results of these calculations to file which you evaluate with some dedicated analysis to extract the information that you need. Since I do not know what you want to achieve I cannot give you more specific hints.