Fit into L1 cache

[LemonParty]

L1 cache has 32 Kb size.
If I process 1 buffer and output the results into another buffer should I process data by 16 Kb chunks (16 Kb input buffer + 16 Kb output buffer = 32 Kb)? Will this scheme increase the performance?

[LeP]

It's a bit more complex than that.
First of all, the cache is not private to the process but to the core. This means that all processes (i.e., threads) running on the same core (and therefore, even in the case of HyperThreading... on both THREADS) share the same cache.
The situation is more statistical than "mathematical," in the sense that if studied carefully, the result is good, with a fairly high percentage.
However, the cache can be invalidated by a very large number of elements, including instructions that by their very nature invalidate the cache (even those of other processes).

[MathMan]

To add to @LeP's reply.

Your assumption about L1 data cache size is incorrect. I.e. on the Intel architectures the L1 data cache size ranges from 32 kByte to 64 kByte - on the Apple M-series it is 128 kByte for the P-cores and 64 kByte for the E-cores - etc.

On modern CPU there can be an assembly of different core architectures, so your approach would need to recognise the current core architecture it is executed on. And if the OS moves your process to a different core type you wouldn' recognise this ...

Beside that all caches nowadays are supported by really clever read-ahead mechanisms. If your access pattern to your data is not totally random they will tune in to that and retrieve required data in advance. My own operating experience was that leaving things as is usually turned out fastest.

As @LeP stated - it can be advantageous, but then you have to have a very special use case and controll execution finely.

[LemonParty]

I am interesting if we have a buffer of ansi chars, let's say of length N. Then we analyze this buffer with SIMD instructions and getting a bitmask of length N/8. Then we return and analyze the buffer and the bitmask. The question is how big should be N so things not drop out from L1 during described process.

[MathMan]

To be blunt - you can't put a fixed size on that for the reasons @LeP (and I) detailled above.

If your analysis (part I & II) runs lineary (increasing or decreasing in memory) through your data than it is best (with high certainty) to simply do a full sweep in both parts. If you fiddle with your wrong assumptions it can produce severe drawbacks. It might be beneficial to reverse the sweep direction between part I & II - but even that is not neccessarily correct for all core architectures.

[LeP]

I thnk the in the modern processors is indifferent this kind of analysis.
Is better to analyze the process, with thread coordination and SIMD analisys ... some SIMD (and AVX) inst. use a full core resources so no parallel execution with same  or simlar instructiona in the THREADS.

If you use Intel, ITD (Intel Thread DIrector) will move your threads around all cores (like siad by @MathMan), so cache is not the issue.

I posted a table with instructions time and "parallel inclusion" somewhere, if you want to do a performance works try to coordinate threads and instructions (with affinity process / threads too).

[creaothceann]

[...] Will this scheme increase the performance?

The only way to know for certain is to measure. Look up QueryPerformanceFrequency and QueryPerformanceCounter if you're on Windows.

[Thaddy]

You can use system.diagnostics.TStopwatch instead. That is cross-platform.
On windows it uses the above.

This probably needs trunk. I don't know if the vcl-compat package is back-ported.

[LeP]

You can use system.diagnostics.TStopwatch instead. That is cross-platform.
On windows it uses the above.

This probably needs trunk. I don't know if the vcl-compat package is back-ported.

Somewhere in this forum is posted the diagnostics unit compatible with every version of Lazarus/FPC (for Windows / Linux)

May be there: https://forum.lazarus.freepascal.org/index.php/topic,73579.msg577508.html#msg577508

[LeP]

I saw in the forum someone write about cache, but no more present.

If anyone want to know more about cache and how the data and instructions operate inside a modern CPU (Intel) can read that manual:

Sector [14.1] of Volume [3A], from page 14-1 of "Intel® 64 and IA-32 Architectures Software Developer’s Manual" update on June 2026.

This is the link with all combined manuals (Volumes 1, 2x, 3x, 4): https://cdrdv2.intel.com/v1/dl/getContent/671200

And this is the link of the Intel site: https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html

[marcov]

I saw in the forum someone write about cache, but no more present.

That was a spammer, and probably AI generated, with the topic titles as input. As there were topics about cache, the AI generated a dummy post about cache. Hidden inside the post was an url to a food delivery service.