Forum > General
Questions on Multi-Thread Speed-Up and Efficiency
MathMan:
Hello all,
I started looking into multi-threaded extensions to some applications of mine. As I'm completely new to this I would appreciate some guidance
on speed-up and efficienvy figures I see. So my questions indeed target people who actually (and successfully) work with multi-threading
developments in Free Pascal.
Here is a simple breakdown of my environment:
- CPU: AMD Ryzen 7 7730U, 8 physical, 16 SMT cores
- Mem: 16 GByte
- OS: Win 11 Home, 25H2, 26200.8655
- Lazarus: 4.2
- FPC: 3.2.2
Here is what I'm doing:
- I have computational tasks that I can divide into fully independent sub-tasks (no common data, no synch requirements, etc.)
- I use threadpool-fp (https://github.com/ikelaiah/threadpool-fp) in the simple 'fire-and-forget' variant to multi-thread
- the data memory footprint ranges from small (1 MByte) over medium (10 MByte) to large (100 MByte)
- I use my own pure Pascal and assembler enhanced compute kernels
- I minimised heap management, allocating one large chunk and doing pointer math to provide fitting parts to sub-tasks
- I don't do anything 'special' yet, like playing with affinity or priority, switch to large/hughe OS memory tables etc.
Here is what i measure:
- using pure Pascal kernels
- 8 threads (aligned to physical core count) provide speed-up ~5.8 vs single thread and efficiency of ~73% vs physical cores
- 16 threads (aligned to SMT core count) provide speed-up ~6.6 vs single thread and efficiency of ~83% vs physical cores
- using assembler enhanced kernels
- 8 threads (aligned to physical core count) provide speed-up ~4.6 vs single thread and efficiency of ~58% vs physical cores
- 16 threads (aligned to SMT core count) provide speed-up ~4.8 vs single thread and efficiency of ~60% vs physical cores
Here are my questions:
- do the above figures look reasonable?
- can I expect substantial/measurable improvements if I invest in 'specials' and what would those be?
Kind regards,
MathMan
Martin_fr:
--- Quote from: MathMan on June 24, 2026, 09:54:35 pm ---So my questions indeed target people who actually (and successfully) work with multi-threading
developments in Free Pascal.
--- End quote ---
I wouldn't say I fit the group, at least not the way you likely mean it. I have done some code using threads, but really not gone deep.
But from my superficial understanding, at least your figures look explainable (and therefore reasonable).
- Since your tasks are independent, I don't expect any locks/critical-section/waiting to happen.
- But your tasks go over huge amount of memory. Likely way outside the cache on your CPU.
The latter would explain the efficiency loss, as now all threads fight for the time to load from RAM. It would also explain the worsening of the effect for your handcrafted asm code, as this would do the compute faster, and the percentage of each thread spending loading from RAM is therefore higher, the slowdown more noticeable.
There are some articles (but I don't recall any) on optimizing memory access/layout. So in case that any of your code repeatedly accesses data, in such ways that it may have been out of cache and needs to be loaded again, then that would be something to look at.
I am not sure if valgrind (cachegrind) may be able to identify this (it wont help with threads, but if you analyze a single thread, then you might learn about its data access...). I haven't done any cache-grinding myself. So I really don't know if it is any good for what you need.
Of course given you have highly optimized your code already, you may already have done some steps of the above...
JCLRQ:
On my algo, it scales perfectly or above with the performance core count. Pass the performance core, going to logical core the scaling slow down by 20-30%. The reason is that my code heavily SIMD aligned and ASM is fighting memory allocation between logical and performance core. I have a AMD 5950X. I use the MultiThread package.
Ask any AI agent to debottleneck your code. I found it much better than me in doing so.
MathMan:
--- Quote from: Martin_fr on June 24, 2026, 10:55:35 pm ---
--- Quote from: MathMan on June 24, 2026, 09:54:35 pm ---So my questions indeed target people who actually (and successfully) work with multi-threading
developments in Free Pascal.
--- End quote ---
I wouldn't say I fit the group, at least not the way you likely mean it. I have done some code using threads, but really not gone deep.
--- End quote ---
Don't belittle yourself - you surely know way more about FPC than me. Though I hinted on the target audience I really appreciate any feedback.
--- Quote from: Martin_fr on June 24, 2026, 10:55:35 pm ---But from my superficial understanding, at least your figures look explainable (and therefore reasonable).
- Since your tasks are independent, I don't expect any locks/critical-section/waiting to happen.
- But your tasks go over huge amount of memory. Likely way outside the cache on your CPU.
The latter would explain the efficiency loss, as now all threads fight for the time to load from RAM. It would also explain the worsening of the effect for your handcrafted asm code, as this would do the compute faster, and the percentage of each thread spending loading from RAM is therefore higher, the slowdown more noticeable.
There are some articles (but I don't recall any) on optimizing memory access/layout. So in case that any of your code repeatedly accesses data, in such ways that it may have been out of cache and needs to be loaded again, then that would be something to look at.
I am not sure if valgrind (cachegrind) may be able to identify this (it wont help with threads, but if you analyze a single thread, then you might learn about its data access...). I haven't done any cache-grinding myself. So I really don't know if it is any good for what you need.
Of course given you have highly optimized your code already, you may already have done some steps of the above...
--- End quote ---
Some food for thought in the above. I haven't played with val-/cachegrind on the single threaded variant - maybe I give it a try.
Regarding memory access pattern - my algo is highly recursive and works more or less over the full data area in memory. I could re-work the recursive descent approach into k (k = recursion depth) linear sweeps over the data area - like things done in modern FFT implementations and such. Will look into that.
Regarding main memory access - yepp, unfortunately all modern consumer CPU are totally memory bandwifth starved, <sigh>
MathMan:
--- Quote from: JCLRQ on June 24, 2026, 11:48:42 pm ---On my algo, it scales perfectly or above with the performance core count. Pass the performance core, going to logical core the scaling slow down by 20-30%.
--- End quote ---
Interesting! Is your memory footprint similar to my own? And do you access memory in a linear fashion (see my answer to @martin_fr)?
--- Quote from: JCLRQ on June 24, 2026, 11:48:42 pm ---The reason is that my code heavily SIMD aligned and ASM is fighting memory allocation between logical and performance core.
--- End quote ---
Same on my side.
--- Quote from: JCLRQ on June 24, 2026, 11:48:42 pm ---I have a AMD 5950X. I use the MultiThread package.
--- End quote ---
So we both run on Zen 3 architecture, but you have 4 times the cache. What exactly do you mean by 'MultiThread package' - the threadpool implementation in FPC?
--- Quote from: JCLRQ on June 24, 2026, 11:48:42 pm ---Ask any AI agent to debottleneck your code. I found it much better than me in doing so.
--- End quote ---
This probably won't happen. I'm a hobbyist and all this is just a pet project. As the freely available AI instances are not up to the task (I tried) and I'm not willing to spend money on this.
Navigation
[0] Message Index
[#] Next page