Recent

Author Topic: TThreadID on Mac - insights?  (Read 553 times)

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
TThreadID on Mac - insights?
« on: June 28, 2022, 11:05:32 pm »
Code: Pascal  [Select][+][-]
  1.                 ThId := BeginThread (@AT1, An);
  2.                 writeln (HexStr(QWord(ThId),16),serno:4, name:8,del:8);
Quote

000070001B556000 109 IZXMLGQ    1193
000070001B959000 110 FDYIEHJ    1167
000070001BD5C000 111 CZJCGLV    3745

Above are four thread elements:    TThreadID (assigned on BeginThread) and three elements from the control records (mine) for each thread.

Any insight into why the last three nybles of TTreadID are '0'?
« Last Edit: June 28, 2022, 11:11:06 pm by AlanTheBeast »
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4738
Re: TThreadID on Mac - insights?
« Reply #1 on: June 29, 2022, 09:59:37 am »
Aligned on a 4K boundary?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

440bx

  • Hero Member
  • *****
  • Posts: 2933
Re: TThreadID on Mac - insights?
« Reply #2 on: June 29, 2022, 10:45:12 am »
Aligned on a 4K boundary?
That's a very reasonable guess particularly considering that the first column looks like an entry point (the thread's entry point.)
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
Re: TThreadID on Mac - insights?
« Reply #3 on: June 29, 2022, 12:44:50 pm »
Aligned on a 4K boundary?

I had the same thought, but...

assuming these are control blocks for each instance (active thread),  4K seems large.

I also assume that the code is not replicated for each instance, only the stack - and I believe the stack allocation is much, much larger by default.
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4738
Re: TThreadID on Mac - insights?
« Reply #4 on: June 29, 2022, 01:02:15 pm »
assuming these are control blocks for each instance (active thread),  4K seems large.

I also assume that the code is not replicated for each instance, only the stack - and I believe the stack allocation is much, much larger by default.

Historically, x86 control structures ("descriptors") were (multiples of) 16 bytes allocated on a 16-byte boundary, and the tables that referenced them ("descriptor tables") were limited to 8K entries. It was this that was the fundamental problem with the architecture, not the limited segment size which was fixed fairly promptly.

"Memory is cheap" and "64-bit address space is unlimited". I for one would be not in the least surprised to see a thread indicated by a 4K-aligned structure, particularly since it probably includes substantial context information and part of stack... not the entire stack, but enough for simple non-graphical threads.

"Those who have not studied history are doomed to repeat it."

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 4301
  • Compiler Developer
Re: TThreadID on Mac - insights?
« Reply #5 on: June 29, 2022, 01:44:21 pm »
Aligned on a 4K boundary?

I had the same thought, but...

assuming these are control blocks for each instance (active thread),  4K seems large.

From the operating system's PoV 4k is not large as it's the same as the page size and thus it's the smallest size that the operating system can restrict per process.

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
Re: TThreadID on Mac - insights?
« Reply #6 on: June 29, 2022, 01:52:20 pm »
"Memory is cheap" and "64-bit address space is unlimited". I for one would be not in the least surprised to see a thread indicated by a 4K-aligned structure, particularly since it probably includes substantial context information and part of stack... not the entire stack, but enough for simple non-graphical threads.

Per https://www.freepascal.org/docs-html/rtl/system/beginthread.html each thread is default allocated a 4Mbi stack.  For my needs that is way over the top - I'll look to reducing it to 16KB or so.
(Examining the assembler code there are no calls other than interlocks, random, sleep and gettickcount64 - 2 KB is probably more than ample).

(Per another doc, it's allocated a stack of a size defined by the OS ...).
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
Re: TThreadID on Mac - insights?
« Reply #7 on: June 29, 2022, 01:54:30 pm »

assuming these are control blocks for each instance (active thread),  4K seems large.

From the operating system's PoV 4k is not large as it's the same as the page size and thus it's the smallest size that the operating system can restrict per process.

Got it.  Thx.
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

PascalDragon

  • Hero Member
  • *****
  • Posts: 4301
  • Compiler Developer
Re: TThreadID on Mac - insights?
« Reply #8 on: June 29, 2022, 02:02:58 pm »
"Memory is cheap" and "64-bit address space is unlimited". I for one would be not in the least surprised to see a thread indicated by a 4K-aligned structure, particularly since it probably includes substantial context information and part of stack... not the entire stack, but enough for simple non-graphical threads.

Per https://www.freepascal.org/docs-html/rtl/system/beginthread.html each thread is default allocated a 4Mbi stack.  For my needs that is way over the top - I'll look to reducing it to 16KB or so.
(Examining the assembler code there are no calls other than interlocks, random, sleep and gettickcount64 - 2 KB is probably more than ample).

(Per another doc, it's allocated a stack of a size defined by the OS ...).

2 kB is impossible as the minimum size is - again - 4 kB (namely the page size). But it might also be that the operating system has different minimal values, e.g. on Windows the minimal value is 64 kB (due to the granularity of Windows' memory manager), so even if you specify less, you'll get at least that much. Don't know whether macOS has a similar minimum there.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4738
Re: TThreadID on Mac - insights?
« Reply #9 on: June 29, 2022, 02:38:27 pm »
Per https://www.freepascal.org/docs-html/rtl/system/beginthread.html each thread is default allocated a 4Mbi stack.  For my needs that is way over the top - I'll look to reducing it to 16KB or so.
(Examining the assembler code there are no calls other than interlocks, random, sleep and gettickcount64 - 2 KB is probably more than ample).

(Per another doc, it's allocated a stack of a size defined by the OS ...).

In the general case: assume that those statements are subject to copy-on-write and demand-paging facilities of the architecture and OS.

You will probably find that a 4Mb allocation means that 4Mb of contiguous address space is being allocated to that stack, not that 4Mb of actual physical RAM (i.e. stuff on chips) is comitted.

And that 4Mb portion of address space will be going through at least one level of page-table translation, with the possibility that some of the page tables aren't set up until they're actually needed... the detail will be highly architecture and OS-specific.

I'm not saying I like it. At a minimum, the people who decided that it was safe to allocate such enormous chunks of address space are repeating the same mistake that the Windows NT designers made when they assumed that a 2Gb address space was adequate for application programs... OTOH, those old fogeys are probably retired by now and what did they know? And of the '286 designers who assumed that 8+8K descriptors would never be restrictive. And for that matter they're repeating the mistake of the IBM PC designers who assumed that 640Kb was enough for the OS and a single running program... on the basis that it was ten times the maximum possible memory capacity of an Apple-II. The World moves in strange ways :-)

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
Re: TThreadID on Mac - insights?
« Reply #10 on: June 29, 2022, 03:39:07 pm »
In the general case: assume that those statements are subject to copy-on-write and demand-paging facilities of the architecture and OS.

You will probably find that a 4Mb allocation means that 4Mb of contiguous address space is being allocated to that stack, not that 4Mb of actual physical RAM (i.e. stuff on chips) is comitted.

I'd be very surprised to find a function stack subject to being paged in as needed and not surprised that a thread be initially allocated 4MBi for a stack.  It's a lot, but given that a thread may be making various calls to s/r's with unpredictable depths, it's a conservative, if generally wasteful size - esp. in the case that I'm simulating (prior to committing it to an ARM project).  On the ARM RTL (Ultibo) there is also a substantial stack allocation to threads.  I've cut that down on several threads but not the main (unit) threads.

(As it is I'm running 200 instances of the same thread (Mac), in the actual project (ARM) it will about 12 instances of 3 threads.  TBD (mucky details to be sorted out).
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4738
Re: TThreadID on Mac - insights?
« Reply #11 on: June 29, 2022, 03:50:29 pm »
I'd be very surprised to find a function stack subject to being paged in as needed

Why? Note that I'm talking about physical RAM here rather than space being reserved, but in any event I'd expect the application-level stack to be in process memory and to both be swapped out and back in depending on whether a program is being actively used.

If a program tells the OS it wants to Sleep() for a few minutes, it's unthinkable that both its code and data be locked in physical memory. In fact real OSes are very likely to both swap data to secondary memory and overwrite the code, on the basis that the code can be reloaded from the original binary file... behaviour which is modified by the successors to the old sticky bit.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

AlanTheBeast

  • Sr. Member
  • ****
  • Posts: 255
  • My software never cras....
Re: TThreadID on Mac - insights?
« Reply #12 on: June 29, 2022, 04:53:44 pm »
I'd be very surprised to find a function stack subject to being paged in as needed

Why? Note that I'm talking about physical RAM here rather than space being reserved, but in any event I'd expect the application-level stack to be in process memory and to both be swapped out and back in depending on whether a program is being actively used.

If a program tells the OS it wants to Sleep() for a few minutes, it's unthinkable that both its code and data be locked in physical memory. In fact real OSes are very likely to both swap data to secondary memory and overwrite the code, on the basis that the code can be reloaded from the original binary file... behaviour which is modified by the successors to the old sticky bit.

Not referring to page outs as needed by the OS.  All good.  Referring to the initial instance of the stack being of less size than determined at link (or load) time.

Given sufficient loading, pre-emption will also result in swaps.  All good too (annoying).

IAC - for the Mac, I have 24 GB of RAM - goes weeks with 0 Swaps unless I really load it up with a lot of apps and big files.

For the ARM (Pi) there is no swapping, period, under Ultibo of which I'm aware.
Everyone talks about the weather but nobody does anything about it.
..Samuel Clemens.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4738
Re: TThreadID on Mac - insights?
« Reply #13 on: June 29, 2022, 05:01:16 pm »
Not referring to page outs as needed by the OS.  All good.  Referring to the initial instance of the stack being of less size than determined at link (or load) time.

The initial space from the memory map will of course be allocated. I'd not expect the physical memory to be.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

skalogryz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2762
    • havefunsoft.com
Re: TThreadID on Mac - insights?
« Reply #14 on: June 29, 2022, 06:32:12 pm »
Any insight into why the last three nybles of TTreadID are '0'?
Should that question be referred to Apple rather than FPC?

BeginThread is based on the underlying thread manager, which defaults to Posix Threads for Unix systems.

which in turn takes us to pthread_create function.
(eventually one might want to dig down to _pthread_allocate(), etc...)
and OS allocates whatever it considers needed for a thread. guard pages, register storage, etc.

 

TinyPortal © 2005-2018