Via the RTL docs at
https://www.freepascal.org/docs.html you get to
https://www.freepascal.org/docs-html/current/rtl/objects/tbytearray.html which demonstrates that TByteArray is an array with a fixed maximum size.
Array of byte, OTOH, is a dynamically-sized array defined by a parameter block with the actual storage on the heap.
As a hypothetical experiment, if you used a debugger you would find that the address of element [ 0] of a static array was the same as the address of the overall array, while that is not the case for a dynamic array.
If you really had to you could cast the elements of a dynamic array starting at [ 0] to a TByteArray, but it would hardly be safe practice.
Generally speaking, TByteArray (or a pointer to one) is used as a convenience when passing a parameter of an arbitrary size. One would not be expected to actually allocate storage space for one- either on your stack (a local variable) or elsewhere in memory: the reason should, by now, be obvious.
MarkMLl