As substantial change in intelligence will probably reside in mutation & selection.
I believe that will remain true until we (finally) figure out the genetic encoding. After that, we can have "designer" human beings (for the record, I am rather uneasy about that possibility and its consequences.)
And here we are touching the "embodiement theory" - more or less stating, that we will not succeed with AI until we can also supply an instance with a body capable of interaction with the physical world like we humans do.
Basically, stating the same thing you have, I see the problem as A.I not having an independent supply of facts. That is, it doesn't have senses and does not have the means to measure and verify any input. To give an analogy, human beings who are deaf from birth are normally unable to talk. Now imagine the limitations of a human being without any senses... that's A.I.
When talking about artificial intelligence, I always remember an article I read in the 80s. In this article, a top chess player explained why computers could never beat the world chess champion. Obviously he was wrong.
Obviously he was but, Deep Blue was not intelligent, instead it had two things that enabled it to outperform a human being, those are, an immense database of moves and the ability to search it very quickly.
AlphaZero and Stockfish both have ELO ratings greater than that of Magnus Carlsen but, they still don't exhibit any intelligence.
The conclusion I want to reach is that AI is capable of anything. If AI can do something, no matter how badly it does it now, it will eventually do it better than humans, since his possibility of improving is infinite.
A.I does not improve itself, badly or otherwise and, without that it is yet to get on the right track.
Whether an AI feels or is aware is arguable, but if it acts as if it does, where is the difference?
The difference is that A.I is unable to detect contradictions in a reality in which it can not participate. For instance, Einstein figured out Special Theory of Relativity because he saw the physical contradictions in the reality described by the then current models. A.I has no concept of that. A.I is like a two dimensional creature unable to discern the existence of a third dimension.
Anyway, as of today, A.I cannot even come up with basic things like a recipe for an edible dish.