Assuming you are compiling natively, simply add the path.
Assuming you are cross-compiling then i created a bit of a predicament there for myself
My initial thought there was something like cross-compiling from windows to linux but it is of course also perfectly ok to cross compile on linux from 32 to 64bit (or vise verse).
In case the latter, simply add the path again (assuming you have the development libs installed correctly). in case the former you can place them everywhere you would like.
In fact you can place all these files wherever you'd like them to be as long as the paths can be found with Free Pascal/Lazarus. Simply do not mess up existing development installations (usually created by gcc or develop package installations).
For instance for cross-compiling from windows with fpc i've created extra additional (SDK) directories inside fpc/bin directory based on target platform/processor.
PS: A bit more information from your side would be welcome as it ain't exact science there and depends on host/target.