I Always get this message to begin with:
"Directory: C:\lazarus\fpc\2.6.4\source\
You get that error when you start the IDE, or while the installer runs?
The answers below are all assuming when you start the IDE.
If "while the installer runs" => check your environment setting PATH => somewhere in there might be an entry pointing to an old fpc install, anywhere on your system.
As do many Windows Applications, Lazarus keeps the user settings when you uninstall it.
They are usually in C:\Users\USERNAME\AppData\Local\lazarus
But you seem to have found them, as you said you deleted them.....
Just in case, when you make a new install, the installer has a checkbox to clear them.
However normally you should not need that checkbox (and if you do, back it up, you can safely restore at least EditorOptions.xml, and some others but I would need to check and find out which ones)
I am a bit puzzled by your description. Lazarus really only looks in 2 places.
The directory in which it is started (the install directory)
The "primary-config-path" => that is C:\Users\USERNAME\AppData\Local\lazarus (unless you give it on the commandline, or have it configured in <install_path>/lazarus.cfg
Both of them you deleted.
There is no registry (not for lazarus / the inno installer creates registry, for associating file locations, and for where the uninstaller is / but none of them are accessed when running lazarus/fpc)
In case you start your Lazarus via a link on the desktop or startmenu, make sure they do not point to some old forgotten installation.
You can open the folder you installed to (e.g. c:\lazarus) and run lazarus.exe directly from that folder.
(Yes for that test you can skip startlazarus)
If that test goes ok, you can test with startlazarus.exe.
You should not need to download fpc separately it is included.
So here are some steps to test, to check your intstall.
After you installed (and make sure your downloaded installer does not contain the word "cross")
You should have (just a selection, there are more) / * can either be i386-win32 or x86_64-win64
c:\lazarus
c:\lazarus\ide
c:\lazarus\mingw\*\gdb.exe
c:\lazarus\fpc\3.2.0\bin\*\fpc.exe
c:\lazarus\fpc\3.2.0\bin\*\fpc.cfg <<< important
If you open fpc.cfg it contains some file/dir locations, they should all be in c:\lazarus\fpc\.....