Its highly unlikely a package like that affects the speed of compilation.
It would/could only if it was constantly "working" => then you would notice the IDE having high CPU usage even if it is completely idle. But again, I don't expect that to be the case.
What may however have happen is that the IDE is now compiled with different optimization. Installing a package recompiles the IDE (usually only very small bits of it, but you can trigger it to be more...).
If that is the case:
- go to: Tools > configure build Lazarus
- Select the profile "optimized IDE"
- and make sure the "custom options" have -O2 and then DO NOT have -gh
- then press "build" and when finished restart the IDE
If the config already is on "optimized IDE" then the rebuild is likely not changing anything.