I'm sorry, I don't have any experience with RasPi/ARM. But maybe you are right: the source of graph.pp uses an x86 unit which probably is not available on the ARM. But in the unix graph folder there is also a "ggigraph.pp" which does not require it. You could look for this one.
Maybe my previous post was confusing, because I mentioned the sources folder, but I think fpc installs without sources. In my Linux the compiled graph unit is in in /usr/lib/fpc/<version>/units/i386-linux/