You are doing it wrong. You should use the BMP from the audio file and sync that with the text, not the other way around. BMP detectors are readily available in Pascal. I wrote one for KOL in 2004. See if I can find it. But there are more.
That does exactly what you want: sync text to music, I used it to sync midi or audio with lyrics, kind of autocue for music, mind that the BMP detector should be run in a separate thread, because of possible tempo changes.
I think my code is still at musicdsp.com. Also note that code was quite complex.
Anyway, if it is not at musicdsp I will have a look if I can find it at home. I do not remember if I wrote it in C or Pascal, but if it was C I can help you translate it.