Windows isn't Linux.
On Linux (may be because its part of the POSIX definition, so may be UNIX too) there's a convention that allows to install various versions of the same library by adding the version id to the library name; and using links you can define different levels of compatibility (i.e liballeg.so.4, liballeg.so.4.4 and liballeg.so.4.4.2 are actually the same library, and your application may use one or another depending of its tolerance to changes, so my Allegro.pas library uses liballeg.so.4.4 by default).
Windows doesn't has that convention so many developers decided to put the DLL files in the same directory than the application. Fortunatelly, several developers have adopted the Linux way and added the version number to the DLL name. For example SQLite library includes such number, but only the major number (sqlite3.dll). Allegro includes two numbers (alleg44.dll/ allegro-5.2.dll).