Just numerical MD5 or something else.

[loaded]

Hi All,
I have a text like Konya-Selçuklu-Akıncılar-101-2 When I convert it to MD5, I get 4abd8df9fadd3ddd19a66ffcd853686e What I actually want is to generate a unique number for the sample text above. Is this possible?

[Zvoni]

Hi All,
I have a text like Konya-Selçuklu-Akıncılar-101-2 When I convert it to MD5, I get 4abd8df9fadd3ddd19a66ffcd853686e What I actually want is to generate a unique number for the sample text above. Is this possible?

Why?

Rewriting 4abd8df9fadd3ddd19a66ffcd853686e to
4a bd 8d f9 fa dd 3d dd 19 a6 6f fc d8 53 68 6e
and seeing that "f" is the highest single "Value" i wouldn't be surprised that this is Hex, which IS a number, with a high chance to being unique for your environment

[Thaddy]

MD5 is simply a 128bit number e.g. $4abd8df9fadd3ddd19a66ffcd853686e is a valid Pascal notation if Pascal would know a 128 bit native OWORD.
You can represent it as a packed record of 2 QWORD and/or 4 DWORD.

[loaded]

Why?

Thank you so much for your answer, Zvoni.
I have a dataset containing only provinces, districts, neighborhoods, attribute 1, and attribute 2. I want to save these to the database with a unique key and also verify the relevance of the unique key to the data.

Yes, I can use MD5 numerically. I just wanted to know if it's simpler or if anyone has done this before.

MD5 is simply a 128bit number e.g. $4abd8df9fadd3ddd19a66ffcd853686e is a valid Pascal notation if Pascal would know a 128 bit native OWORD.
You can represent it as a packed record of 2 QWORD and/or 4 DWORD.

Thank you for your answer, Commander Thaddy. I'll try to do that.

[Zvoni]

Thank you so much for your answer, Zvoni.
I have a dataset containing only provinces, districts, neighborhoods, attribute 1, and attribute 2. I want to save these to the database with a unique key and also verify the relevance of the unique key to the data.

Yes, I can use MD5 numerically. I just wanted to know if it's simpler or if anyone has done this before.

See Thaddy's answer regarding 128-bit
As for Database-Storage:
have you thought about using a GUID instead of MD5?
A GUID is basically a 128-Bit-Number, too. Nevermind, that there are DBMS which offer a ready to use GUID-Datatype for Columns

Since Thaddy mentioned representing a 128-Bit number as a record of 2 QWords:
In your DB-Table define 2 Columns with 64-Bit Unsigned Integer each (for the Lower and Higher Part), and declare them COMBINED as a Primary Key.
Storing the MD5 as Text/String/Varchar or using a GUID not withstanding

EDIT: If your backend is SQLite, i think to remember there is a loadable extension offering the MD5-Function, which can be used directly within SQL-Statements

[loaded]

I think I understand a little more. Thanks for the ideas. I'll try it out as soon as possible and let you know what's going on. Thanks, Zvoni. 🙋‍♂️

[440bx]

@loaded,

Basically, what you are doing is hashing.  In this case using MD5 as a hash algorithm, which is perfectly fine.  The advantage of doing that, which I'm sure you already know, is that you don't actually need to store the hash result since you can recompute it at any time.

With the above out of the way, _any_ hash algorithm cannot guarantee general uniqueness.  It can provide a very high probability of uniqueness.  This is because a hash is a function that maps points from one domain to another but the first domain is larger than the second one, therefore duplicates can always occur but if the second domain is very large, e.g, a 128 or 256 bit number, the probability of a duplicate/collision is very small (provided a reasonably decent hash algorithm which MD5 is.)

if the consequences of having a duplicate are "benign", you may want to accept the possibility.  if the consequences are "dire", you may want to detect the occurrence and perform and additional transformation to eliminate the collision (the double-hashing principle.)

As far as other people having done it before, yes, it's extremely common and it has been done in countless ways.   A search for "hashing algorithm" will yield a lot of information.

HTH.

[Zvoni]

@loaded,

Basically, what you are doing is hashing.  In this case using MD5 as a hash algorithm, which is perfectly fine.  The advantage of doing that, which I'm sure you already know, is that you don't actually need to store the hash result since you can recompute it at any time.

With the above out of the way, _any_ hash algorithm cannot guarantee general uniqueness.  It can provide a very high probability of uniqueness.  This is because a hash is a function that maps points from one domain to another but the first domain is larger than the second one, therefore duplicates can always occur but if the second domain is very large, e.g, a 128 or 256 bit number, the probability of a duplicate/collision is very small (provided a reasonably decent hash algorithm which MD5 is.)

if the consequences of having a duplicate are "benign", you may want to accept the possibility.  if the consequences are "dire", you may want to detect the occurrence and perform and additional transformation to eliminate the collision (the double-hashing principle.)

As far as other people having done it before, yes, it's extremely common and it has been done in countless ways.   A search for "hashing algorithm" will yield a lot of information.

HTH.

I have a dataset containing only provinces, districts, neighborhoods, attribute 1, and attribute 2. I want to save these to the database with a unique key and also verify the relevance of the unique key to the data.

I think he wants to store his Data, including the hash to check if someone has changed something in the database itself (say with an external tool).

Anything else wouldn't make sense, as you correctly pointed out, that he can recalculate the hash anytime

[440bx]

The point was that, a hash is a very common thing to use and general hashes, from one domain that is larger than another cannot every yield a unique number (since the source domain is larger than the target) but, given a target domain that is large, the probability of a collision is usually small (provided a reasonably decent hash algorithm which MD5 definitely is.)

It's up to him to use that information however it suits his needs best.

[loaded]

Thank you very much, valuable information from 440bx Academician. 👨‍🎓

[Thaddy]

Note that md5 has a small, but higher, chance on hash collisions compared to GUID, which has a timestamp component