Forum > Database

distinct values from very large table

(1/5) > >>

Joanna:
Hi everyone
I have an sqlite  table with over 73 million rows that has 8 numerical columns. I want to gather the distinct values from the 8 columns and put them into a combobox.

The process of getting distinct values for even one column seems to be very slow and consumes a lot of cpu cycles.

Is there any other way to do this ?

cdbc:
Hi
I would say SQL, but the material you're providing is a bit on the skinny side  :D
Regards Benny

Joanna:
Maybe I'm missing an index for this?

cdbc:
Hi
Hmmm, that depends... Does this table of yours come with chairs?
...And what is it made of? ...old apple-crates?!?
More info, please  ;D
Regards Benny

Zvoni:
As Benny said: SQL is (usually) the way to go, though deduplicating is always expensive

To get a better idea:
of those 73 Million rows, per each column of those 8 columns: What's the ratio between distinct values and the count of rows?
e.g. of 73 million Values in Column 4, there are 20 distinct values and so on.

I have an idea which might just work

OH, AND WHICH DBMS?

Navigation

[0] Message Index

[#] Next page