So I've solved my problem by ... resorting to TStringList and TStringGrid after trying TCSVDataSet, TsdfDataSet and TBugDataSet. These facilities offer some advantage over a plain TStringList, because you can hook them up to a TDBGrid in no time and you can see you data immediately.
However, if you require more flexibility such as sorting, hiding columns and filtering data, your options are between a SQL powered data set or TStringGrid.
In my case, my data reside in CSV files and it is constantly updated. I don't feel like stuffing the data inside a database just so that I can visualize it. I ended up writing a bit of code with TStringGrid and TAChart and I am able to filter, do some complex numeric sort, hide columns as needed and display the content of the string grid as a graph in TAChart.
On the other hand, if you require the power of SQL then you might want to use the SQL power data set connected to SQLLite or any of the other database engines supported by Lazarus.