Recent

Author Topic: PostgreSQL table edit, memory consumption  (Read 578 times)

tatamata

  • Hero Member
  • *****
  • Posts: 736
    • ZMSQL - SQL enhanced in-memory database
PostgreSQL table edit, memory consumption
« on: June 16, 2020, 08:01:12 am »
Hello.
Dynamically created PostgreSQL table (size is not known in advance) is queried by TSQLQuery and presented in RxDBGrid for reading, editing, searching, filtering and exporting to spreadsheet or plain text files. The dataset must be editable and forward/backward browseable, readonly or unidirectional is not an option!
When underlying table is big, memory (16 GB) is completely consumed.
I need recommendations about alternative approaches - instead of TSQLQuery I need some other way to browse/edit underlying PostgreSQL table that will not load all rows in memory at once, but in chunks while browsing.
Is ZTable (from ZeosDBO) a viable alternative?
Is there any other dataset components, instead of TSQLQuery, which can browse underlying PostgreSQL table (for editing), in small chunks, not overconsuming RAM?
« Last Edit: June 16, 2020, 11:28:26 pm by tatamata »

tatamata

  • Hero Member
  • *****
  • Posts: 736
    • ZMSQL - SQL enhanced in-memory database
Re: PostgreSQL table edit, memory consumption
« Reply #1 on: June 16, 2020, 11:35:05 pm »
Hi, any Lazarus Zeosdbo user to provide some recommendation/experience in dealing with browsing/editing huge PostgreSQL tables?

tatamata

  • Hero Member
  • *****
  • Posts: 736
    • ZMSQL - SQL enhanced in-memory database
Re: PostgreSQL table edit, memory consumption
« Reply #2 on: December 02, 2020, 12:57:52 pm »
Just to update on this topic, if anybody else will have similar issue - I have solved my memory consumption trouble by switching from TSQLQuery to Zeso TZTable component.
Dynamically created sql query with "crate table ...as" is executed, resultset is saved as PostgreSQL table, which is then loaded by ZTable component into RxDBGrid.
Indeed, Zeos TZTable does not have memory issues like SQLQuery, with loading huge PostgreSQL table.

 

TinyPortal © 2005-2018