Recent

Author Topic: MultiFormat File - Simplest - Flexible  (Read 24060 times)

Fungus

  • Sr. Member
  • ****
  • Posts: 354
Re: MultiFormat File - Simplest - Flexible
« Reply #30 on: August 23, 2016, 12:27:17 pm »
@technipixel: You should really try to think about the code you are writing before asking why is does not work. You are typecasting streams to strings, that will defenately cause trouble. There are no issues with TMemoryStream polluting the stack since it allocates its memory on the heap. The only reason for a TMemoryStream (or any other object) to cause problems in the stack is if the objects it uninitialized. Some of your examples are trying to read past EOS, when something is written to a stream, its position will be at the end of what's written. You must code the position to be where your read operation should start ( beginning: Stream.Position:= 0; or Stream.Seek(0, soFromBeginning); ).

Also; If you want to combine textual and binary resources in a single file you must either accept the lowest level (binary) dictating the format or you must convert the lowest level to eqal the highest levet (text). Converting from binary to text (most common method being Base64) will result in a bulky and inefficient resource system that will be horrid to work with.

@GetMem + Phil: The only problems there is involved in attaching a payload to an executable is problems with signing. The process itself will not be blocked by the OS or by any antivirus since it is nothing more than copying data. Of course, you cannot/should not write to a running program since this might trigger some warnings. Executing a file that has a payload will neither cause problems unless the payload itself is an executable format - this should defenately trigger good antivirus or even the OS.

I've created sooooo many executables with resources / data attached and never, not even a single time, have I had any issues with this procedure. Some 20 years ago I've even written a TAttachedResources object which can handle both textual and binary data attached to a binary or as a standalone and it has never ever caused any trouble :-)
« Last Edit: August 23, 2016, 12:30:22 pm by Fungus »

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #31 on: August 23, 2016, 02:11:32 pm »
@technipixel

As other people pointed out, writing to an external exe is not a very good idea. In some cases the OS will block your application(writer.exe) not to mention the antivirus softwares.
If you still want to go with external resources, here is a working example(windows only):
https://drive.google.com/file/d/0B9Me_c5onmWoZFhwRnAtVnp3bUE/view

1. Compile/Build reader.lpi
2. Compile/Build writer.lpi
3. Start writer.exe(make sure reader.exe is not running). Write the image, text to reader.exe
4. Start reader.exe and load the resources

Basically you can write anything to reader exe, but you have to specify a unique resource name, both when you read and write data.

Okay... this can work.
But, I looked at the code and it looks like you are writing to the EXE anyways.
So, how is the RES external (which everyone is saying I should do)?

So, if I move the Reader.exe to another location it loads the file... so it has to be stored into the EXE, right?

If so, this is exactly what I was asking for a few days ago... a way to embed a RES file into the exe.
That is what I did in VB6 & VB.NET

Do, I have this right?
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

balazsszekely

  • Guest
Re: MultiFormat File - Simplest - Flexible
« Reply #32 on: August 23, 2016, 02:25:56 pm »
@technipixel
Quote
But, I looked at the code and it looks like you are writing to the EXE anyways.
What do you mean? This is the exact requirement from your first post. Take any file(text, image, zip, etc) and write with the help of the first exe(writer) into the second exe(reader). No res(rc) files are involved, this is the whole point. After you write the data(files), please check with a resource editor the rc_data section(reader.exe), you will find two new resource entries: myimage and mytext(see attachment). This was written by the writer, at runtime without any res file.

Quote
So, if I move the Reader.exe to another location it loads the file... so it has to be stored into the EXE, right?
Yes. Reader.exe contains all the data you need. You can take reader.exe to any other computer, it will be able to read back the data with one condition, to know the rersouce name(see attachment again)

Quote
If so, this is exactly what I was asking for a few days ago... a way to embed a RES file into the exe.
Sorry, but I don't know what exactly you asked a few days ago! Can you link the thread?
« Last Edit: August 23, 2016, 02:34:04 pm by GetMem »

derek.john.evans

  • Guest
Re: MultiFormat File - Simplest - Flexible
« Reply #33 on: August 23, 2016, 02:26:15 pm »
If so, this is exactly what I was asking for a few days ago... a way to embed a RES file into the exe.
That is what I did in VB6 & VB.NET. Do,

I have this right?

Not quite. I think you were told a few days ago to google for Delphi/FreePascal code using BeginUpdateResource, UpdateResource & EndUpdateResource. Which is what GetMem is using.

At the end of the day. Its all the same. You should end up with a TStream with your data in it. Which will either be a TResourceStream, TFileStream or TMemoryStream.

Where the data came from shouldn't matter to your reader code if you think in an abstract way.

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #34 on: August 23, 2016, 02:29:40 pm »
@technipixel: You should really try to think about the code you are writing before asking why is does not work. You are typecasting streams to strings, that will defenately cause trouble. There are no issues with TMemoryStream polluting the stack since it allocates its memory on the heap. The only reason for a TMemoryStream (or any other object) to cause problems in the stack is if the objects it uninitialized. Some of your examples are trying to read past EOS, when something is written to a stream, its position will be at the end of what's written. You must code the position to be where your read operation should start ( beginning: Stream.Position:= 0; or Stream.Seek(0, soFromBeginning); ).

Also; If you want to combine textual and binary resources in a single file you must either accept the lowest level (binary) dictating the format or you must convert the lowest level to eqal the highest levet (text). Converting from binary to text (most common method being Base64) will result in a bulky and inefficient resource system that will be horrid to work with.

@GetMem + Phil: The only problems there is involved in attaching a payload to an executable is problems with signing. The process itself will not be blocked by the OS or by any antivirus since it is nothing more than copying data. Of course, you cannot/should not write to a running program since this might trigger some warnings. Executing a file that has a payload will neither cause problems unless the payload itself is an executable format - this should defenately trigger good antivirus or even the OS.

I've created sooooo many executables with resources / data attached and never, not even a single time, have I had any issues with this procedure. Some 20 years ago I've even written a TAttachedResources object which can handle both textual and binary data attached to a binary or as a standalone and it has never ever caused any trouble :-)

I'm sure the code worked for you. And I understand what it was doing.
But, I kept getting an error for the word Result (Identifier not found).
I set it as a var, but people said don't... but didn't tell how to fix it, so I removed the code.

Remember... I am trying figure out Pascal and the docs don't always help (sometimes they are wrong or outdated)

I agree... writing to an EXE isn't always good for anti-virus and OS's.
But, it doesn't mean it won't work.

I would rather of used an RES file and just embedded it into the EXE.
And I knew LAZ does it through the IDE, but I wanted to do it through code... BUT just could not find any code that created RES file using code.

Still haven't found code on creating and actual external RES file that doesn't require writing to the EXE
I want to figure out how both ways.

GetMEM offered the RES/EXE version and it works perfectly.... That is all I wanted in the beginning.
I would rather not have to create extensive code and write into an EXE the hard way.

So, hopefully I am going where I want to go.

Writing apps and saving normal files is no problem using LAZ. And using a DB isn't a problem

I am just trying to figure out the RES and RES/EXE parts of LAZ/Pascal in conjunction with creating IDE->EXE (user products)

So, I'm sorry it seems I am all over the place... but going from VB to Pascal is not a walk in the park.

:)
« Last Edit: August 23, 2016, 02:43:29 pm by technipixel »
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #35 on: August 23, 2016, 02:35:28 pm »
@technipixel
Quote
But, I looked at the code and it looks like you are writing to the EXE anyways.
What do you mean? This is the exact requirement from your first post. Take any file(text, image, zip, etc) and write with the help of the first exe(writer) into the second exe(reader). No res(rc) files are involved, this is the whole point. After you write the data(files), please check with a resource editor the rc_data section, you will find two new resource entries: myimage and mytext(see attachment). This was written by the writer, at runtime without any res file.

I was saying that because you said it was an example of external RES file .
In my mind a Resource file is an actual physical file.... my mistake
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #36 on: August 23, 2016, 02:41:22 pm »
If so, this is exactly what I was asking for a few days ago... a way to embed a RES file into the exe.
That is what I did in VB6 & VB.NET. Do,

I have this right?

Not quite. I think you were told a few days ago to google for Delphi/FreePascal code using BeginUpdateResource, UpdateResource & EndUpdateResource. Which is what GetMem is using.

At the end of the day. Its all the same. You should end up with a TStream with your data in it. Which will either be a TResourceStream, TFileStream or TMemoryStream.

Where the data came from shouldn't matter to your reader code if you think in an abstract way.

I did search for those, couldn't find any examples. Most of what I found was dealing with the LAZ internal RES window in the IDE.
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

derek.john.evans

  • Guest
Re: MultiFormat File - Simplest - Flexible
« Reply #37 on: August 23, 2016, 03:01:19 pm »
I did search for those, couldn't find any examples. Most of what I found was dealing with the LAZ internal RES window in the IDE.

It was in the post?
http://forum.lazarus.freepascal.org/index.php/topic,33763.msg219219.html#msg219219

http://forum.codecall.net/topic/73254-modify-resource-content-of-an-executable/

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #38 on: August 23, 2016, 03:06:05 pm »
To Give everyone a clear understanding of What I know about Resource files in VB.NET, see the graphic below.

The reason I like External RES files & Embbeded RES files in EXE... is because a Resource file can contain many format types, and you can access them with code by just calling the "item name"

There are many uses for resource files out side of flat or db files (even though you can use a db file to store many formats too)

I have read how LAZ has embraced the Resource File concept... I am just trying to find code example on how to do both versions...

1) As a physical external file that can be read by an exe
2) An embedded Resource file that is stored in the exe

I did it in .NET, now trying figure out hoe Pascal does it.

Finding good code has been very challenging... not a lot out there.
And the docs are weak.
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #39 on: August 23, 2016, 03:10:58 pm »
I did search for those, couldn't find any examples. Most of what I found was dealing with the LAZ internal RES window in the IDE.

It was in the post?
http://forum.lazarus.freepascal.org/index.php/topic,33763.msg219219.html#msg219219

http://forum.codecall.net/topic/73254-modify-resource-content-of-an-executable/

Oh, I did look at that and tried the example... but it was really basic just dealing with a couple of lines of text, not images, rtf, binary etc... so I kept looking.
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

Fungus

  • Sr. Member
  • ****
  • Posts: 354
Re: MultiFormat File - Simplest - Flexible
« Reply #40 on: August 23, 2016, 03:11:08 pm »
Windows Resources are handled differently depending on the compiler used. Often the resources are nothing but text (*.RC) files that are compiled with a resource compiler (eg. GNU WindRes.exe) to *.RES and then embedded in the executable.

https://sourceware.org/binutils/docs/binutils/windres.html
https://msdn.microsoft.com/en-us/library/windows/desktop/ms648008%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

But.. Windows resources are NOT portable - they do not work at all on Linux and MacOS. If you are developing for portability you should not consider using them at all.

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #41 on: August 23, 2016, 03:12:52 pm »
Windows Resources are handled differently depending on the compiler used. Often the resources are nothing but text (*.RC) files that are compiled with a resource compiler (eg. GNU WindRes.exe) to *.RES and then embedded in the executable.

https://sourceware.org/binutils/docs/binutils/windres.html
https://msdn.microsoft.com/en-us/library/windows/desktop/ms648008%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

But.. Windows resources are NOT portable - they do not work at all on Linux and MacOS. If you are developing for portability you should not consider using them at all.

I know that are not portable.. so what is an alternative way to it beside just do individual files.
How does MAC and Linux handle Resource Files?
« Last Edit: August 23, 2016, 03:16:52 pm by technipixel »
Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

Fungus

  • Sr. Member
  • ****
  • Posts: 354
Re: MultiFormat File - Simplest - Flexible
« Reply #42 on: August 23, 2016, 03:16:05 pm »
I know that are not portable.. so what is an alternative way to it beside just do individual files.
How does MAX and Linux handle Resource Files?

I've tried to help you with that in several threads now :o

Just look here: http://wiki.freepascal.org/Lazarus_Resources

derek.john.evans

  • Guest
Re: MultiFormat File - Simplest - Flexible
« Reply #43 on: August 23, 2016, 03:17:39 pm »
Finding good code has been very challenging... not a lot out there.
And the docs are weak.

? Delphi has been around for 20+ years. BeginUpdateResource() has been in Windows since 2000.
ie: There is a ton of information. Available documentation and code has _never_ been better.

I think its a case of you thinking, "how do I" = "can someone code it for me, and I'll say I wrote it".

pixelink

  • Hero Member
  • *****
  • Posts: 1260
Re: MultiFormat File - Simplest - Flexible
« Reply #44 on: August 23, 2016, 03:25:57 pm »
I know that are not portable.. so what is an alternative way to it beside just do individual files.
How does MAX and Linux handle Resource Files?

I've tried to help you with that in several threads now :o

Just look here: http://wiki.freepascal.org/Lazarus_Resources

I have read that a couple of times.
I get the impression that is creating it while you are in LAZ (through coding in Lazarus itself)
Or lazres through a compiler.

I am talking about creating the Resource file as function in my already compiled EXE.
Am I missing something on that page?

How would a user create the Resfile using the compiler of LAZ? they can't)
If I wanted to do this within LAZ, I would just use the Resource window in LAZ

THINK USER CREATED STUFF from an IDE exe.

:)



Can't Type - Forgetful - Had Stroke = Forgive this old man!
LAZ 2.2.0 •  VSSTUDIO(.Net) 2022 • Win10 • 16G RAM • Nvida GForce RTX 2060

 

TinyPortal © 2005-2018