Recent

Author Topic: PSA: Don't use FreeAndNil  (Read 9992 times)

alpine

  • Hero Member
  • *****
  • Posts: 1038
Re: PSA: Don't use FreeAndNil
« Reply #75 on: May 25, 2023, 10:49:37 am »
Quote
Again virtual methods and exceptions are the same idea, just in oposite directions. In one the chain is constructed bottom up, in the other one it's top down. Yet in both you delegate your execution to the first instance in the chain that registers a handler for it.
About this, I remembered the joke about the instruction called COMEFROM which, like in the above statement, is semantically equivalent to GOTO: The Worst Programming Language Ever.

Here is an article, Exception Handling Considered Harmful. It's worth a read.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

440bx

  • Hero Member
  • *****
  • Posts: 3946
Re: PSA: Don't use FreeAndNil
« Reply #76 on: May 25, 2023, 11:48:38 am »
Here is an article, Exception Handling Considered Harmful. It's worth a read.
Nice to see there are some programmers who haven't bought into the exceptions' mostly-b.s.

Exceptions are the only reasonable and appropriate solution in very, very few cases and, no case comes to mind where an exception that goes across stack frames is correct.

Well designed, bug-free programs don't use nor need exceptions.
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: PSA: Don't use FreeAndNil
« Reply #77 on: May 25, 2023, 02:34:19 pm »
Here is an article, Exception Handling Considered Harmful. It's worth a read.

I just skimmed over it, and here is a point the author thinks is so important to repeat in big letters to draw extra attention to it
Quote
Forcing the calling code to handle the error right away is the correct approach, because it forces the programmer to think about the possibility of an error occurring. That's a key point. The fact that this clutters the code with error checking is unfortunate, but it is a small price to pay for correctness of operation. Exceptions tend to allow, even encourage, programmers to ignore the possibility of an error, assuming it will be magically handled by some earlier exception handler.

Where is this forcing supposed to happen? Let's look at classical, non exception code that is in production in the FCL right now: https://gitlab.com/freepascal.org/fpc/source/-/blob/main/packages/fcl-net/src/netdb.pp#l1642
Code: Pascal  [Select][+][-]
  1. fpsendto(sock,@qry,qrylen+12,0,@SA,SizeOf(SA));
  2.   // Wait for answer.
  3.   RTO:=TimeOutS*1000+TimeOutMS;
  4.   fpFD_ZERO(ReadFDS);
  5.   fpFD_Set(sock,readfds);
  6.   if fpSelect(Sock+1,@readfds,Nil,Nil,RTO)<=0 then
  7.     begin
  8.     fpclose(Sock);
  9.     exit;
  10.     end;
  11.   AL:=SizeOf(SA);
  12.   L:=fprecvfrom(Sock,@ans,SizeOf(Ans),0,@SA,@AL);
  13.   fpclose(Sock);
  14.  
This code does no error handling, especially because it is not forced to handle errors. If instead send and recv would throw exceptions rather than having a return value based error management (and as you can see the return value is easily ignored), errors would not go through unnoticed. But this way there is simply no error handling at all.

Take as a comparison this java code doing up (first Google result): https://stackoverflow.com/questions/10556829/sending-and-receiving-udp-packets#13780614
Code: C  [Select][+][-]
  1. try {
  2.         DatagramSocket serverSocket = new DatagramSocket(port);
  3.         byte[] receiveData = new byte[8];
  4.         String sendString = "polo";
  5.         byte[] sendData = sendString.getBytes("UTF-8");
  6.  
  7.         System.out.printf("Listening on udp:%s:%d%n",
  8.                 InetAddress.getLocalHost().getHostAddress(), port);    
  9.         DatagramPacket receivePacket = new DatagramPacket(receiveData,
  10.                            receiveData.length);
  11.  
  12.         while(true)
  13.         {
  14.               serverSocket.receive(receivePacket);
  15.               String sentence = new String( receivePacket.getData(), 0,
  16.                                  receivePacket.getLength() );
  17.               System.out.println("RECEIVED: " + sentence);
  18.               // now send acknowledgement packet back to sender    
  19.               DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length,
  20.                    receivePacket.getAddress(), receivePacket.getPort());
  21.               serverSocket.send(sendPacket);
  22.         }
  23.       } catch (IOException e) {
  24.               System.out.println(e);
  25.       }
  26.       // should close serverSocket in finally block
  27.     }

It does error handling with try-except. If you remove that try except, the compiler.will refuse to compile, because unlike non exception code, the java compiler actually forces you to handle the exception. So the point that is made in the article is not true for non exception base code, but holds for exception based java. So the article is just simply wrong here.

So I like exceptions because exactly this property. It is very easy to ignore return values, and unlike the blog post, this is not just my personal feeling, I have real world examples to show that this is the case, where classical return value based error handling is not working because programmers just forgott doing error handling.
Exceptions on the other hand can't be missed. The code above will just continue after the error in an invalid state, while an exception, if not handled would do the only right thing, and crash your program, for not being handled.

So if we are talking about the real world, with real code, exceptions solve very real problems.you find in a lot of actual code.

This blog post does not give a single real world code example, it's just the opinion of the author, and also kept extremely vague. I'm sorry, but I try to solve real existing problems I can point to
« Last Edit: May 25, 2023, 02:40:53 pm by Warfley »

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: PSA: Don't use FreeAndNil
« Reply #78 on: May 25, 2023, 03:10:02 pm »
It's important to give return codes, if only to tell the caller if the action was successful, even if errors are handled by the called function. Also, if the called function can only do the action partially, that's very interesting to know for the caller.

Then again, you can also test the result of the action to see what happened. And if the called function handles errors itself, it isn't required to check the result in all cases. Often, nothing more can be done.

In that case, each state change isn't a lineair process that executes multiple actions in sequence. Every next action can depend on a previous one. It's more like a tree.

Let's say you have data of which you need to make a report, for example in the format of a Word document. You can simply process everything in a predefined (possibly recursive) fashion, and throw an exception if anywhere in that process an error is encountered. The result is either a flawless document that contains all the data, or an error.

On the other hand, not all data might be relevant, or require display methods that aren't supported or implemented (yet). Some data might be broken or nonsense. Or it might be in the wrong language / character set. Or the required font might not be available. It might not support some of the full Unicode glyphs. Etc. But you could still generate a partial document, that contains all data that could be processed. Optionally showing where it is incomplete.

Where this difference is shown very often, is when a database is converted from an old application / system to a new one. The database schema is different and very likely much stricter. Much of the old data won't fit very well or at all and is certainly incomplete. In a fault tolerant system, this is not a problem. A strict one won't work at all.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: PSA: Don't use FreeAndNil
« Reply #79 on: May 25, 2023, 03:15:04 pm »
So I like exceptions because exactly this property. It is very easy to ignore return values, and unlike the blog post, this is not just my personal feeling, I have real world examples to show that this is the case, where classical return value based error handling is not working because programmers just forgott doing error handling.
Exceptions on the other hand can't be missed. The code above will just continue after the error in an invalid state, while an exception, if not handled would do the only right thing, and crash your program, for not being handled.

How do you guarantee that Q&A discovers all the specific edge-cases before release? And how do you handle changed standards and connections to third-party and/or external libraries and systems? Unit tests won't help you there, because they only check for expected errors, not the future, unexpected ones.

alpine

  • Hero Member
  • *****
  • Posts: 1038
Re: PSA: Don't use FreeAndNil
« Reply #80 on: May 25, 2023, 04:34:13 pm »
Here is an article, Exception Handling Considered Harmful. It's worth a read.
...

This blog post does not give a single real world code example, it's just the opinion of the author, and also kept extremely vague. I'm sorry, but I try to solve real existing problems I can point to
From where do you think my (and most people here) experience COMEFROM;) I don't need code samples to agree with the article.

See Let Exceptions Propagate, and then: https://medium.com/swlh/how-lines-of-code-made-a-rocket-explode-77df73deb0a4 for a "real world" implementation.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: PSA: Don't use FreeAndNil
« Reply #81 on: May 25, 2023, 05:46:20 pm »
How do you guarantee that Q&A discovers all the specific edge-cases before release? And how do you handle changed standards and connections to third-party and/or external libraries and systems? Unit tests won't help you there, because they only check for expected errors, not the future, unexpected ones.

I will take java as an example, there are other tools for other languages but in Java it is baked into the language.

If your function can raise an exception, you must add that exception to the function signature. When another function calls that function and does not have an try except block catching this exact exception (exceptions are identified by their type, so you should always have a custom type for every type of exception to be able to distinguish them), the compiler will throw an error. Alternatively you can add that exception to the signature of the calling function and pass the "problem" down the Line. Meaning no matter if you want to ignore the exception or handle it, both is always a conscious decision and your exception must be handled at some point. Also because passing the exception down by adding it to the signature just pushes the problem down the line, and you have the same problem just one level down, you are incentives to handle as early as possible, because passing just increases the effort (compared to pascal where just passing down is no effort compared to handling).

When you now later on edit the code of the function and introduce some new exception, you need to add that to your function signature, meaning your compiler will now throw an error on all locations where you called the function requiring you to either handle it there, or add it to their signature as well.

This means at all times, as soon as you introduce a new exception to your function, you are forced to add code for handling it at some point. At its best, there is no way you can miss an exception. There are some practical points with java exceptions (namely inheritance and exceptions that can always be thrown), which in practice does not make it as useful, but in concept you can't have I handled error in your program, and the compiler enforces every time you add new exceptions.

But of course your ability to catch edge cases is only as good as the granularity of your exceptions. If you throw general exceptions all the time, not having custom types for different edge cases, you can't distinguish them, and may miss them. Also if you just catch all exceptions in one block (by not catching a specia exception type but just the base exception type) you also loose all granularity. Modern java tools help you with that, you get warnings when you use too broad of exceptions and automatic code generation helps you to generate try catch for all exceptions. But in the end, the tool can only be as good as it's user

From where do you think my (and most people here) experience COMEFROM;) I don't need code samples to agree with the article.

"The joke language that was designed to be useless is useless therefore this other concept that was designed to actually solve a problem is also useless".
Good thing I don't advocate for comefrom, because comefrom is stupid. Maybe you have some argument that is not "look the thing that was purposefully designed to be useless is useless.

If you don't see the differences between comefrom and exceptions, I expect that you also think that goto is basically the same as a function call.

See Let Exceptions Propagate, and then: https://medium.com/swlh/how-lines-of-code-made-a-rocket-explode-77df73deb0a4 for a "real world" implementation.
Oh yeah the Arianna 5, where they explicitly wrote unsafe code... Great example on how "exceptions" are at fault, hen they literally circumvented Compilerchecks to put a 64 bit float into 16 Bit integer.
I'm sorry, but exceptions where not the error here. Circumventing the strong typing of Ada and then writing wrong values in there was. Just one question, what do you think would have happened if there was no hardware exception and instead the arithmetic would have silently broken down producing undefined values for it's controls?

Remember, the alternative to u cought exceptions are not properly handled errors, it's most of the time just ignored errors. If you expect an error to happen, and you write code to check it, then you would have also wrote an exception handler. The problem with the Ariana 5 was that this error was unexpected, which is why they did not have a proper handler for the exception. This also means, if they didn't expect the error, they wouldn't have code to detect and handle it without exceptions.

A rocket can't navigate on undefined parameters. The exception wasn't the problem, the error that caused the problem would not have gone away without the exception
« Last Edit: May 25, 2023, 05:54:07 pm by Warfley »

Swami With The Salami

  • Guest
Re: PSA: Don't use FreeAndNil
« Reply #82 on: May 25, 2023, 06:40:47 pm »
Quote
...exceptions are identified by their type, so you should always have a custom type for every type of exception...

There is really nothing to say about that - WOW!

Leo Fender was already shipping electric peddle steel guitars when he screwed one of his magnetic pickups on to that brutally ugly thing we eventually called a Telecaster. Thousands of great Western Swing and Rockabilly records featured those guitars. In 1951 Ike Turner was using an electric guitar through a busted amp when he recorded the (legendary) first Rock 'N' Roll record. The electric guitar was the dominant Rock 'N' Roll instrument for the next twenty years until Jimi Hendrix sadly pass away and we got to see Jimmy Page play the damn thing live in concert with a violin bow.

It was all over.

The last Rock 'N' Roll song ever recorded was "I Feel The Earth Move" by Carol King. After that there was "Hocus Pocus" by Focus. That's where we should have just stopped making records altogether.I mean, really, what more could be said?

Anything that was ever cool was only cool for a relatively short time before it become ludicrous.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: PSA: Don't use FreeAndNil
« Reply #83 on: May 25, 2023, 06:53:25 pm »
@Warfley: my question is: how are you going to make sure that all those exceptions aren't raised after release? That is, to me, the main question.

And for a follow-up question: what do you do if you encounter bad data inside your sanitized state machine, like with an imported database that doesn't fit exactly, or interesting new Unicode extensions?

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: PSA: Don't use FreeAndNil
« Reply #84 on: May 25, 2023, 07:38:58 pm »
@Warfley: my question is: how are you going to make sure that all those exceptions aren't raised after release? That is, to me, the main question.

Well that's the Beauty of this approach, there will never be an unhanded exception on release (at least in theory), because if there was one you couldn't compile the program. And as I already said, there is no problem with raising an exception, the fact that an exception was raised does not mean it is an error, Just that the called function was prematurely interruptes because something happens that means the function could not finish normally.
But if you handle that exception this is not a problem. E.g. if you have a calculator with user input. If the user tries to divide by 0, an exception will be raised. If you catch that exception and notify the user that they need to enter a non 0 divider, and let them retry, it's not an error, nur just normal functioning of the program.

Similarly, if you have a chat program and read from a TCP stream and the other party ends the chat by closing the stream, you get an exception when trying to fetch the next message, but this isn't an error, Just a notification that the stream was closed. So you catch that exception and write code that then prints the "chat was closed by the other side" line. (E.g. in websockets, while there is a close message, it is not required to send this message, if the TCP stream closes that also ends the websockets stream, so while you expect a close most of the time, a disconnect when waiting for the next message is a totally valid way to close the connection)

Exceptions are just a way to notify the programmer about an unexpected event. If you have code to handle that event correctly, there is no error that needs to be removed, it's just the way you handle unexpected situations.

And for a follow-up question: what do you do if you encounter bad data inside your sanitized state machine, like with an imported database that doesn't fit exactly, or interesting new Unicode extensions?
If you have mechanism (like the aforementioned custom types and typechecking) to ensure that this can't happen (so when having an email type you know that it is definitely valid formatted according to RFC 5322), unless you did something very unsafe (like pointer casting, using move, or anything else that circumvents typechecking), this simply can't happen.

But of course sometimes you may not have this, either because compiler based checking are too restricive, or you just don't have the time/capacities to write all this extra typing code, you need testing, lots of testing. The problem here is that you have functions which expect a certain (sanitized) input, so you need to verify that this input will be provided by the other components.

This is what integration tests have been developed for. If you are familiar with the V model, they are the step above the unit tests. Where unit tests test if a function works correctly, integration tests test if the function is used correctly. As such this is not a problem that can be solved easily on a code level, but require a bit more holistic approach, and should be designed before the first line of code is actually written, during system design.

Tooling in this area is usually part of component design tools. For example when planning you can create a basic design and architecture in UML and use OCL (object constraint language) to define interfaces through the use of pre and post conditions. You can then use a verifier to verify that you UML/OCL model is internally consistent, and then what you need  to do is to ensure that your code behaves according to this interface specification.
Part of this is by ensuring that you interface matches the UML definition, which is quite easy to verify. It's harder to show that the code actually follows the constraints set out by OCL (basically if the post conditions hold under assumption of preconditions). But this is what unit tests should ensure. (Also you can use model checking or automatic verification and testing to validate this)

Sadly such tooling is not really exostant for Lazarus, and it's on my list of things I want to make some day but will probably never find time for.
Also if you have a fully integrated design process you can use code generation to generate the bare one structure of your UML/OCL design, ensuring that what you build will be always conforming to your architecture design you modeled beforehand

So it's build on one another. You write your low level unit tests (which is where you would check that exceptions are handled or thrown correctly and so on) and then use integration tests to ensure that your assumptions about the usage and combination of these low level functions behaves correctly

The problem is that testing is hard, it's really hard to write tests for all circumstances, and there is no (non trivial) software that is truly bug free. You just can try to test as thorough as possible and try to divide an conquer to reduce complexity (which is why there is the distinction between unit tests, integration tests, module tests and whole program tests, it tries to reduce from exponential complexity to steps of linear complexity)
« Last Edit: May 25, 2023, 07:45:41 pm by Warfley »

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: PSA: Don't use FreeAndNil
« Reply #85 on: May 25, 2023, 08:25:49 pm »
So, basically, you want to demand, that every possible exception has to be handled immediately at the first level where that can be done. And that there is a test for each of them, to see if the handlers function as designed. Automated or manual. And that classes and libraries never change.

I do the same thing without exceptions, or at least whenever possible (third-party code again, yes, including the FPC and Lazarus libraries). Without the required testing, as that will at most be done by the users, in most cases. And in a way that handles most changes automatically.


Ok, I do like exceptions in one specific case: when quickly prototyping a minimal proof of concept. But as you can see with all my database questions, most edge cases in the included libraries are "handled" by throwing an exception, requiring me to figure out how it works and implementing a functional solution myself. In general, they handle the top 80% of cases well, and throw an exception when encountering the unusual, last 20%. And the framework used is quite large, good luck figuring out how it works.

And unless I make a merge request, which is a lot of work (mostly because you have to keeping multiple builds in sync across platforms, which are hard to deploy), if I just post a working solution, ready to compile, it gets downloaded at most a few times, if that.

In short, you have to find a workaround or discard the whole framework and implement your own, low-level solution.


Which is the whole point of the "either handle it immediately or throw the exception and let it crash" approach: you try to force the devs to fix it. The problem with it is, that they might not care, that they implement a very sloppy fix, or that nobody tells them that it might or will crash in this specific case.

For FPC and Lazarus, that is frustrating but understandable. But if you pay a lot of money for that application, you damn well want it to work flawlessly, or at least "best effort", which definitely means that it shouldn't crash.

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: PSA: Don't use FreeAndNil
« Reply #86 on: May 25, 2023, 08:49:58 pm »
I do the same thing without exceptions, or at least whenever possible (third-party code again, yes, including the FPC and Lazarus libraries). Without the required testing, as that will at most be done by the users, in most cases. And in a way that handles most changes automatically.
Yes you can handle an exception in other ways, as I said many times before. But the other ways have one crucial flaw, if you forgett to do something you will be in an inconsistent state. Exceptions force you to act. Again look at a few posts before where I showed you the fpc Udp code currently in the FCL and the java Udp code. Java will not let the user ignore the connection exceptions. If you forget to do error checking your code will simply not compile, while in the FCL code it's not just that it does compile, but also stays unnoticed for years (if not even decades, netdb is quite old code).

If you so a mistakey would you rather find it early or not find it at all?

To quote what I have written earlier:
Quote
So basically every function that can raise an exception amends the output space. But the thing is, compared e.g. to TryStrToInt, you can ignore the result, in fact you need to actively write code to test if it was successful, this is opt-in handling, where if you do nothing you ignore it. Exceptions on the other hand will, if not handled crash your program. So if you want to ignore them you need to actively write code to catch them. So they are an opt-out way

And this is the biggest difference. When you have something the user needs to react to (e.g. a network stream closes), then you should use the exception, because the user can't easily ignore this. If you have a function that the user should be able to easily ignore (like if the logging file cannot be opened, and logging is not essential to the application).

Exceptions are good if you need the user to react, return types are good when you can savely ignore it

For FPC and Lazarus, that is frustrating but understandable. But if you pay a lot of money for that application, you damn well want it to work flawlessly, or at least "best effort", which definitely means that it shouldn't crash.

This reads to me as if you are comparing code with an error that causes the application to crash with an application that has no error at all. But that's not a fair comparison. If you get rid of the exception you won't get rid of the error that caused it, you just ignore it.

Take the calculator example from above, you divide by 0 but instead of an exception being thrown, nothing happens and the result will just have an undefined (so for the purposes of this example just consider it random) result.
If you have an unhanded exception, your application crashes. Which if I understand you correctly is really bad, because the person that bought your calculator wants that it doesn't crash.
How exactly do you think the customer reacts if I stead of crashing the calculator just spits out wrong/random results? Is having wrong results really better than a crash?
Imagine the customer uses this calculator to calculate rent prices for their tenants. The software doesn't crash, but just puts out wrong result. The unsuspecting user of that software uses the wrong results to bill their tenants, the tenants complain and now the landlord has to recompute everything by hand just to find out your application made a mistake.

In what world is this scenario better than the software crashing. Sure the landlord needs to start from scratch, but it's still much less hassle than if the error is found much later. Even in a scenario where the landlord recognizes the error as soon as the result is produced, because it is not any sensible result, he still has gone through all the steps of the computation, potentially spending an hour or so, while the error may have been produced in the beginning, so a crash could still have saved half an hour of time. More importantly if your software does produce wrong results, you probably would want your money back anyway. It's not like users hate crashes and love getting their time wasted through other malfunctions.
An early crash is always better than getting garbage results later on.

Not using exceptions will not magically make you handle errors you wouldn't otherwise. In the java example it literally means that you would potentially ignore errors that the compiler would otherwise force you to handle. It's just a question what you want to do when you have an unhanded error. So you want to continue and produce garbage output, or do you want a safe crash?
« Last Edit: May 25, 2023, 08:58:13 pm by Warfley »

Thaddy

  • Hero Member
  • *****
  • Posts: 14205
  • Probably until I exterminate Putin.
Re: PSA: Don't use FreeAndNil
« Reply #87 on: May 25, 2023, 08:54:46 pm »
No, basically what he writes is what I wrote many times: an Exception is an Exception, an unforeseeable part of your code.  Exceptions are not for catching foreseeable things. THEY SHOULD BE USED WITH CARE.
Proper code does not need exceptions. It is a sign of bad programming in most cases (not all).
Specialize a type, not a var.

Swami With The Salami

  • Guest
Re: PSA: Don't use FreeAndNil
« Reply #88 on: May 25, 2023, 09:06:06 pm »
Who among us knows what all the exceptions are? We don't even know what an exception is! That's why they're called exceptions...

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: PSA: Don't use FreeAndNil
« Reply #89 on: May 25, 2023, 10:46:55 pm »
No, basically what he writes is what I wrote many times: an Exception is an Exception, an unforeseeable part of your code.  Exceptions are not for catching foreseeable things. THEY SHOULD BE USED WITH CARE.
Proper code does not need exceptions. It is a sign of bad programming in most cases (not all).
This isn't too different from what I say, in my opinion you should use exceptions when something unexpected happens.

For example what I have been bringing up all the time is when you have a TCP stream, which gets closed. A stream can be closed while you try to read or write (or any other blocking operation on the stream). A TCP stream closing is not unforseen, as every TCP stream may be closed at any time, but from the perspective of the read or write function it is unexpected (in the sense when you call recv you expect to receive data, and if the stream is closed it is an exceptions to this expected functionality).
So the stream closing is for seen, but when it happens is going to be unexpected. That's why I think an exception there is valid.

On the other hand, take a non blocking read. Here you expect that most of the time you get no data, therefore I think that an exception (as well as the error return and error being set from the C API, which basically is the C alternative to an exception) is not warranted, but rather the read should just return an empty value.

I've never argued for using exceptions as the normal mode of operation. I'm just saying that you should not try to avoid them, because in such situations (like stream closing, error when writing to a socket, etc.) Exceptions are better than the alternative, because a crash is better than producing undefined garbage

 

TinyPortal © 2005-2018