Some observations on the use of RTTI

[vfclists]

Do you know of any examples which are less UI oriented?

There are two man use cases for RTTI. First the most obvious one is data serialization and deserialization. For example, if you want to take a record and serialize it into json, right now you have to do it like this:

Code: Pascal  [Select][+][-]

1. jsonObject.Strings['MyField1'] := MyRec.MyField1;
2. jsonObject.Integers['MyField2'] := MyRec.MyField2;
3. jsonObject.Arrays['MyArrayField'] := SerializeArray(MyRec.MyArrayField);

Extremely annoying and error prone. With Extended RTTI, where field names are written into the RTTI, you could write a serializer that will just take all public fields and writes them down, recursively descending into other records or arrays. Then it would just be:

Code: Pascal  [Select][+][-]

1. SerializeRecord(@MyRec, TypeInfo(TMyRec));

And when you update your record, rename fields, etc. the serializer will still work because of RTTI.

This is especially useful combined with the custom attributes, where you could do things like that:

Code: Pascal  [Select][+][-]

1. type
2.   TMyRecord = record
3.     [SerializeName('Field1'), SerializeOptional('Default')]
4.     FField1: String;
5.     [SkipSerialize]
6.     FHiddenField: Integer;
7.   //...

so SerializeName would be an attribute to have a custom name, e.g. for backwards compatibility, or to not have the leading F as in this case. SerializeOptional could be an attribute to say when deserializing and this object is not there, fill it with 'Default'. SkipSerialize would tell the serializer that FHiddenField is an internal field not to be serialized into json.

This way you do not need to write a single line of (error prone) code to convert your data into other formats, as all the required information can be encoded into the definition of the type.

The second main use case is reflection. Basically altering the behavior of your code at runtime. For example you could have a module system where you scan your filesystem for a module (e.g. in form of a dynamic library) and if a new module is found the module is loaded and RTTI is used to e.g. load these function pointers into existing objects/classes at runtime without having to restart the application.

For example you could create a software where classes objects have logger fields, and on startup, the module loader would look if there is a logger module available, and if so, scan the classes, if any has a logger field, and dynamically load the logger into that field.

Is the kind of ExtendedRTTI you speak of available in current FreePascal?

[Warfley]

In trunk yeah, but not yet stable

[Fibonacci]

Another issue with RTTI is that it facilitates reverse engineering of the program. It's extremely frustrating to see all type names (classes, records), unit names, and even the program name (from the first line of code). There should be an option to limit RTTI to the absolute minimum when it's not used.

[Joanna from IRC]

Another issue with RTTI is that it facilitates reverse engineering of the program. It's extremely frustrating to see all type names (classes, records), unit names, and even the program name (from the first line of code). There should be an option to limit RTTI to the absolute minimum when it's not used.

That concerns me too. Does anyone know what was the last version of fpc before rtti was introduced?

[vfclists]

Another issue with RTTI is that it facilitates reverse engineering of the program. It's extremely frustrating to see all type names (classes, records), unit names, and even the program name (from the first line of code). There should be an option to limit RTTI to the absolute minimum when it's not used.

I believe this depends on some {$M+} compiler directive which is set by default.

Others may know more.

[Thaddy]

Since today there is some better support for rtti in trunk. I would read the commit discussion if that solves your issue.

Note that $M+ works for anything that derives fromTPersistent, because that is compiled in $M+ mode. So not a default otherwise.

[Bogen85]

Another issue with RTTI is that it facilitates reverse engineering of the program. It's extremely frustrating to see all type names (classes, records), unit names, and even the program name (from the first line of code). There should be an option to limit RTTI to the absolute minimum when it's not used.

That concerns me too. Does anyone know what was the last version of fpc before rtti was introduced?

Not "facilitating" reverse engineering only makes it difficult for the beginner reverse engineers, not the advanced ones. With AI those that are beginners and want to be advanced, using AI to augment one's effort (if they don't get bogged down in wrong direction and are adaptive, eager, and willing to intelligently embrace and leverage AI) can make the lack of RTTI largely irrelevant.

While RTTI might be concerning to some, we are in the 2020s now, lack of RTTI is not going to stop ambitious reverse engineers, even if they are beginners.

[Thaddy]

I do not think there is a real concern about rtti. Nowadays it is a firmly rooted part of languages like C++ and Object Pascal.
On the Pascal side it started with D1 and Lazarus had to provide for that since it's own day one too.
The reason that I do not see a security concern is merely that it allows a standard object streaming mechanism for compiled languages. And you will need that at some point. (ORM comes to mind)
If something can be compiled it follows that it can always be decompiled. For a devoted team of programmers that can sureley be done, but I have rarely seen proper decompilers for compiled languages that compile back to compilable sourcecode. But what they do provide is provide insight.

[Joanna from IRC]

@bogen85 do you have a better way of preventing reverse engineering?

[Thaddy]

Other than a complete criminal law article in every country's criminal law???
You would have to forbid compilers and hexbrowsers.(the latter the simplest form of decompiler)
It is simply silly.

[Fibonacci]

You would have to forbid compilers and hexbrowsers.(the latter the simplest form of decompiler)

There are always individuals like @Bogen85 who say it doesn’t matter to hide things (RTTI in this case). A sophisticated adversary specialized in this narrow field of binary analysis can reverse engineer your binary… Well, I certainly don’t want just anyone to open my .exe LITERALLY in NOTEPAD and say "oh, so you used FPC and this component, noob, maybe I will do something similar and sell it for half your price".

[Thaddy]

Perfect obfuscation.

Code: Pascal  [Select][+][-]

1. [unit fourwaylogic;
2. {$mode objfpc}
3. interface
4. type
5.   Cauldron = set of (false,true);
6. const
7.   EyeOfNewt:Cauldron = [];
8.   ToeOfFrog:Cauldron = [false];
9.   WoolOfBat:Cauldron = [true];
10.   TongueOfDog:Cauldron = [false,true];
11.  
12. {$push}{$notes off}{$hints off}
13. operator and (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
14. operator or (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
15. operator xor (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
16. operator not (const EyeOfNewt:Cauldron):Cauldron;inline;
17. operator in (const EyeOfNewt,ToeOfFrog:Cauldron):Boolean;inline;
18. {$pop}
19. function stable(const EyeOfNewt:Cauldron):Boolean;
20.  
21. implementation
22.  
23. operator and (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
24. begin
25.   result := EyeOfNewt * ToeOfFrog;    // intersection
26. end;
27.  
28. operator or (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
29. begin
30.   result := EyeOfNewt + ToeOfFrog;    // union
31. end;
32.  
33. operator xor (const EyeOfNewt,ToeOfFrog:Cauldron):Cauldron;inline;
34. begin
35.   result := EyeOfNewt >< ToeOfFrog;   // symmetric difference
36. end;
37.  
38. operator not (const EyeOfNewt:Cauldron):Cauldron;inline;
39. begin
40.   result := TongueOfDog - EyeOfNewt; // difference, inverse
41. end;
42.  
43. operator in (const EyeOfNewt,ToeOfFrog:Cauldron):Boolean;inline;
44. begin
45.   result := EyeOfNewt <= ToeOfFrog;  // contains
46. end;
47.  
48. function stable(const EyeOfNewt:Cauldron):Boolean;inline;
49. begin
50.   result := not (EyeOfNewt = TongueOfDog);
51. end;
52. end.

I suppose you know your Shakespeare ? 

[Joanna from IRC]

Thaddy that’s some funny code 
I’m going to agree with Fibonacci. Should we all get rid of door locks because lock pickers exist?

[Warfley]

Another issue with RTTI is that it facilitates reverse engineering of the program. It's extremely frustrating to see all type names (classes, records), unit names, and even the program name (from the first line of code). There should be an option to limit RTTI to the absolute minimum when it's not used.

I myself am not that great at reverse engineering, but know a lot of people in the field, and while it's true that using binaries with symbol names left in is usually recommended to beginners, e.g. especially platforms like macOS or Android native where default build setups usually leave the symbols in the executable (e.g. Android native apps are build as a dynamic library thats loaded by a Java app, and while it would be possible to just export the symbols required my java, many build environments export all symbols from the native code), it's at most considered a slight hint.
Especially if you consider what information RTTI contains. It does not contain information about the functions called, it doesn't even necessarily contain information about what type is used where. Sure you can find managed types due to their initialization procedures and classes through their constructor but thats it. Additionally names alone don't give you anything

So anyone proficient enough in reverse engineering with RTTI enabled will also get there without RTTI. The advantage someone gets for reverse egnineering your code through RTTI is absolutely minimal.

Yet RTTI is required by the FPC for the core functionality for Object Pascal. Managed types (Strings, Arrays, Variant, Records) require RTTI for their management operations. Composition types (Classes, Records, Objects) require RTTI to manage their managed children (e.g. a record containing a dynamic string itself becomes managed by extension).
So while you gain extremely little advantage by turning off RTTI, changing FPC to a degree that it would support both RTTI enabled and RTTI disabled code, would be a massive challange and workload that basically doubles the maintainance work.

So you are requesting adding a lot of additional work for completely miniscule benefits.

[Warfley]

I’m going to agree with Fibonacci. Should we all get rid of door locks because lock pickers exist?

I mean it's a question of costs and benefit. Would you buy a doorlock that costs 10k€ but could be easily bypassed my anone with a 5€ tool?

Because thats in the end what we are talking about, how much it "costs" (in term of manpower) to implement maintan two different systems in the FPC, compared how much it "costs" (also in term of manpower) the attacker to circumvent this.

Also I'm always quite curious whats your use-case, what business risk do you want to mitigate with that measure? Because reverse engineering protection is a very niche use-case. Looking at other dev environments like Java, .Net or even LLVM backed Swift and Objective C on MacOS, do generally publish all their symbols. For the vast majority of System reverse engineering is not a threat that needs to be mitigated. There are commercially available obfuscators for many different languages, but they are a very small niche of products, most developers don't need protection against this.

You usually see it in game development and in IT-Security products. Both things that are usually not the main domain of FPC