Pascal Security

[Warfley]

Maybe it's for you personally a dealbreaker, but for a lot of people it isn't.

The thing I am wondering is, people here start wondering about all sorts of problems this would entail, when in reality, this is not new territory by any means. Unicode support in identifiers is a staple of modern programming languages for over a decade now. Swift easiely allows you to use emojis as identifiers since Swift 3, i.e. over 5 years. In python unicode identifiers are since python 3.1 released over 12 years ago. In C and C++ the unicode identifiers where already part of the C 98/99 standard, so nearly 25 years old, but where only implemented around the 2010s by the compilers, making them still at least 10 years old. C# allows also unicode identifiers, explicitly mentioning that control and formatting sequences (like right-to-left direction changes) are allowed within an identifier (as we had the topic earlier), and while I have not found out when this was added (potentially at release around 20 years ago) I think I have first seen unicode in C# around the 2010s making this also at least one decade.

So this is not anything new, if this would be implemented it would be utilizing 10-20 years of experience gatherd by millions of programmers using the most popular languages around. This stuff has proven to work well.

[MarkMLl]

Pascal already bucks the trend by ignoring case, so direct comparisons are hazardous.

C# allows also unicode identifiers, explicitly mentioning that control and formatting sequences (like right-to-left direction changes) are allowed within an identifier (as we had the topic earlier),

I think the only way that is compatible with Pascal's tradition of robustness is to allow a direction specifier at the front of an identifier, but not embedded once the visible characters have started. Modifiers to insert an accent etc. are a different issue, as are things like emoji (am I alone in feeling that those ought to be pluralised as "emojim"?).

MarkMLl

[winni]

Hi!

The future pascal looks like that:

if 😈 = ⓿ then inc(🙎);

???

Winni

[Warfley]

English is lingua franca all over Europe.
[...]
English is lingua franca in all those parts of the world which once were occupied by Great Britain like parts of Arabia, Africa  and Asia.

In India it is official language.

About this thing, just because it is a governmental language, i.e. that the government will publish their documents in that language, does not necessary mean that the population will speak this. See https://www.wikiwand.com/en/List_of_countries_by_English-speaking_population
To take your example, India only 14.18% of the population has learned english as first, second or third language. In pakistan it is 49%. In France it is 39% and even in canda its just 83%

Considering developing countries like most of Africa it is even worse. Namibia as an example has 9 official languages. The reason is exactly that there is no language that is understood by all the people, but it is more like that in some regions many people speak german, in others Afrikaans in others english and so on. English literacy here is 17%
This is not comparable to european countries, most of the former countries are not nation-states as in the european sense, but more like souvren governments without a national identity. So you have a lot of different cultures under one government that do not share much cultural commonalities, including language.


Also english literacy is not evenly distributed, where english is tought as second or third language, it is usally at the mid to late teens before the people are confident in speaking english. Similarly the older a person gets the harder it is to learn a new language. Besides this, especially in the developing world, but even in many european countries, the ability to learn english heavily depends on your socio economic background. If you have enough money to go to a school where english is tought well, you will learn pretty decent english early on. If you don't have the money to go to school at all, or only to very bad schools, or only where able to go to school very briefly before having to go to work to feed their family, you won't learn english.

So do you really think that like half the world, especially children, the elderly and the poor should be excluded from being able to learn programming just so you don't have to look at cyrillic characters?

The future pascal looks like that:

if 😈 = ⓿ then inc(🙎);

This could be possible, thats completely fine by me. Swift allows this for over 5 years now, and while it is theoretically possible, I never saw any serios code using something like this.

This is the next thing, just because something is possible it doesn't mean a large share of programmers will use or abuse this.
If we are talking about excluding a large percentage of the population, the argument should be better than "something that will never happen in reality is theoretically possible"

I think the only way that is compatible with Pascal's tradition of robustness is to allow a direction specifier at the front of an identifier, but not embedded once the visible characters have started. Modifiers to insert an accent etc. are a different issue, as are things like emoji (am I alone in feeling that those ought to be pluralised as "emojim"?).

Yes there can of course be restrictions, not every language goes full C# of Swift allowing pretty much all of unicode. But even though I have a firm interest in natural languages and how to implement this cross compatible, I simply do not have the knowledge about all of the nuances of the different languages and how this can be implemented both securely but without to many restrictions for native speakers. If I had to solve this, I would look at the different language specifications and the release notes, which usually contain discussion about the ideas and why this was chosen.
Thats why I am saying, this isn't some new problem we have to solve ourselves, there are a lot of solutions out there, we just need to look at what is done where and why, and what would fit Pascal best.

Personally, for me pascal was always about readability, and sticking close to natural language, so I think when implementing unicode it should continue with that idea and allow the most natural type of writing for the programmer. But thats just my personal oppinion

[MarkMLl]

Besides this, especially in the developing world, but even in many european countries, the ability to learn english heavily depends on your socio economic background. If you have enough money to go to a school where english is tought well, you will learn pretty decent english early on. If you don't have the money to go to school at all, or only to very bad schools, or only where able to go to school very briefly before having to go to work to feed their family, you won't learn english.

This is thoroughly politically-incorrect and will probably elicit howls of outrage, but if somebody has had abominably-bad schooling he is most unlikely to ever to be in a position to use a computer, let alone to program one.

I am all for improving peoples' lot by schooling, all for making sure that they have access to educational facilities throughout their life, and still slightly optimistic that making computers available to deprived areas will instantly improve the inhabitants' lot: irrespective of plentiful evidence otherwise. But if somebody wants or needs to take up programming, particularly later in life, unfamiliarity with English is going to be the least of his problems.

MarkMLl

[Warfley]

This is thoroughly politically-incorrect and will probably elicit howls of outrage, but if somebody has had abominably-bad schooling he is most unlikely to ever to be in a position to use a computer, let alone to program one.

Thats not necessarily true. First there are some programs of the kind where there are centers providing access to computers for educational purposes (e.g. for learning programming) often also providing courses on how to use them, to people who do not have access to those. Often libraries also provide computer access for people free of charge.
For example a few years ago I've seen a movie made by someone from Uganda who completely edited it on the local libraries computer because it was the only computer they had access to.
And sure having access to computers does not automatically create loads of programmers, but we don't need to make it harder than it already is for them.

I am all for improving peoples' lot by schooling, all for making sure that they have access to educational facilities throughout their life, and still slightly optimistic that making computers available to deprived areas will instantly improve the inhabitants' lot: irrespective of plentiful evidence otherwise. But if somebody wants or needs to take up programming, particularly later in life, unfamiliarity with English is going to be the least of his problems.

Well improving the school education around the world is a neat goal, but it's not something we as a community can archive. What we can archive is make the language more inclusive to mitigate the effects of a bad education system.
Just because we can't fully solve the problem doesn't mean we shouldn't try to do the little things we can do to help.

Also you should look at it from another perspective, Pascal is already not a very widely used language. Depending on the metrics you look at it doesn't even make it into the top 50 and usually is around the #20 of most used languages. There will be loads of more programmers from the developing world, and if they have the choice between a language where they can use their native language and one where they can't, they will probably not use the language that makes programming for them more uncomfortable.
I personally would really like seeing pascal being used more, not less

[440bx]

What we can archive is make the language more inclusive to mitigate the effects of a bad education system.

It seems to me that only perpetuates the flaws in the educational system.   Mitigation is not problem solving and should be used very sparingly, when there are no other options.

[MarkMLl]

What we can archive is make the language more inclusive to mitigate the effects of a bad education system.

It seems to me that only perpetuates the flaws in the educational system.   Mitigation is not problem solving and should be used very sparingly, when there are no other options.

There is absolutely no justification for diluting Pascal's robustness in the interest of "inclusivity". Anybody who wants a manky language should stick with BASIC, PHP or Python (not to mention FORTRAN and COBOL), all of which started off with a bad design and have had enormous difficulty escaping that over multiple decades.

I'm not a Pascal purist and believe that there are facilities in other languages which could be usefully supported by the compiler. But I'd rather see more restrictions in the fundamentals (type checking, identifier equivalence and so on) rather than less.

MarkMLl

[Warfley]

It seems to me that only perpetuates the flaws in the educational system.   Mitigation is not problem solving and should be used very sparingly, when there are no other options.

We aren't talking about like a local education system, but the problem of the poor state of education in the developing world especially for the poor population. The root problem here is global poverty, and I am very confident in saying, solving global poverty is so far out of reach that mitigation is the only thing we can do

There is absolutely no justification for diluting Pascal's robustness in the interest of "inclusivity". Anybody who wants a manky language should stick with BASIC, PHP or Python (not to mention FORTRAN and COBOL), all of which started off with a bad design and have had enormous difficulty escaping that over multiple decades.

But what robustness? Against the attack mentioned here? This still works because unicode is allowed in strings and in comments. Again all the other hypothetical problems mentioned in this thread, like people using lookalike characters from different alphabeths etc. simply don't happen in reality as proven by the history of unicode characters in other programming languages.

There is absolutely no reason to not allow unicode identifiers, except maybe for implementation overhead in the compiler

[Blade]

Maybe I'm out of place in understanding the finer details of the debate, but if Free Pascal is to have greater inclusivity, I'm of the opinion that such helpfulness might be better directed towards documentation, eBooks, and additional forums in different languages.

[PascalDragon]

And a hodgepodge of English programming syntax and native identifiers isn't better. Even more so: you should never put the content of strings directly in the code, especially if you want it to work in different languages.

And that's where you're wrong and thinking to English-centric. Point is that there are non-English people that use non-English words in their identifiers (some German users come to bind here). If those that can use ASCII-based identifiers for their own language why should we not allow that for other languages as well? That is unfair to those users and again stresses this typical “the Western countries are soooo much better” attitude.
Also it might be hard to translate a domain specific non-English word to English or you'd need to teach such translation to everyone that starts to work with your codebase.

Unicode consists of two different ways to create characters (glyphs). The first is select one of the predefined ones. The second is to build your own glyph by combining glyphs and marks. There is no real way to translate between those two. Also check the Zalgo text link on that page.

The keyword here is Normalization. This is documented in the (again) Unicode Identifier and Pattern Syntax which is the base for all this.

Programming languages analyze the syntax and produce a symbol tree. They read a byte stream and convert that into symbols. It is really hard to do that if each, unique, character can be multiple byte sequences, each unique identifier can consist of different characters, the identifiers and syntax are different between different languages and if you want to combine those, each word has its own reading direction.

Yes, I know. AI is improving exponentially at the moment. So, in 100 years or so this might be totally normal. But at the moment, there is no way we can do any of those things at all, whatsoever.

This has nothing to with AI, reading direction or whatever. This is simple Unicode character handling which is well defined and used for years already (not only Unicode in general, but Unicode identifiers in programming languages especially). This isn't any new science and would work on a e.g. 80286 as well.

English is lingua franca all over Europe.

And yet not everyone speaks English. On the German forum there are often users that prefer to use the German forum because they say their English is not good enough. They simply feel more confident with their own language. Why should this be any different for others?