Generative AI - opinion

[ccrause]

Triggered by a comment by TRon about AI and its dangers, I thought I should air my opinion in a quiet corner, and maybe learn something from more informed opinions.

As a test (easy for trained humans I hope) I entered the following prompt in ChatGPT:

Code: Text  [Select][+][-]

1. Write Pascal code to convert temperature from Celsius to bar[1]

It cheerfully complied and generated the following:

Code: Pascal  [Select][+][-]

1. program CelsiusToBarConverter;
2.  
3. uses
4.   SysUtils;
5.  
6. function CelsiusToKelvin(celsius: Real): Real;
7. begin
8.   CelsiusToKelvin := celsius + 273.15;
9. end;
10.  
11. function KelvinToBar(kelvin: Real): Real;
12. begin
13.   KelvinToBar := kelvin / 100000;  // Conversion factor from Kelvin to bar
14. end;
15.  
16. var
17.   celsiusTemperature: Real;
18.   kelvinTemperature, barTemperature: Real;
19.  
20. begin
21.   // Input the temperature in Celsius
22.   Write('Enter temperature in Celsius: ');
23.   ReadLn(celsiusTemperature);
24.  
25.   // Convert Celsius to Kelvin
26.   kelvinTemperature := CelsiusToKelvin(celsiusTemperature);
27.  
28.   // Convert Kelvin to bar
29.   barTemperature := KelvinToBar(kelvinTemperature);
30.  
31.   // Output the temperature in bar
32.   WriteLn('Temperature in bar: ', barTemperature:0:4);
33. end.

It also provided the following explanation:

1. **CelsiusToKelvin function**: Converts a temperature from Celsius to Kelvin by adding 273.15 (since 0°C is 273.15 Kelvin).
2. **KelvinToBar function**: Converts a temperature from Kelvin to bar. The conversion factor used here is 1 bar = 100000 Pascal.
3. **Main program**:
   - Reads the temperature in Celsius from the user.
   - Converts it first to Kelvin using `CelsiusToKelvin`.
   - Then converts Kelvin to bar using `KelvinToBar`.
   - Outputs the temperature in bar.

When you run this program, you'll be prompted to enter a temperature in Celsius, and it will output the equivalent temperature in bar. Adjust the output format (`:0:4` in `WriteLn`) as needed for your desired precision.

So everything looks very neat, with apparently valid code (untested), well commented and a nice explanation to help the unwary code user.

Here mixing coding and physics domains resulted in garbage, the better answer would have been something along the lines of Please do not waste this chatbot's time..  Unfortunately, being a generative model, it cooked up a convincing piece of code that generates complete garbage.  This reminds one about the garbage in, garbage out concept, but then again I expect "intelligence" to imply a wider understanding of the problem domain than the person/entity submitting a prompt.

Note
1. This may be confusing for users of Imperial units, alternatively exchange Celsius with Fahrenheit and bar with atmosphere.

[Jonax]

Pure gold. I'm still laughing. 

[gidesa]

Generative AI is only a statistical algorithm, quite successful to generate pseudo-human phrases.
And, inverse, quite successful to scompose an input phrase in parts: verbs, nouns, etc. So primarily a powerful tool to create a natural language interface from human to computer, alternative to keyboard, mouse, et al.
Capital importance has the training data, as you surely know. If it's trained on thousands of Python programs  and only dozens of Pascal programs, of course the results would be better for Python.
ChatGpt is a "showroom" AI model, trained on a very general and generic data sources. Typically Wikipedia, Github, and similar.
So, regarding computer languages, it's totally biased to some languages (same for human languages, it's totally biased to english).
Conclusion: if you train a GAI model on the right database, it could be a valid STATISTICAL tool also for producing a computer program. "Statistical" in the sense than it will answer quite well, on average, doing a certain number of test with the same question.
You can try to ask ChatGpt the same question, for example, 10 times, and observe the results.   And try to ask to write Python code.

[MarkMLl]

That truly is a classic: thinks for wasting your time on our collective behalf to such good effect :-)

MarkMLl

[af0815]

I think a lot of pupils will give that sample as homerwork away :-) And are woundering if it goes bad. I have seen, pupils wite as result Error - without thinking -> and thinking that was the correct result.

The whole world is now so stupid and crazy.

[MarkMLl]

That code has now appeared in a programming forum, so it will be pounced upon by every LLM out there.

Obligatory xkcd: https://xkcd.com/978/

MarkMLl

[ccrause]

Generative AI is only a statistical algorithm, quite successful to generate pseudo-human phrases.
And, inverse, quite successful to scompose an input phrase in parts: verbs, nouns, etc. So primarily a powerful tool to create a natural language interface from human to computer, alternative to keyboard, mouse, et al.
Capital importance has the training data, as you surely know. If it's trained on thousands of Python programs  and only dozens of Pascal programs, of course the results would be better for Python.
ChatGpt is a "showroom" AI model, trained on a very general and generic data sources. Typically Wikipedia, Github, and similar.
So, regarding computer languages, it's totally biased to some languages (same for human languages, it's totally biased to english).
Conclusion: if you train a GAI model on the right database, it could be a valid STATISTICAL tool also for producing a computer program. "Statistical" in the sense than it will answer quite well, on average, doing a certain number of test with the same question.
You can try to ask ChatGpt the same question, for example, 10 times, and observe the results.   And try to ask to write Python code.

What I was hoping to illustrate with the example is that the language aspect (Pascal) is quite good (and syntactically sound, although I did not bother to try and compile it), however the business logic is not sound.  I would hope that a real expert would flag the prompt as bogus or nonsensical, not dish up something that doesn't make sense.  So the problem I see is that using correct grammar (or programming syntax) does not guarantee that the stated logic is sensible or even coherent.  A bit like politics.

[ccrause]

That code has now appeared in a programming forum, so it will be pounced upon by every LLM out there.

Obligatory xkcd: https://xkcd.com/978/

MarkMLl

Like a snake eating its own tail (which would result in a tragic end for a real snake).  Except in the AI business where the marketing hype would just intensify.

[Curt Carpenter]

https://www.valvesonline.com.au/references/steam-tables/

[Thaddy]

two others:
temperature and color
temperature and distance
This particular - the steam - equation is called the Antione equation and AI came up with the correct answer

Code: Pascal  [Select][+][-]

1. program CalculateSteamPressure;
2. uses SysUtils;
3.  
4. var
5.   T: Real;  // Temperature in Celsius
6.   P: Real;  // Pressure in mmHg
7.  
8. begin
9.   Write('Enter temperature in Celsius: ');
10.   ReadLn(T);
11.  
12.   // Calculate pressure using Antoine equation
13.   P := Exp(20.386 - 5132/T);
14.  
15.   WriteLn('Estimated steam pressure: ', P:0:2, ' mmHg');
16. end.

[gidesa]

What I was hoping to illustrate with the example is that the language aspect (Pascal) is quite good (and syntactically sound, although I did not bother to try and compile it), however the business logic is not sound.  I would hope that a real expert would flag the prompt as bogus or nonsensical, not dish up something that doesn't make sense.  So the problem I see is that using correct grammar (or programming syntax) does not guarantee that the stated logic is sensible or even coherent.  A bit like politics.

Indeed a GAI doesn't not "know" anything, as I wrote.  ChatGpt, or similar, isn't a "Delphi oracle" (in the sense of ancient Greece, not languages and databases :-)), that all knows and can answer to whatever question.
So your example doesn't illustrate nothing of new: the AI ... doesn't think, and doesn't knows everything! :-)
ChatGpt is the smoke and mirrors for quite silly people, a great marketing tool.
Nevertheless generative AI are an useful tool, if used in the correct mode.

[Curt Carpenter]

...
Nevertheless generative AI are an useful tool, if used in the correct mode.

If you hold today's primitive AI up as a mirror to our own human capacities, you might be tempted to say "Never the less, human knowledge and creativity are useful tools, if used in the correct mode"    The most profound impact of all this "AI stuff" may lie in the way it leads us to understand ourselves.  Who knows what the next fifty years will yield -- in terms of machine capabilities and our own self-awareness.

[Weiss]

This is not simply "garbage" but outright lie. I am still trying to comprehend. There must have been at least some Machiavellian intelligence, to invent this

Code: Pascal  [Select][+][-]

1. KelvinToBar := kelvin / 100000;  // Conversion factor from Kelvin to bar

[Curt Carpenter]

I'm not sure where the specific 100000 number came from, but as the steam table link points out "Steam and temperature posess a reliable realationship whereby if the pressure of the steam is known, it's temperature can be predicted (and vice versa)."  The appropriate response, I suppose, would be to put the question to ChatGPT   

[ccrause]

This is not simply "garbage" but outright lie. I am still trying to comprehend. There must have been at least some Machiavellian intelligence, to invent this

Code: Pascal  [Select][+][-]

1. KelvinToBar := kelvin / 100000;  // Conversion factor from Kelvin to bar

In the metric system the base unit for pressure is Pascal. The derived unit bar is 100000 Pa. Because the original prompt mentioned bar the AI model used the conversion from Pa to bar because it was (I assume) statistically likely to be relevant.