Nothing You See Here is Real - Artificial Intelligence

[schuler]

  Nothing You See Here is Real

Before I start typing, I refer to this video:
https://www.youtube.com/watch?v=qgx0yt6UUtA

My question to FPC/Lazarus lovers is: would we benefit from image generative neural networks? I'm asking to get an idea if I should or should not develop examples capable of image generation in plain/pure object pascal as everything else on CAI Project.

To whom doesn't know CAI, this is the latest project update:
https://forum.lazarus.freepascal.org/index.php/topic,39049.0.html

  Wish everyone happy coding 

[Eugene Loza]

Well, I wouldn't be so worried about neural network faking some images, because people already do
However, I'm trying to follow this progress, I'm extremely interested in neural network driven map generation like this: http://s3.amazonaws.com/digitaltrends-uploads-prod/2016/12/invisible_cities_patchwork_04.jpg
Just as I'm working at the moment at the "overworld" map generation for my game, and it'd be very convenient for me to draw biomes by hand and then just watch AI picking the right landscape and filling it with trees, rocks and other 3D objects However, still, I'm really unsure if neural network can provide both quality and speed (both in terms of run-time and development) to solve the task in comparison to "manual" algorithm based on kinda perlin noise generation + erosion.
Maybe, you have some ideas on the task? I'd be very grateful (so that I'd know for sure what direction should I take at the very beginning ).

[schuler]

Dear Eugene,
This is a good idea indeed.

Decided to share links on the same subject:
https://www.hallada.net/2016/01/06/neural-style.html
https://opendot.github.io/ml4a-invisible-cities/implementation/

[kupferstecher]

Wouldn't it be possible to compile using AI? With the source code as input (let's say pascal:) and the AI produces the according assembly.

I mean writing assembly requires a lot of thinking, but its all a quite mechanical thing. A step before a complete AI-compiler could be a optimisation unit with assembly as input.

(Sorry for OT~)

[schuler]

Hello kupferstecher,
In this message, I'll try to explore creativity.

Byte code is the way you have to program a X86 processor as example. Weights are the way to "code" a neural network. Therefore, byte code to X86 is equivalent to weights to a Neural Network (NN). The NN is your processor in this example.

In supervised learning, the NN learns the code itself given inputs and expected outputs. In the case that you want to stick with boolean algebra, you can easily have NN layers representing logic ports OR, AND, NOT, NAND,... It's quick to perceive that you can encode any boolean logic into NN. If you get an FPGA, you can make neurons to work in parallel...

Going back to your question, if you really want asm code as output, I would suggest you to use an interpreter associated with evolutionary computing to create the best asm code via reproduction and selection of the best fit. This approach might work for small problems such as "what is the best (or just a very good) dot product algorithm". I have an example with evolutionary algorithm to create magic squares.

[schuler]

Hello Eugene,
I have had success with making neural networks being able to convert images into higher resolution images as long as I have a proper training set. In my case, I have been using CIFAR-10. Soon, I'll have generative examples for everyone interested to download and try.

This is an example of NN that can increase the image resolution from 16x16 RGB to 32x32 RGB

NN.AddLayer( TNNetInput.Create(16,16,3) );
NN.AddLayer( TNNetDeMaxPool.Create(2) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,3,1,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(64,1,0,0) );
NN.AddLayer( TNNetConvolutionReLU.Create(3,1,0,0) );
NN.SetLearningRate(0.0001,0.9);

My question to you is: in regards to map generation, do you have a training set for me to give a go?

New neuronal layers have been added since the last post for inverse (generation) work:
* TNNetDeLocalConnect
* TNNetDeLocalConnectReLU
* TNNetDeconvolution
* TNNetDeconvolutionReLU
* TNNetDeMaxPool

  Wish everyone happy pascal coding

[Eugene Loza]

in regards to map generation, do you have a training set for me to give a go?

What should it look like? Some sort of images pairs (of equal resolution) where similar features are "overpainted" with a reference color? I think it's not allowed to use uncertain license/copyright images like satellite images or aerial photography (unless Public Domain by NASA), right?

[schuler]

@EUGENE
I still need to do some googling to be able to properly reply.

I have good news: I've just added the first image generative NN example in pure lazarus/free pascal. I trained the NN with pairs of images (16x16,32x32) in the hope that the NN would learn how to increase resolution given a lower resolution image. The code is located here:
https://sourceforge.net/p/cai/svncode/HEAD/tree/trunk/lazarus/IncreaseResolution/

When increasing resolution, the NN needs to have some understanding of what is doing. I added an example that transforms a 32x32 image into 64x64, 128x128 and 256x256. From 32x32 to 256x256, you can see that some pixels were transformed into a bumper or into a round eye. In my opinion, this is only possible if the NN has some understanding of the nature of the image that it's transforming. I intend to make a video about it soon.

In the image of the cat, both eyes look good in 256x256. In the image of the fire squad, look at created bumper and front lights in 256x256. I don't think that a this would be possible without the NN having some understanding about the underlying structure. Shadows look correct too.

256x256 images are far from perfect. But considering that a NN guessed pixels for the final image, result is impressive in my opinion.

Attached images aren't the best nor the worst images. I picked images that are in the middle of quality/artificial creativity.

This is a work in progress. There is plenty more to come.