Recent

Author Topic: Free AI to use in local  (Read 6925 times)

BSaidus

  • Hero Member
  • *****
  • Posts: 545
  • lazarus 1.8.4 Win8.1 / cross FreeBSD
Free AI to use in local
« on: March 28, 2024, 10:57:23 pm »
Hello.
I'm very newbe in AI.
I wonder if it exists such AI software that I can install in my local server, and feed it with informations (data) that I can exploit using lazarus application.

Thank you.
lazarus 1.8.4 Win8.1 / cross FreeBSD
dhukmucmur vernadh!

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Free AI to use in local
« Reply #1 on: March 29, 2024, 07:36:24 am »
That depends on what you understand to be AI. If you want something close to chatGPT that will always need an internet connection. If you want to explore AI in a different, local meaning, why do you not have a look and explore the possibilities that the work of our member schuler has done for his PH.d.
See a.o. this thread:
https://forum.lazarus.freepascal.org/index.php/topic,39049.0.html

By now it is much more than just about neural networks (which is a form of A.I.)
If you want something more or find it too difficult - it is difficult - I wrote an interface proof of concept for chatGPT but that now requires a payed subscription and can not run locally unless you run your own datacenters.
schuler's work requires a lot of training data for most solutions but is theoretically very, very sound and earned him his PH.d in artificial intelligence.
There is no such thing as "easy" regarding AI. but schuler's work is world class, written in freepascal and can be used locally. (if you put the effort in for the needed training data)
He is also very helpful and active on this forum.

A few experiments I have done myself with his software is simple voice recognition where the model can recognize the voice of all four members in the household and the two dogs and the cat too. Took me over a year... be prepared. I based that on his picture recognition example and used the wave forms of the voices in the same manner.
It can do this: recognize a voice and print the name of who is speaking miauwing or barking with about 92% accuracy. Nothing more...

for an introduction in fuzzy logic, which is much easier to understand and also a base of aI i wrote a zadeh logic a couple of years ago:
https://forum.lazarus.freepascal.org/index.php/topic,41144.msg295348.html#msg295348
If you can work with that, you will probably also understand schuler's code.
basically, you define ranges you want to explore, feed data in the zadeh logic, which can determine or narrow down these ranges for likelyhood, which ultimately can be narrowed down to ternary logic and in the ideal case to simple boolean logic. So zadeh enables you to narrow down your initial assumptions. Neural networks are based on similar, but different theory: essentially multi-dimensional. Both rely on training data, though, although Zadeh's logic can be used in embedded systems because it is very lightweight. note that the ternary logic I wrote for Rosetta code is a much improved version than the one on this forum, so use that if you want to reduce from the distributions calculated from the Zadeh logic.
« Last Edit: March 29, 2024, 09:19:15 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

BSaidus

  • Hero Member
  • *****
  • Posts: 545
  • lazarus 1.8.4 Win8.1 / cross FreeBSD
Re: Free AI to use in local
« Reply #2 on: March 29, 2024, 10:03:59 am »
Thanks @Thaddy.
What I want really is :
   Some opensource ( Free engine ) that I can download and install ( generally small executable ), when executing it download some templates and install them, after that I can use it offline ( no internet connection ). And have a possibility to feed this IA.
      [-----------------]                       [---------------------------------------]   
      [ AI .exe engine] <---------------[Download DB template & go offline]
      [-----------------]                       [---------------------------------------]
       ^            ^
       |             |
       |             |
       |             |
       |             | (user1) ... (user2)   ( will use AI in office without internet 
       |
       |(user0 feed AI with adequate information concerning the office business).

The documents, informations feeded by user0 can't go online.
The final thing is the possiblity for me to exploite this AI with FPC/lazarus program (Ex: program feeding AI and exploiting AI is writing FPC/LAZARUS).

Thanks you.
lazarus 1.8.4 Win8.1 / cross FreeBSD
dhukmucmur vernadh!

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Free AI to use in local
« Reply #3 on: March 29, 2024, 10:19:31 am »
well, it is not so much the code, but suffient enough data; TB's, gb's, not mb's.
Schuler has some links to public domain data sources.
chatGPT can be licensed to work off-line but requires your own datacenter! not just a network. And it is very expensive. What you obviously want to do is use something like chatGPT, general purpose AI, and that is in practice simply not possible or feasable to do locally and there are no open source solutions that you can run on a local network.
For example my simple voice recognition software using schuler's software required about 3-4gb data for only 7 voices to achieve 92% accuracy and then it can only couple it to the name, it does not understand words, just the sound of a voice using FFT to analyze the spectrum and feed that into schuler's neural network code.
The program itself is less than 2000 lines of code (excluding schuler's code). That is not the problem. The problem is the data needed and sufficient hardware capabillities to perform it in reasonable time.
OTOH for specific simple tasks, with e.g. Zadeh combinary logic you can do things like setting an iso value based on light intensity with increasing accuracy so that it operates on the collected data and learns from its use. That is less data intensive, but specific and that is not  what you mean.

Also note that one of the big three has the intention to open source its general purpose AI (I think it is facebook, meta) but as I said, it is all about the data and processing power. Less useful for you and me...
« Last Edit: March 29, 2024, 10:42:52 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

Dzandaa

  • Sr. Member
  • ****
  • Posts: 252
  • From C# to Lazarus
Re: Free AI to use in local
« Reply #4 on: March 29, 2024, 10:40:01 am »
Hi,

@BSaidus

First, you need to define what type of AI you want to use.

If you're thinking of creating a system comparable to ChatGPT, then you're going to need a big computer, with one or more NVIDIA power cards, lots of memory and hard disk, and a lot of free time to train your model, not to mention data collection.

Training a model from a dataset can take hours or even days.

Most AI programs are written in Python and call libraries written in C and C++, and are more suited to the Linux world

The only 2 exceptions, as far as I know, are schuler's CAI, written in Pascal and allowing graphics acceleration thanks to OpenCL and Darknet/Yolo (specialized in fast object detection)

So, I advise first, if you want to use the Pascal, to take a look at the excellent work of Joao-Paulo Schuler (CAI) and try his examples.

Now, it is also possible to create a GUI in Pascal for an existing AI program and call it from the GUI.

I did it for Yolo/Darknet.

But the first thing to do is to define what your AI is going to be used for.

B->

Dzandaa

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Free AI to use in local
« Reply #5 on: March 29, 2024, 10:55:29 am »
Hi,

@BSaidus

First, you need to define what type of AI you want to use.
Correct
Quote
If you're thinking of creating a system comparable to ChatGPT, then you're going to need a big computer, with one or more NVIDIA power cards, lots of memory and hard disk, and a lot of free time to train your model, not to mention data collection.
You greatly under-estimate the hardware requirements here.
Quote
Training a model from a dataset can take hours or even days.
years. but the better part of data collection can be automated.
Quote
Most AI programs are written in Python and call libraries written in C and C++, and are more suited to the Linux world
You mean python is used as an interface? I have done the same for chatGPT, remember.
That is not written in python... interfacing can be done in any lightweight language. True AI can not be written in python, it simply does not have the speed or capabilities.
Quote
The only 2 exceptions, as far as I know, are schuler's CAI, written in Pascal and allowing graphics acceleration thanks to OpenCL and Darknet/Yolo (specialized in fast object detection)
there are many more using opencl, I know of hundreds, many on github.
Quote
So, I advise first, if you want to use the Pascal, to take a look at the excellent work of Joao-Paulo Schuler (CAI) and try his examples.
I could not agree more AND you would be capable of achieving professional results, because that is professionaly written top notch code.
Quote
Now, it is also possible to create a GUI in Pascal for an existing AI program and call it from the GUI.

I did it for Yolo/Darknet.

But the first thing to do is to define what your AI is going to be used for.

B->
Nice!
« Last Edit: March 29, 2024, 10:57:55 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

cdbc

  • Hero Member
  • *****
  • Posts: 1085
    • http://www.cdbc.dk
Re: Free AI to use in local
« Reply #6 on: March 29, 2024, 10:57:46 am »
Hi
There's this https://github.com/PassByYou888/Z-AI1.4
You'll have to translate texts from chinese, but afaics it's written in Delphi...
Maybe, just maybe it could be something to work with...
The Author was on this forum once to announce it, one could probably contact him/her, I dunno...
Regards Benny
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Free AI to use in local
« Reply #7 on: March 29, 2024, 11:05:25 am »
BTW I am willing to share my code, but that is basically the same as one of schuler's examples. The data would be useless to others because it is trained based on data for my particular household.
OTOH I'll see if I can create some examples for fuzzy logic. That is much less data intensive and a good introduction to a form of AI that many people can start using right away.
scores:
cat                100% out of 1
me                100% out of 1
dogs               90% out of 2
female voices   88% out of 3

rates                92.0% from about 350 samples times 7.
Raw sample data at 48000 hz is about 2GB
4096 point fft's on the raw data makes up the rest.
The fft's are the data for the model, but it should be possible to train the model on the raw data at the cost of quite a lot more samples and longer processing time. have not tried that yet.
« Last Edit: March 29, 2024, 11:39:32 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

gidesa

  • Jr. Member
  • **
  • Posts: 69
Re: Free AI to use in local
« Reply #8 on: March 29, 2024, 10:14:48 pm »
Thanks @Thaddy.
What I want really is :
   Some opensource ( Free engine ) that I can download and install ( generally small executable ), when executing it download some templates and install them, after that I can use it offline ( no internet connection ). And have a possibility to feed this IA.
      [-----------------]                       [---------------------------------------]   
      [ AI .exe engine] <---------------[Download DB template & go offline]
      [-----------------]                       [---------------------------------------]
       ^            ^
       |             |
       |             |
       |             |
       |             | (user1) ... (user2)   ( will use AI in office without internet 
       |
       |(user0 feed AI with adequate information concerning the office business).

The documents, informations feeded by user0 can't go online.
The final thing is the possiblity for me to exploite this AI with FPC/lazarus program (Ex: program feeding AI and exploiting AI is writing FPC/LAZARUS).

Thanks you.

Hello, there is koboldcpp:  https://github.com/LostRuins/koboldcpp.
This is an AI  server, written in C++, for Windows and Linux. It needs a LLM (large language model) to load, in various formats as GGUF.
So you need only two files, in your local machine: koboldcpp executable (maybe with a config file), and the LLM.
Note that a GPU card with a large memory is a must, if you want good speed in responses.
You can connect with browser to the internal web server.
But, more useful in your case, you can connect with a simple web service, at localhost:5001/v1 , that use the OpenAI protocol plus some extensions.
Here: https://github.com/landgraf-dev/openai-delphi a package for FPC with examples.
Don't mind to OpenAI name, you can connect to Koboldcpp only changing IP and port in the examples.
In picture an example of Koboldcpp console with web service Json.

 

« Last Edit: March 29, 2024, 10:17:45 pm by gidesa »

schuler

  • Full Member
  • ***
  • Posts: 229
Re: Free AI to use in local
« Reply #9 on: April 02, 2024, 06:52:13 am »
@Thaddy,
Regarding
Quote
recognize a voice and print the name of who is speaking miauwing or barking with about 92% accuracy. Nothing more...
, I told my gf (she has a PhD in Biology) about this. From time to time, your experiment comes to our chat (we are the kind of people that enjoy talking about miauwing/barking problems). It's so interesting! If we think in terms of time series, applications are plenty. If we think in terms of a cross-species experiment, it's to cool!

@Thaddy, if you decide to publish your experiment in an open repository, it will be an honor to me adding a link to your project in the main CAI's readme.

schuler

  • Full Member
  • ***
  • Posts: 229
Re: Free AI to use in local
« Reply #10 on: April 02, 2024, 07:07:41 am »
An interesting site for GGUFs is: https://gpt4all.io/index.html . From the available GGUF files, the one I like the most is the mistral-7b. For its size, it looks good. You can run without video card with just 8GB of RAM. I know a guy to runs it with 64 cores.

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Free AI to use in local
« Reply #11 on: April 02, 2024, 10:47:55 am »
@Thaddy,
Regarding
Quote
recognize a voice and print the name of who is speaking miauwing or barking with about 92% accuracy. Nothing more...
, I told my gf (she has a PhD in Biology) about this. From time to time, your experiment comes to our chat (we are the kind of people that enjoy talking about miauwing/barking problems). It's so interesting! If we think in terms of time series, applications are plenty. If we think in terms of a cross-species experiment, it's to cool!

@Thaddy, if you decide to publish your experiment in an open repository, it will be an honor to me adding a link to your project in the main CAI's readme.
Of course, I will clean it up (the code) and send it to you for review. i will give you access to the data.
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

alpine

  • Hero Member
  • *****
  • Posts: 1064
Re: Free AI to use in local
« Reply #12 on: April 02, 2024, 11:40:50 am »
OTOH I'll see if I can create some examples for fuzzy logic. That is much less data intensive and a good introduction to a form of AI that many people can start using right away.
I am also very interested in this one and how it can be accomplished, especially the training phase.
In the past I've read many articles for fuzzy logic in IA (industrial automation) embedded, for MICR, OCRA, OCRB, defects classification, etc. but all they have pencil-and-paper initial analysis, not training.

I'ill greatly appreciate if you share how the fuzzy logic can be applied on the meaow problem.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

 

TinyPortal © 2005-2018