- for batch operations, decide whether use binaries or shell scripts or both;
...
- this program that control everything can send EMAIL, SMS, Facebook, Whatsapp, Telegram, etc for a group when the bad things happen.
I am currently a Postgresql DBA. Most of the monitoring scripts I am about to inherit were done by a team member. For those just need to control that we don't spam out alerts when something goes down.
The coordination part are ETL jobs done by multiple teams/people.
I am hoping to present my, open sourced, program to the different people/teams and see if we can (long term) have sort of centralized dependency tree. Right now as I mentioned we have some system (I don't use it myself) for managing crons, but that is purely a distributed type of cron system. It doesn't know about dependencies.
I understand what you are saying about binary or script, but if I were to go to all these people/teams with these 2 options
* Fix your program so it doesn't spam / coordinate with other people teams more tightly so jobs run in the proper slot
* Use my script to avoid alerts from spamming and use my script so you can easily have your bash/python script run when it is supposed to
I think the second option is likely going to work better, if for no other reason that it will be less work for all those other teams/people.
Other than the scripts I am about to inherit most of the issues don't impact/involve me. But I thought I could create something to make the work easier for the other teams.
Some Linux log solutions such as syslog-ng can log directly into a database:
...
Talking about file handling in Pascal, have you read http://wiki.freepascal.org/File_Handling_In_Pascal ?
Not trying to log, but to have basically control data.. how many times has this shell script run this hour? For now to keep it simple for my first phase will create a single file per script. Later could use something like sqlite to centralize to a single file.
Imagine a cron like:
* */2 * * * monitoring_program1
* */2 * * * monitoring_program2
With my little program I just need to have the programs call my utility with something that likely will be like
if (runcontrol -h 1 monitoring_program1)
then
do work
else
exit
fi
Same for each monitoring program. I will then create one control file in ~/runcontrol/monitoring_program1 with how many times the program has run in the given hour (as an example). So, in this initial phase, I will just have one crontrol file per shell script.