perl - Efficient way to continually read in data from a text file -
There is a script at our FTP endpoint that monitors the FIP log generated by our FTP daemon. What we're currently doing is that a Perl script essentially runs a tail-F on the file and sends every single row to a remote MySQL database, which has slightly different column content depending on the record type.
This table table for content of FTP user functions, along with the name / content of the tablet table, said the package; Downloads, deletes, and everything else logs the VSTPTP.
I consider it particularly bad, but I'm not sure what's better.
The goal of replacement is still to get the log file as soon as possible to file the contents in a database. I am doing something like creating a FIFO / Pipe file instead of the FIP Log file, I can read once in a while, to ensure that I do not read the same thing twice. Considering VSFTTPD, it will look good (I'm thinking that this will not happen, insights are welcome!).
The FTP daemon is VSFTPd, at least I'm definitely sure that the limit of their logging capacitance is: xfter style log, vsftpd style log, both, or no logging.
The question is, what is better than what we are already doing, if anything?
Honestly, I do not look very wrong with what you are doing now. The only real problem with tail -f
is that if your watch script ever dies, then it loses position (to solve the log-file with rotation There is a semi-difficult problem) CPAN is also a good module which protects you from the person and there is some good optimization available in it.
Use a FIFO as the log can work (try to unlink and reconfigure the logfile at any point where vsftpd does not do, But I see a big problem with it. If no one is studying from the other end of the FIFO (for example, if your script crashes, or never started), then after a short time, all who write to FIFA will begin to stop. And I have not tested this, but it is very likely that hanging the whole server from the logfile write block will not be a very beautiful scenario.
Comments
Post a Comment