Discussion:
Resetting rsync failed jobs
(too old to reply)
Eric Seager
2006-08-29 18:12:53 UTC
Permalink
I am having issues with rsync and have been working on restarting failed jobs. Basically anything in the log that has a SendBuffer error will be need to restarted. Does anyone have anything that does anything near what I am looking for? Any help would be greatly appreciated. Thanks in advance.
Eric Seager
2006-08-30 19:51:15 UTC
Permalink
The rsyncd.log contains the following info:
2006/08/24 21:04:02 [21] rsync to CRE/CRE/ from unknown (192.168.135.3)

This line is very important becasue it contains the PID and the site. The pid being 21 the site being CRE.

So far this is what I have for a script to find the jobs in the rsyncd.log.

Sort1.pl
#Created By Eric Seager
#This script will parse lines from a log file and output them to a text file
#Please delete the output.txt after every use
#08/22/2006

my $outfile = 'output.txt';

open(LOG,"rsyncd.log") or die "Unable to open logfile:$!\n";
open(OF,'>>'.$outfile) or die("Unable to open $outfile: $!\n");

while(<LOG>)
{
print OF if /\b] wrote\b/i;
print OF if /\b] SendBuffer\b/i;
}
close(LOG);
close(OF);

It creates and output.txt that contains in the following format:
2006/08/24 20:34:34 [4] wrote 2467089 bytes read 15054665 bytes total size 8099905083
2006/08/24 20:35:10 [11] wrote 594641 bytes read 231504 bytes total size 3017650046
2006/08/24 20:36:52 [9] wrote 1471567 bytes read 11921312 bytes total size 5861921629
2006/08/24 20:39:23 [5] wrote 1363729 bytes read 22370958 bytes total size 3493489374
2006/08/24 20:48:39 [14] wrote 702457 bytes read 5296827 bytes total size 1171217448
2006/08/24 20:51:54 [2] wrote 13891613 bytes read 132523843 bytes total size 83943885995
2006/08/24 21:29:40 [24] wrote 579526 bytes read 9126936 bytes total size 768947940
2006/08/24 21:32:07 [6] wrote 5447648 bytes read 34235606 bytes total size 14127499277
2006/08/24 21:32:27 [13] wrote 3707529 bytes read 8144431 bytes total size 13264807322
2006/08/24 21:47:19 [21] SendBuffer: Error WSASend fd=117 nRet=-1 err=10053
2006/08/24 21:47:24 [19] wrote 336065 bytes read 26039613 bytes total size 9768815996

What I need to happen is everytime a SendBuffer Error occurs it back references the PID to find the Site from the string at the beginning of the post. It will then use the site ID to fire retry-<SITEID.PL. Is there anyone out there that has something close to this I can doctor up most scripts to work. Thanks in advance.
Brian Mantler
2006-09-05 11:05:28 UTC
Permalink
HI Eric,

Check out the cpr-backup project. There is a script can be modified to
re-run an rsync on a failure.

I have modified the error checking on my version of the cpr-backup script
to look something like this below. All credit should go to the devolper,
Iain McLaren.

Hope this helps


sub check_error_log {
# checks the log file for errors
# returns 0 if no errors are found


# Read in the rsync log for processing, in 'slurp' mode
open IN, $_[0] or return 1;

$/=undef;
# Read the file
$log_to_parse=<IN>;
close IN;

# Check for failure/error conditions
foreach $f (@failures)
{

return 2 if ($log_to_parse =~ m/$f/i);
}




# Check for the word 'error' Note a file named 'error' will trigger a
false positive

#if ($log_to_parse =~ m/\[.*\].*\ error/i)
if ($log_to_parse =~ m/\[.*1.*\].*\ error/i)
{
return 3;
}

# now check explicitly for success
# I want to see a [, ], and the word speedup all on one line

if ($log_to_parse =~ m/\[.*\].*speedup/i)
{
return 0;
}
else
{

return 4;
}
}
Post by Eric Seager
2006/08/24 21:04:02 [21] rsync to CRE/CRE/ from unknown
(192.168.135.3)
This line is very important becasue it contains the PID and the site.
The pid being 21 the site being CRE.
So far this is what I have for a script to find the jobs in the rsyncd.log.
Sort1.pl
#Created By Eric Seager
#This script will parse lines from a log file and output them to a
text file #Please delete the output.txt after every use
#08/22/2006
my $outfile = 'output.txt';
open(LOG,"rsyncd.log") or die "Unable to open logfile:$!\n";
open(OF,'>>'.$outfile) or die("Unable to open $outfile: $!\n");
while(<LOG>)
{
print OF if /\b] wrote\b/i;
print OF if /\b] SendBuffer\b/i;
}
close(LOG);
close(OF);
2006/08/24 20:34:34 [4] wrote 2467089 bytes read 15054665 bytes
total size 8099905083 2006/08/24 20:35:10 [11] wrote 594641 bytes
1471567 bytes read 11921312 bytes total size 5861921629 2006/08/24
20:39:23 [5] wrote 1363729 bytes read 22370958 bytes total size
3493489374 2006/08/24 20:48:39 [14] wrote 702457 bytes read 5296827
bytes total size 1171217448 2006/08/24 20:51:54 [2] wrote 13891613
bytes read 132523843 bytes total size 83943885995 2006/08/24
21:29:40 [24] wrote 579526 bytes read 9126936 bytes total size
768947940 2006/08/24 21:32:07 [6] wrote 5447648 bytes read 34235606
bytes total size 14127499277 2006/08/24 21:32:27 [13] wrote 3707529
bytes read 8144431 bytes total size 13264807322 2006/08/24 21:47:19
[21] SendBuffer: Error WSASend fd=117 nRet=-1 err=10053 2006/08/24
21:47:24 [19] wrote 336065 bytes read 26039613 bytes total size
9768815996
What I need to happen is everytime a SendBuffer Error occurs it back
references the PID to find the Site from the string at the beginning
of the post. It will then use the site ID to fire retry-<SITEID.PL. Is
there anyone out there that has something close to this I can doctor
up most scripts to work. Thanks in advance.
Eric Seager
2006-09-07 21:25:19 UTC
Permalink
That is pretty kewl how do I use the script? NLM? NCF? Thanks for the help so far. Let me know.
Brian Mantler
2006-09-11 11:57:04 UTC
Permalink
Hi,

cpr-backup is a perl script you can run to control your rsync's.

Go to http://forge.novell.com/modules/xfmod/project/?cpr-backup and
download the perl script found there. You can then modify the
check_error_log routine to suite our needs.


Hope this helps.
Post by Eric Seager
That is pretty kewl how do I use the script? NLM? NCF? Thanks for the
help so far. Let me know.
Loading...