How do I search a file for a pattern, then extract part of the pattern or supply a default if the pattern does not exist?

up vote
1
down vote

favorite

I'm trying to write a script that searches through a pre-made list of running processes across a series of machines. I'm specifically looking for rsyslogd running on those devices, and attempting to find what configuration file they're using (if not using the default /etc/rsyslog.conf).

Right now I'm successfully searching through the machines with this for loop:

for root_file in $TARGET_DIR/RESULTS*/ps_results; do
 grep rsyslogd $root_file | awk 's = ""; for (i = 15; i <= NF; i++) s = s $i " "; print s'
done

And it'll return a list like

# get_configs $TARGET_DIR/
/usr/sbin/rsyslogd -n
/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
/usr/sbin/rsyslogd -n -f /usr/syslog.conf
...

But what I want is a list that shows the IP address of the machine that is being checked (which is in the * of the RESULTS* of the script as-is) followed by just the path to the configuration file, like so:

# get_configs $TARGET_DIR/
172.16.10.1 /etc/syslog.conf
172.16.10.2 /etc/syslog.conf
172.17.5.245 /usr/syslog.conf
...

I'll be taking this list and parsing through the files and finding any additional configuration files that they might link to with the $IncludeConfig directive, but I've got to get through cleaning up my list of files first.

The mental block I'm hitting is testing for the -f option following rsyslogd. Since rsyslogd doesn't require -f and it runs with the default /etc/rsyslog.conf, how do I handle testing for the option and extracting the path following it or supplying a default?

I considered using sed or cut to isolate ' -f /path/to/file' and return /etc/rsyslog.conf on an empty result, but I'm not managing to accomplish that.

asked 1 hour ago

BigDamnHero

153

add a commentÂ |Â

up vote
1
down vote

favorite

Right now I'm successfully searching through the machines with this for loop:

for root_file in $TARGET_DIR/RESULTS*/ps_results; do
 grep rsyslogd $root_file | awk 's = ""; for (i = 15; i <= NF; i++) s = s $i " "; print s'
done

And it'll return a list like

# get_configs $TARGET_DIR/
/usr/sbin/rsyslogd -n
/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
/usr/sbin/rsyslogd -n -f /usr/syslog.conf
...

# get_configs $TARGET_DIR/
172.16.10.1 /etc/syslog.conf
172.16.10.2 /etc/syslog.conf
172.17.5.245 /usr/syslog.conf
...

I considered using sed or cut to isolate ' -f /path/to/file' and return /etc/rsyslog.conf on an empty result, but I'm not managing to accomplish that.

asked 1 hour ago

BigDamnHero

153

add a commentÂ |Â

up vote
1
down vote

favorite

Right now I'm successfully searching through the machines with this for loop:

for root_file in $TARGET_DIR/RESULTS*/ps_results; do
 grep rsyslogd $root_file | awk 's = ""; for (i = 15; i <= NF; i++) s = s $i " "; print s'
done

And it'll return a list like

# get_configs $TARGET_DIR/
/usr/sbin/rsyslogd -n
/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
/usr/sbin/rsyslogd -n -f /usr/syslog.conf
...

# get_configs $TARGET_DIR/
172.16.10.1 /etc/syslog.conf
172.16.10.2 /etc/syslog.conf
172.17.5.245 /usr/syslog.conf
...

I considered using sed or cut to isolate ' -f /path/to/file' and return /etc/rsyslog.conf on an empty result, but I'm not managing to accomplish that.

asked 1 hour ago

BigDamnHero

153

Right now I'm successfully searching through the machines with this for loop:

for root_file in $TARGET_DIR/RESULTS*/ps_results; do
 grep rsyslogd $root_file | awk 's = ""; for (i = 15; i <= NF; i++) s = s $i " "; print s'
done

And it'll return a list like

# get_configs $TARGET_DIR/
/usr/sbin/rsyslogd -n
/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
/usr/sbin/rsyslogd -n -f /usr/syslog.conf
...

# get_configs $TARGET_DIR/
172.16.10.1 /etc/syslog.conf
172.16.10.2 /etc/syslog.conf
172.17.5.245 /usr/syslog.conf
...

I considered using sed or cut to isolate ' -f /path/to/file' and return /etc/rsyslog.conf on an empty result, but I'm not managing to accomplish that.

shell-script wildcards control-flow

asked 1 hour ago

BigDamnHero

153

asked 1 hour ago

BigDamnHero

153

asked 1 hour ago

BigDamnHero

153

asked 1 hour ago

BigDamnHero

153

asked 1 hour ago

BigDamnHero

153

add a commentÂ |Â

2 Answers
2

active

oldest

votes

up vote
2
down vote

accepted

Supposing we have the following input files

$ grep . */ps_results
RESULTS-172.16.10.1/ps_results:/usr/sbin/rsyslogd -n
RESULTS-172.16.10.2/ps_results:/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
RESULTS-172.17.5.245/ps_results:/usr/sbin/rsyslogd -n -f /usr/syslog.conf
$

Then how about

$ awk '/rsyslogd/gsub("^.*RESULTS-","",FILENAME);gsub("/ps_results","",FILENAME);b="/etc/rsyslog.conf";for(a=0;a++<NF;)if($a=="-f")b=$(a+1);printf "%-15s%sn",FILENAME,b' RESULTS*/ps_results
172.16.10.1 /etc/rsyslog.conf
172.16.10.2 /etc/rsyslog.conf
172.17.5.245 /usr/syslog.conf
$

Same code, annotated+formatted:

awk '/rsyslogd/ 
 # strip everything up to (and including) RESULTS- from the filename
 gsub("^.*RESULTS-","",FILENAME)
 # strip the /ps_results from the filename
 gsub("/ps_results","",FILENAME)
 # set the default rsyslog conf file
 b="/etc/rsyslog.conf"
 # look for a -f parameter: if found, grab conf file
 for(a=0;a++<NF;)
 if($a=="-f") 
 b=$(a+1)
 
 
 # print the ip addr and conf file
 printf "%-15s%sn",FILENAME,b
' RESULTS*/ps_results

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

1

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

add a commentÂ |Â

up vote
1
down vote

You could extend your existing awk like this (this is a contrived self-contained example):

awk ' s=$0; 
 if (match($s, / -f [^ ]+/)) 
 print substr($s, RSTART+4, RLENGTH-4) 
 
 else 
 print "/etc/syslog.conf" 
 ' < input

This asks if the string $s contains the regular expression "(space) -f (space) (one or more non-space characters)"; if so, then print the substring that matched the regular expression, offset over by 4 characters (and correspondingly shortened by 4 characters), to account for the (space) -f (space). If $s does not contain the above regular expression, print the default syslog.conf path.

The above code would go at the end of your existing awk script, replacing the print s portion (and, of course, not reassigning s=$0 -- that's just there so I had data to work with).

answered 47 mins ago

Jeff Schaller

32.4k849110

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f469281%2fhow-do-i-search-a-file-for-a-pattern-then-extract-part-of-the-pattern-or-supply%23new-answer', 'question_page');

);

Post as a guest

Name

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

up vote
2
down vote

accepted

Supposing we have the following input files

$ grep . */ps_results
RESULTS-172.16.10.1/ps_results:/usr/sbin/rsyslogd -n
RESULTS-172.16.10.2/ps_results:/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
RESULTS-172.17.5.245/ps_results:/usr/sbin/rsyslogd -n -f /usr/syslog.conf
$

Then how about

$ awk '/rsyslogd/gsub("^.*RESULTS-","",FILENAME);gsub("/ps_results","",FILENAME);b="/etc/rsyslog.conf";for(a=0;a++<NF;)if($a=="-f")b=$(a+1);printf "%-15s%sn",FILENAME,b' RESULTS*/ps_results
172.16.10.1 /etc/rsyslog.conf
172.16.10.2 /etc/rsyslog.conf
172.17.5.245 /usr/syslog.conf
$

Same code, annotated+formatted:

awk '/rsyslogd/ 
 # strip everything up to (and including) RESULTS- from the filename
 gsub("^.*RESULTS-","",FILENAME)
 # strip the /ps_results from the filename
 gsub("/ps_results","",FILENAME)
 # set the default rsyslog conf file
 b="/etc/rsyslog.conf"
 # look for a -f parameter: if found, grab conf file
 for(a=0;a++<NF;)
 if($a=="-f") 
 b=$(a+1)
 
 
 # print the ip addr and conf file
 printf "%-15s%sn",FILENAME,b
' RESULTS*/ps_results

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

1

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

add a commentÂ |Â

up vote
2
down vote

accepted

Supposing we have the following input files

$ grep . */ps_results
RESULTS-172.16.10.1/ps_results:/usr/sbin/rsyslogd -n
RESULTS-172.16.10.2/ps_results:/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
RESULTS-172.17.5.245/ps_results:/usr/sbin/rsyslogd -n -f /usr/syslog.conf
$

Then how about

$ awk '/rsyslogd/gsub("^.*RESULTS-","",FILENAME);gsub("/ps_results","",FILENAME);b="/etc/rsyslog.conf";for(a=0;a++<NF;)if($a=="-f")b=$(a+1);printf "%-15s%sn",FILENAME,b' RESULTS*/ps_results
172.16.10.1 /etc/rsyslog.conf
172.16.10.2 /etc/rsyslog.conf
172.17.5.245 /usr/syslog.conf
$

Same code, annotated+formatted:

awk '/rsyslogd/ 
 # strip everything up to (and including) RESULTS- from the filename
 gsub("^.*RESULTS-","",FILENAME)
 # strip the /ps_results from the filename
 gsub("/ps_results","",FILENAME)
 # set the default rsyslog conf file
 b="/etc/rsyslog.conf"
 # look for a -f parameter: if found, grab conf file
 for(a=0;a++<NF;)
 if($a=="-f") 
 b=$(a+1)
 
 
 # print the ip addr and conf file
 printf "%-15s%sn",FILENAME,b
' RESULTS*/ps_results

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

1

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

add a commentÂ |Â

up vote
2
down vote

accepted

Supposing we have the following input files

$ grep . */ps_results
RESULTS-172.16.10.1/ps_results:/usr/sbin/rsyslogd -n
RESULTS-172.16.10.2/ps_results:/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
RESULTS-172.17.5.245/ps_results:/usr/sbin/rsyslogd -n -f /usr/syslog.conf
$

Then how about

$ awk '/rsyslogd/gsub("^.*RESULTS-","",FILENAME);gsub("/ps_results","",FILENAME);b="/etc/rsyslog.conf";for(a=0;a++<NF;)if($a=="-f")b=$(a+1);printf "%-15s%sn",FILENAME,b' RESULTS*/ps_results
172.16.10.1 /etc/rsyslog.conf
172.16.10.2 /etc/rsyslog.conf
172.17.5.245 /usr/syslog.conf
$

Same code, annotated+formatted:

awk '/rsyslogd/ 
 # strip everything up to (and including) RESULTS- from the filename
 gsub("^.*RESULTS-","",FILENAME)
 # strip the /ps_results from the filename
 gsub("/ps_results","",FILENAME)
 # set the default rsyslog conf file
 b="/etc/rsyslog.conf"
 # look for a -f parameter: if found, grab conf file
 for(a=0;a++<NF;)
 if($a=="-f") 
 b=$(a+1)
 
 
 # print the ip addr and conf file
 printf "%-15s%sn",FILENAME,b
' RESULTS*/ps_results

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

Supposing we have the following input files

$ grep . */ps_results
RESULTS-172.16.10.1/ps_results:/usr/sbin/rsyslogd -n
RESULTS-172.16.10.2/ps_results:/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
RESULTS-172.17.5.245/ps_results:/usr/sbin/rsyslogd -n -f /usr/syslog.conf
$

Then how about

$ awk '/rsyslogd/gsub("^.*RESULTS-","",FILENAME);gsub("/ps_results","",FILENAME);b="/etc/rsyslog.conf";for(a=0;a++<NF;)if($a=="-f")b=$(a+1);printf "%-15s%sn",FILENAME,b' RESULTS*/ps_results
172.16.10.1 /etc/rsyslog.conf
172.16.10.2 /etc/rsyslog.conf
172.17.5.245 /usr/syslog.conf
$

Same code, annotated+formatted:

awk '/rsyslogd/ 
 # strip everything up to (and including) RESULTS- from the filename
 gsub("^.*RESULTS-","",FILENAME)
 # strip the /ps_results from the filename
 gsub("/ps_results","",FILENAME)
 # set the default rsyslog conf file
 b="/etc/rsyslog.conf"
 # look for a -f parameter: if found, grab conf file
 for(a=0;a++<NF;)
 if($a=="-f") 
 b=$(a+1)
 
 
 # print the ip addr and conf file
 printf "%-15s%sn",FILENAME,b
' RESULTS*/ps_results

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

edited 5 mins ago

answered 30 mins ago

steve

12.9k22149

answered 30 mins ago

steve

12.9k22149

answered 30 mins ago

steve

12.9k22149

1

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

add a commentÂ |Â

1

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

This was perfect. gsub seems like a useful tool for a lot of what I do. Thanks.
â€“Â BigDamnHero
5 mins ago

add a commentÂ |Â

up vote
1
down vote

You could extend your existing awk like this (this is a contrived self-contained example):

awk ' s=$0; 
 if (match($s, / -f [^ ]+/)) 
 print substr($s, RSTART+4, RLENGTH-4) 
 
 else 
 print "/etc/syslog.conf" 
 ' < input

The above code would go at the end of your existing awk script, replacing the print s portion (and, of course, not reassigning s=$0 -- that's just there so I had data to work with).

answered 47 mins ago

Jeff Schaller

32.4k849110

add a commentÂ |Â

up vote
1
down vote

You could extend your existing awk like this (this is a contrived self-contained example):

awk ' s=$0; 
 if (match($s, / -f [^ ]+/)) 
 print substr($s, RSTART+4, RLENGTH-4) 
 
 else 
 print "/etc/syslog.conf" 
 ' < input

The above code would go at the end of your existing awk script, replacing the print s portion (and, of course, not reassigning s=$0 -- that's just there so I had data to work with).

answered 47 mins ago

Jeff Schaller

32.4k849110

add a commentÂ |Â

up vote
1
down vote

You could extend your existing awk like this (this is a contrived self-contained example):

awk ' s=$0; 
 if (match($s, / -f [^ ]+/)) 
 print substr($s, RSTART+4, RLENGTH-4) 
 
 else 
 print "/etc/syslog.conf" 
 ' < input

The above code would go at the end of your existing awk script, replacing the print s portion (and, of course, not reassigning s=$0 -- that's just there so I had data to work with).

answered 47 mins ago

Jeff Schaller

32.4k849110

You could extend your existing awk like this (this is a contrived self-contained example):

awk ' s=$0; 
 if (match($s, / -f [^ ]+/)) 
 print substr($s, RSTART+4, RLENGTH-4) 
 
 else 
 print "/etc/syslog.conf" 
 ' < input

The above code would go at the end of your existing awk script, replacing the print s portion (and, of course, not reassigning s=$0 -- that's just there so I had data to work with).

answered 47 mins ago

Jeff Schaller

32.4k849110

answered 47 mins ago

Jeff Schaller

32.4k849110

answered 47 mins ago

Jeff Schaller

32.4k849110

answered 47 mins ago

Jeff Schaller

32.4k849110

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

Search This Blog

Iyfjky