sed on OS X - extract all text that is between square brackets

up vote
3
down vote

favorite

Given this stream:

[foo] 123 [bar]
[gar] dsa [har] 345
[uf] 88 [gc] 43 [br]

I want to process this with sed (or anything else) so the output would be:

foo bar
gar har
uf gc br

I tried cat myfile | sed -e 's/^.*[//;s/].*$//'

But it gives me only the last instance.

My real input is something like:

53f42d4 [the contacts are duplicated] Adding support in picking email verified users [https://trello.com/c/663]
3c454b0 [the contacts are duplicated] splitting contact by phone numbers and emails and changing contact model to contain only 1 email [https://trello.com/c/663]
0e63e5b [we should not let a user confirm his email if we have a user with this confirmed email already] better doc [https://trello.com/c/643]
02671b7 [we should not let a user confirm his email if we have a user with this confirmed email already] preventing updating email if already in used by other user [https://trello.com/c/643]

So I'd like to get for the first line:

the contacts are duplicated https://trello.com/c/663

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

add a commentÂ |Â

up vote
3
down vote

favorite

Given this stream:

[foo] 123 [bar]
[gar] dsa [har] 345
[uf] 88 [gc] 43 [br]

I want to process this with sed (or anything else) so the output would be:

foo bar
gar har
uf gc br

I tried cat myfile | sed -e 's/^.*[//;s/].*$//'

But it gives me only the last instance.

My real input is something like:

53f42d4 [the contacts are duplicated] Adding support in picking email verified users [https://trello.com/c/663]
3c454b0 [the contacts are duplicated] splitting contact by phone numbers and emails and changing contact model to contain only 1 email [https://trello.com/c/663]
0e63e5b [we should not let a user confirm his email if we have a user with this confirmed email already] better doc [https://trello.com/c/643]
02671b7 [we should not let a user confirm his email if we have a user with this confirmed email already] preventing updating email if already in used by other user [https://trello.com/c/643]

So I'd like to get for the first line:

the contacts are duplicated https://trello.com/c/663

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

add a commentÂ |Â

up vote
3
down vote

favorite

Given this stream:

[foo] 123 [bar]
[gar] dsa [har] 345
[uf] 88 [gc] 43 [br]

I want to process this with sed (or anything else) so the output would be:

foo bar
gar har
uf gc br

I tried cat myfile | sed -e 's/^.*[//;s/].*$//'

But it gives me only the last instance.

My real input is something like:

53f42d4 [the contacts are duplicated] Adding support in picking email verified users [https://trello.com/c/663]
3c454b0 [the contacts are duplicated] splitting contact by phone numbers and emails and changing contact model to contain only 1 email [https://trello.com/c/663]
0e63e5b [we should not let a user confirm his email if we have a user with this confirmed email already] better doc [https://trello.com/c/643]
02671b7 [we should not let a user confirm his email if we have a user with this confirmed email already] preventing updating email if already in used by other user [https://trello.com/c/643]

So I'd like to get for the first line:

the contacts are duplicated https://trello.com/c/663

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

Given this stream:

[foo] 123 [bar]
[gar] dsa [har] 345
[uf] 88 [gc] 43 [br]

I want to process this with sed (or anything else) so the output would be:

foo bar
gar har
uf gc br

I tried cat myfile | sed -e 's/^.*[//;s/].*$//'

But it gives me only the last instance.

My real input is something like:

53f42d4 [the contacts are duplicated] Adding support in picking email verified users [https://trello.com/c/663]
3c454b0 [the contacts are duplicated] splitting contact by phone numbers and emails and changing contact model to contain only 1 email [https://trello.com/c/663]
0e63e5b [we should not let a user confirm his email if we have a user with this confirmed email already] better doc [https://trello.com/c/643]
02671b7 [we should not let a user confirm his email if we have a user with this confirmed email already] preventing updating email if already in used by other user [https://trello.com/c/643]

So I'd like to get for the first line:

the contacts are duplicated https://trello.com/c/663

sed osx regular-expression

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

edited 18 mins ago

Peter Mortensen

84358

edited 18 mins ago

Peter Mortensen

84358

edited 18 mins ago

Peter Mortensen

84358

asked 13 hours ago

YardenST

1216

New contributor

asked 13 hours ago

YardenST

1216

asked 13 hours ago

YardenST

1216

New contributor

YardenST is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

add a commentÂ |Â

5 Answers
5

active

oldest

votes

up vote
3
down vote

accepted

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '' 'for (i=2; i<=NF; i+=2) printf "%s ", $i; print ""' file

With sed, I'd write

sed -E 's/(^|])[^*($|[)/ /g' file

answered 12 hours ago

glenn jackman

49.4k467106

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

add a commentÂ |Â

up vote
4
down vote

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^*[([^]]*)][^*/1 /g' file
foo bar
gar har
uf gc br

Description:

sed ' # start a sed script
 s/ # start a substitute command
 [^* # match all leading characters (except [)
 [ # match an explicit [
 ([^]]*) # capture text inside brackets.
 ] # match the closing ]
 [^* # match trailing text (if any).
 /1 / # replace everything matched by the captured text.
 g # repeat for all the line.
 ' file # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^*[([^]]*)][^*/1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '[K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk 'print gensub(/[([^]]*)][^*/,"\1 ","g")' file

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

add a commentÂ |Â

up vote
3
down vote

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant RE processors.

Since Perl should be available on macos by default, perhaps this is viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=[)(.+?)(?=]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5

answered 12 hours ago

wwerner

1312

New contributor

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

add a commentÂ |Â

up vote
2
down vote

This seems to work:

$ sed -E 's/ [^[a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d ''
foo bar
gar har
uf gc br

answered 13 hours ago

DopeGhoti

41.7k55180

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

add a commentÂ |Â

up vote
1
down vote

sed -n '/[/ s-[^*--; s-[([^]]*)][^*- 1-g; s- --p '

The algorithm is:

ignore lines that do not contain brackets,

remove text before first bracket,

replace pairs of backets and optional trailing text by spaces, leaving the text inside the brackets,

and remove the initial space, leaving only spaces in between.

answered 12 hours ago

Juergen

787

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

YardenST is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f476352%2fsed-on-os-x-extract-all-text-that-is-between-square-brackets%23new-answer', 'question_page');

);

Post as a guest

Name

5 Answers
5

active

oldest

votes

5 Answers
5

active

oldest

votes

up vote
3
down vote

accepted

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '' 'for (i=2; i<=NF; i+=2) printf "%s ", $i; print ""' file

With sed, I'd write

sed -E 's/(^|])[^*($|[)/ /g' file

answered 12 hours ago

glenn jackman

49.4k467106

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

add a commentÂ |Â

up vote
3
down vote

accepted

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '' 'for (i=2; i<=NF; i+=2) printf "%s ", $i; print ""' file

With sed, I'd write

sed -E 's/(^|])[^*($|[)/ /g' file

answered 12 hours ago

glenn jackman

49.4k467106

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

add a commentÂ |Â

up vote
3
down vote

accepted

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '' 'for (i=2; i<=NF; i+=2) printf "%s ", $i; print ""' file

With sed, I'd write

sed -E 's/(^|])[^*($|[)/ /g' file

answered 12 hours ago

glenn jackman

49.4k467106

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '' 'for (i=2; i<=NF; i+=2) printf "%s ", $i; print ""' file

With sed, I'd write

sed -E 's/(^|])[^*($|[)/ /g' file

answered 12 hours ago

glenn jackman

49.4k467106

answered 12 hours ago

glenn jackman

49.4k467106

answered 12 hours ago

glenn jackman

49.4k467106

answered 12 hours ago

glenn jackman

49.4k467106

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

add a commentÂ |Â

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

The sed solution adds leading and trailing spaces (per line).
â€“Â Isaac
6 hours ago

add a commentÂ |Â

up vote
4
down vote

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^*[([^]]*)][^*/1 /g' file
foo bar
gar har
uf gc br

Description:

sed ' # start a sed script
 s/ # start a substitute command
 [^* # match all leading characters (except [)
 [ # match an explicit [
 ([^]]*) # capture text inside brackets.
 ] # match the closing ]
 [^* # match trailing text (if any).
 /1 / # replace everything matched by the captured text.
 g # repeat for all the line.
 ' file # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^*[([^]]*)][^*/1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '[K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk 'print gensub(/[([^]]*)][^*/,"\1 ","g")' file

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

add a commentÂ |Â

up vote
4
down vote

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^*[([^]]*)][^*/1 /g' file
foo bar
gar har
uf gc br

Description:

sed ' # start a sed script
 s/ # start a substitute command
 [^* # match all leading characters (except [)
 [ # match an explicit [
 ([^]]*) # capture text inside brackets.
 ] # match the closing ]
 [^* # match trailing text (if any).
 /1 / # replace everything matched by the captured text.
 g # repeat for all the line.
 ' file # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^*[([^]]*)][^*/1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '[K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk 'print gensub(/[([^]]*)][^*/,"\1 ","g")' file

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

add a commentÂ |Â

up vote
4
down vote

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^*[([^]]*)][^*/1 /g' file
foo bar
gar har
uf gc br

Description:

sed ' # start a sed script
 s/ # start a substitute command
 [^* # match all leading characters (except [)
 [ # match an explicit [
 ([^]]*) # capture text inside brackets.
 ] # match the closing ]
 [^* # match trailing text (if any).
 /1 / # replace everything matched by the captured text.
 g # repeat for all the line.
 ' file # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^*[([^]]*)][^*/1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '[K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk 'print gensub(/[([^]]*)][^*/,"\1 ","g")' file

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^*[([^]]*)][^*/1 /g' file
foo bar
gar har
uf gc br

Description:

sed ' # start a sed script
 s/ # start a substitute command
 [^* # match all leading characters (except [)
 [ # match an explicit [
 ([^]]*) # capture text inside brackets.
 ] # match the closing ]
 [^* # match trailing text (if any).
 /1 / # replace everything matched by the captured text.
 g # repeat for all the line.
 ' file # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^*[([^]]*)][^*/1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '[K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk 'print gensub(/[([^]]*)][^*/,"\1 ","g")' file

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

edited 4 hours ago

answered 12 hours ago

Isaac

8,54011140

answered 12 hours ago

Isaac

8,54011140

answered 12 hours ago

Isaac

8,54011140

add a commentÂ |Â

up vote
3
down vote

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant RE processors.

Since Perl should be available on macos by default, perhaps this is viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=[)(.+?)(?=]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5

answered 12 hours ago

wwerner

1312

New contributor

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

add a commentÂ |Â

up vote
3
down vote

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant RE processors.

Since Perl should be available on macos by default, perhaps this is viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=[)(.+?)(?=]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5

answered 12 hours ago

wwerner

1312

New contributor

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

add a commentÂ |Â

up vote
3
down vote

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant RE processors.

Since Perl should be available on macos by default, perhaps this is viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=[)(.+?)(?=]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5

answered 12 hours ago

wwerner

1312

New contributor

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant RE processors.

Since Perl should be available on macos by default, perhaps this is viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=[)(.+?)(?=]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5

answered 12 hours ago

wwerner

1312

New contributor

answered 12 hours ago

wwerner

1312

New contributor

answered 12 hours ago

wwerner

1312

answered 12 hours ago

wwerner

1312

New contributor

wwerner is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

add a commentÂ |Â

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

Nice. An alternative: perl -nE 'say join " ", (/ (?<=[) .*? (?=]) /xg)' file
â€“Â glenn jackman
12 hours ago

add a commentÂ |Â

up vote
2
down vote

This seems to work:

$ sed -E 's/ [^[a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d ''
foo bar
gar har
uf gc br

answered 13 hours ago

DopeGhoti

41.7k55180

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

add a commentÂ |Â

up vote
2
down vote

This seems to work:

$ sed -E 's/ [^[a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d ''
foo bar
gar har
uf gc br

answered 13 hours ago

DopeGhoti

41.7k55180

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

add a commentÂ |Â

up vote
2
down vote

This seems to work:

$ sed -E 's/ [^[a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d ''
foo bar
gar har
uf gc br

answered 13 hours ago

DopeGhoti

41.7k55180

This seems to work:

$ sed -E 's/ [^[a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d ''
foo bar
gar har
uf gc br

answered 13 hours ago

DopeGhoti

41.7k55180

answered 13 hours ago

DopeGhoti

41.7k55180

answered 13 hours ago

DopeGhoti

41.7k55180

answered 13 hours ago

DopeGhoti

41.7k55180

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

add a commentÂ |Â

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

thanks, but I get sed: input: No such file or directory
â€“Â YardenST
13 hours ago

@YardenST "input" was a placeholder for an input file you would need to supply yours there or omit that and pipe in the source.
â€“Â Chris Stratton
13 hours ago

Thanks got it :) but it does not seem to work with my input. My input is coming from git log commit1..commit2 --oneline. Ive updated the question with an example
â€“Â YardenST
13 hours ago

add a commentÂ |Â

up vote
1
down vote

answered 12 hours ago

Juergen

787

add a commentÂ |Â

up vote
1
down vote

answered 12 hours ago

Juergen

787

add a commentÂ |Â

up vote
1
down vote

answered 12 hours ago

Juergen

787

answered 12 hours ago

Juergen

787

answered 12 hours ago

Juergen

787

answered 12 hours ago

Juergen

787

answered 12 hours ago

Juergen

787

add a commentÂ |Â

YardenST is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

YardenST is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Post as a guest

Name

Search This Blog

Iyfjky