Remove one or more fields, delimited by a “-”, at end of lineHow to remove trailing whitespace at the end of the line in given files (more than one)?Print certain fields of each line until a marker is encountered, then print whole lines till the end of filesed remove end of line for specific linesAWK remove one line?Comparing delimited fieldsConcatenate multiple fields separately based on one (key) columnPrinting more than one fieldAwk: remove first few fields from CSVawk remove lines with digits at endFormat Date fields in Pipe Delimited File
Characters in a conversation
What does it take to recreate microchips like 68000 and 6502 in their original process nodes nowadays?
Race condition interview question: Min and Max range of an integer
What are the limits on an impeached and not convicted president?
Narrow streets behind houses
I'm made of obsolete parts
Are 1 in 6 deaths in the USA due to lead exposure?
Is there any problem with students seeing faculty naked in university gym?
Is there any way to ward an area against Sending?
How does Donald Trump manage to remain so popular over a rather long period of time?
In the Star Trek: TNG continuity is cloning illegal?
Can/should you swim in zero G?
Is there a penalty for switching targets?
Remove one or more fields, delimited by a "-", at end of line
Is there any specific significance of inverse demand?
What cartridges were typically used together on Commodore 64 systems?
How to be productive while waiting for meetings to start, when managers are casual about being late
An example of a "simple poset" which does not belong to a convex polytope
The work of mathematicians outside their professional environment
Spectrometer vs Spectrometry vs Spectroscopy
Successive amplitudes in quantum mechanics
Meaning/Translation of title "The Light Fantastic" By Terry Pratchet
Minimum perfect squares needed to sum up to a target
What does the whole letter in Black Panther by Prince NJobu say?
Remove one or more fields, delimited by a “-”, at end of line
How to remove trailing whitespace at the end of the line in given files (more than one)?Print certain fields of each line until a marker is encountered, then print whole lines till the end of filesed remove end of line for specific linesAWK remove one line?Comparing delimited fieldsConcatenate multiple fields separately based on one (key) columnPrinting more than one fieldAwk: remove first few fields from CSVawk remove lines with digits at endFormat Date fields in Pipe Delimited File
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;
I am going to parse data googleapis.txt
bucket,abc-def-ghi-45gjd4-wwxis
bucket,dde-wwq-ooi-66ciow-po22q
instance,jkl-mno-1-zzz-68dkakw-oo9w8
disk,pqr-stu-10-kuy-l2oxapw-rp4lt
I expect the result like these below
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
I am thinking that i have to change -
to be a space and then run this command
cat googleapis.txt | awk '$NF="";sub(/[ t]+$/,"")1' | awk '$NF="";sub(/[ t]+$/,"")1'
I got that from this https://stackoverflow.com/a/27794421/8162936
After parsed, i will change the space to be a hypen
-
back.
Does anyone know the best practice or one-liner shell command to parse it ?
Thanks all
text-processing awk
add a comment
|
I am going to parse data googleapis.txt
bucket,abc-def-ghi-45gjd4-wwxis
bucket,dde-wwq-ooi-66ciow-po22q
instance,jkl-mno-1-zzz-68dkakw-oo9w8
disk,pqr-stu-10-kuy-l2oxapw-rp4lt
I expect the result like these below
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
I am thinking that i have to change -
to be a space and then run this command
cat googleapis.txt | awk '$NF="";sub(/[ t]+$/,"")1' | awk '$NF="";sub(/[ t]+$/,"")1'
I got that from this https://stackoverflow.com/a/27794421/8162936
After parsed, i will change the space to be a hypen
-
back.
Does anyone know the best practice or one-liner shell command to parse it ?
Thanks all
text-processing awk
1
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, settingNF
definitely is not portable).
– Kusalananda♦
14 hours ago
1
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago
add a comment
|
I am going to parse data googleapis.txt
bucket,abc-def-ghi-45gjd4-wwxis
bucket,dde-wwq-ooi-66ciow-po22q
instance,jkl-mno-1-zzz-68dkakw-oo9w8
disk,pqr-stu-10-kuy-l2oxapw-rp4lt
I expect the result like these below
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
I am thinking that i have to change -
to be a space and then run this command
cat googleapis.txt | awk '$NF="";sub(/[ t]+$/,"")1' | awk '$NF="";sub(/[ t]+$/,"")1'
I got that from this https://stackoverflow.com/a/27794421/8162936
After parsed, i will change the space to be a hypen
-
back.
Does anyone know the best practice or one-liner shell command to parse it ?
Thanks all
text-processing awk
I am going to parse data googleapis.txt
bucket,abc-def-ghi-45gjd4-wwxis
bucket,dde-wwq-ooi-66ciow-po22q
instance,jkl-mno-1-zzz-68dkakw-oo9w8
disk,pqr-stu-10-kuy-l2oxapw-rp4lt
I expect the result like these below
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
I am thinking that i have to change -
to be a space and then run this command
cat googleapis.txt | awk '$NF="";sub(/[ t]+$/,"")1' | awk '$NF="";sub(/[ t]+$/,"")1'
I got that from this https://stackoverflow.com/a/27794421/8162936
After parsed, i will change the space to be a hypen
-
back.
Does anyone know the best practice or one-liner shell command to parse it ?
Thanks all
text-processing awk
text-processing awk
edited 22 mins ago
cas
44.2k4 gold badges63 silver badges114 bronze badges
44.2k4 gold badges63 silver badges114 bronze badges
asked 14 hours ago
Nicky PuffNicky Puff
475 bronze badges
475 bronze badges
1
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, settingNF
definitely is not portable).
– Kusalananda♦
14 hours ago
1
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago
add a comment
|
1
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, settingNF
definitely is not portable).
– Kusalananda♦
14 hours ago
1
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago
1
1
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, setting
NF
definitely is not portable).– Kusalananda♦
14 hours ago
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, setting
NF
definitely is not portable).– Kusalananda♦
14 hours ago
1
1
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago
add a comment
|
4 Answers
4
active
oldest
votes
with sed
you can do:
sed -E 's/(-[^-]*)2$//' infile
match a pattern like -anything
twice (...)2
from end $
of every line and remove it.
add a comment
|
$ sed 's/-[[:alnum:]]*-[[:alnum:]]*$//' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This uses sed
to match the last two dash-delimited substrings on each line and remove them. [[:alnum:]]
will match any alphanumeric character.
You may shorten it down to
sed 's/(-[[:alnum:]]*)2$//' file
i.e., match and delete two sets of -[[:alnum:]]*
ath the end of each line.
With GNU awk
, you could also do
$ awk -F '-' 'BEGIN OFS=FS NF -= 2; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
but changing NF
like this is not portable, and should be avoided (there's no guarantee that it changes the current record). It would not work with BSD awk
, for example.
With standard awk
, without resorting to using sub()
(which would be to just mimic sed
), you would have to recreate the current record from the fields that you'd want to use (in our case, all but the last two dash-delimited fields):
$ awk -F '-' 'BEGIN OFS=FS nf = split($0,a) - 2; $0=""; for (i=1; i<=nf; ++i) $i = a[i]; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
add a comment
|
$ perl -F- -lane 'print join "-", @F[0..($#F-2)]' googleapis.txt
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This autosplits each input line into array @F
, using delimiter -
.
Then it prints an array slice of all but the last two fields, re-joined with -
characters.
add a comment
|
With rev
and cut
:
rev file | cut -d'-' -f3- | rev
Reverse the lines, cut
field 3 to the end of the line and reverse the text back again.
With grep
(and PCRE):
grep -Po '.*(?=(-[^-]*)2$)' file
-P
use perl-compatible regular expressions with a positive lookahead(?...)
containing two matches of-
followed by any non--
characters-o
print only matched parts
add a comment
|
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f544299%2fremove-one-or-more-fields-delimited-by-a-at-end-of-line%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
with sed
you can do:
sed -E 's/(-[^-]*)2$//' infile
match a pattern like -anything
twice (...)2
from end $
of every line and remove it.
add a comment
|
with sed
you can do:
sed -E 's/(-[^-]*)2$//' infile
match a pattern like -anything
twice (...)2
from end $
of every line and remove it.
add a comment
|
with sed
you can do:
sed -E 's/(-[^-]*)2$//' infile
match a pattern like -anything
twice (...)2
from end $
of every line and remove it.
with sed
you can do:
sed -E 's/(-[^-]*)2$//' infile
match a pattern like -anything
twice (...)2
from end $
of every line and remove it.
answered 14 hours ago
αғsнιηαғsнιη
19k11 gold badges35 silver badges72 bronze badges
19k11 gold badges35 silver badges72 bronze badges
add a comment
|
add a comment
|
$ sed 's/-[[:alnum:]]*-[[:alnum:]]*$//' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This uses sed
to match the last two dash-delimited substrings on each line and remove them. [[:alnum:]]
will match any alphanumeric character.
You may shorten it down to
sed 's/(-[[:alnum:]]*)2$//' file
i.e., match and delete two sets of -[[:alnum:]]*
ath the end of each line.
With GNU awk
, you could also do
$ awk -F '-' 'BEGIN OFS=FS NF -= 2; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
but changing NF
like this is not portable, and should be avoided (there's no guarantee that it changes the current record). It would not work with BSD awk
, for example.
With standard awk
, without resorting to using sub()
(which would be to just mimic sed
), you would have to recreate the current record from the fields that you'd want to use (in our case, all but the last two dash-delimited fields):
$ awk -F '-' 'BEGIN OFS=FS nf = split($0,a) - 2; $0=""; for (i=1; i<=nf; ++i) $i = a[i]; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
add a comment
|
$ sed 's/-[[:alnum:]]*-[[:alnum:]]*$//' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This uses sed
to match the last two dash-delimited substrings on each line and remove them. [[:alnum:]]
will match any alphanumeric character.
You may shorten it down to
sed 's/(-[[:alnum:]]*)2$//' file
i.e., match and delete two sets of -[[:alnum:]]*
ath the end of each line.
With GNU awk
, you could also do
$ awk -F '-' 'BEGIN OFS=FS NF -= 2; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
but changing NF
like this is not portable, and should be avoided (there's no guarantee that it changes the current record). It would not work with BSD awk
, for example.
With standard awk
, without resorting to using sub()
(which would be to just mimic sed
), you would have to recreate the current record from the fields that you'd want to use (in our case, all but the last two dash-delimited fields):
$ awk -F '-' 'BEGIN OFS=FS nf = split($0,a) - 2; $0=""; for (i=1; i<=nf; ++i) $i = a[i]; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
add a comment
|
$ sed 's/-[[:alnum:]]*-[[:alnum:]]*$//' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This uses sed
to match the last two dash-delimited substrings on each line and remove them. [[:alnum:]]
will match any alphanumeric character.
You may shorten it down to
sed 's/(-[[:alnum:]]*)2$//' file
i.e., match and delete two sets of -[[:alnum:]]*
ath the end of each line.
With GNU awk
, you could also do
$ awk -F '-' 'BEGIN OFS=FS NF -= 2; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
but changing NF
like this is not portable, and should be avoided (there's no guarantee that it changes the current record). It would not work with BSD awk
, for example.
With standard awk
, without resorting to using sub()
(which would be to just mimic sed
), you would have to recreate the current record from the fields that you'd want to use (in our case, all but the last two dash-delimited fields):
$ awk -F '-' 'BEGIN OFS=FS nf = split($0,a) - 2; $0=""; for (i=1; i<=nf; ++i) $i = a[i]; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
$ sed 's/-[[:alnum:]]*-[[:alnum:]]*$//' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This uses sed
to match the last two dash-delimited substrings on each line and remove them. [[:alnum:]]
will match any alphanumeric character.
You may shorten it down to
sed 's/(-[[:alnum:]]*)2$//' file
i.e., match and delete two sets of -[[:alnum:]]*
ath the end of each line.
With GNU awk
, you could also do
$ awk -F '-' 'BEGIN OFS=FS NF -= 2; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
but changing NF
like this is not portable, and should be avoided (there's no guarantee that it changes the current record). It would not work with BSD awk
, for example.
With standard awk
, without resorting to using sub()
(which would be to just mimic sed
), you would have to recreate the current record from the fields that you'd want to use (in our case, all but the last two dash-delimited fields):
$ awk -F '-' 'BEGIN OFS=FS nf = split($0,a) - 2; $0=""; for (i=1; i<=nf; ++i) $i = a[i]; print ' file
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
edited 13 hours ago
answered 14 hours ago
Kusalananda♦Kusalananda
166k20 gold badges324 silver badges518 bronze badges
166k20 gold badges324 silver badges518 bronze badges
add a comment
|
add a comment
|
$ perl -F- -lane 'print join "-", @F[0..($#F-2)]' googleapis.txt
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This autosplits each input line into array @F
, using delimiter -
.
Then it prints an array slice of all but the last two fields, re-joined with -
characters.
add a comment
|
$ perl -F- -lane 'print join "-", @F[0..($#F-2)]' googleapis.txt
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This autosplits each input line into array @F
, using delimiter -
.
Then it prints an array slice of all but the last two fields, re-joined with -
characters.
add a comment
|
$ perl -F- -lane 'print join "-", @F[0..($#F-2)]' googleapis.txt
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This autosplits each input line into array @F
, using delimiter -
.
Then it prints an array slice of all but the last two fields, re-joined with -
characters.
$ perl -F- -lane 'print join "-", @F[0..($#F-2)]' googleapis.txt
bucket,abc-def-ghi
bucket,dde-wwq-ooi
instance,jkl-mno-1-zzz
disk,pqr-stu-10-kuy
This autosplits each input line into array @F
, using delimiter -
.
Then it prints an array slice of all but the last two fields, re-joined with -
characters.
answered 13 hours ago
cascas
44.2k4 gold badges63 silver badges114 bronze badges
44.2k4 gold badges63 silver badges114 bronze badges
add a comment
|
add a comment
|
With rev
and cut
:
rev file | cut -d'-' -f3- | rev
Reverse the lines, cut
field 3 to the end of the line and reverse the text back again.
With grep
(and PCRE):
grep -Po '.*(?=(-[^-]*)2$)' file
-P
use perl-compatible regular expressions with a positive lookahead(?...)
containing two matches of-
followed by any non--
characters-o
print only matched parts
add a comment
|
With rev
and cut
:
rev file | cut -d'-' -f3- | rev
Reverse the lines, cut
field 3 to the end of the line and reverse the text back again.
With grep
(and PCRE):
grep -Po '.*(?=(-[^-]*)2$)' file
-P
use perl-compatible regular expressions with a positive lookahead(?...)
containing two matches of-
followed by any non--
characters-o
print only matched parts
add a comment
|
With rev
and cut
:
rev file | cut -d'-' -f3- | rev
Reverse the lines, cut
field 3 to the end of the line and reverse the text back again.
With grep
(and PCRE):
grep -Po '.*(?=(-[^-]*)2$)' file
-P
use perl-compatible regular expressions with a positive lookahead(?...)
containing two matches of-
followed by any non--
characters-o
print only matched parts
With rev
and cut
:
rev file | cut -d'-' -f3- | rev
Reverse the lines, cut
field 3 to the end of the line and reverse the text back again.
With grep
(and PCRE):
grep -Po '.*(?=(-[^-]*)2$)' file
-P
use perl-compatible regular expressions with a positive lookahead(?...)
containing two matches of-
followed by any non--
characters-o
print only matched parts
answered 14 hours ago
FreddyFreddy
8,3281 gold badge6 silver badges29 bronze badges
8,3281 gold badge6 silver badges29 bronze badges
add a comment
|
add a comment
|
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f544299%2fremove-one-or-more-fields-delimited-by-a-at-end-of-line%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
What is the logic telling us where to chop that second field off? The second field on the first two lines are chopped of after three dash-delimited substrings, while it's chopped of after four on the remaining lines. Your pipeline does not seem to have anything at all to do with your problem (also, setting
NF
definitely is not portable).– Kusalananda♦
14 hours ago
1
Ah, you want to remove the last two dash-delimited substrings?
– Kusalananda♦
14 hours ago
yes... i should change the title... sorry my bad
– Nicky Puff
14 hours ago