extract lines from bottom until regex matchAWK match string, not Regexextract last match from logfile till endRegex match in CLIawk + print lines from the first line until match wordJava process seems to always be bound to a single CPUHow to extract lines between same patterns from a fileRemove newline linefeed before a regex matchRegEx match + additional line removalRegex match fix string in hostnameExtract multiple lines if match
Random point on a sphere
Evidence that matrix multiplication cannot be done in O(n^2 poly(log(n))) time
Why is the T-1000 humanoid?
How can I fix a framing mistake so I can drywall?
How do I politely hint customers to leave my store, without pretending to need leave store myself?
Are there any instances of members of different Hogwarts houses coupling up and marrying each other?
Why should I always enable compiler warnings?
Using the pipe operator ("|") when executing system commands
How is the Team Scooby Doo funded?
Sol Ⅲ = Earth: What is the origin of this planetary naming scheme?
Can a new chain significantly improve the riding experience? If yes - what else can?
Contract Employer Keeps Asking for Small Things Without Pay
Which ping implementation is Cygwin using?
Do any aircraft carry boats?
Renewed US passport, did not receive expired US passport
ArcMap not displaying attribute table?
Do all humans have an identical nucleotide sequence for certain proteins, e.g haemoglobin?
Can I use ratchet straps to lift a dolly into a truck bed?
Exact Brexit date and consequences
Tracks in the snow
Why did they ever make smaller than full-frame sensors?
A Little Riddle
How could a imperial dynasty keep a loose collection of pirates, raiders, etc unified?
Can I say "I have encrypted something" if I hash something?
extract lines from bottom until regex match
AWK match string, not Regexextract last match from logfile till endRegex match in CLIawk + print lines from the first line until match wordJava process seems to always be bound to a single CPUHow to extract lines between same patterns from a fileRemove newline linefeed before a regex matchRegEx match + additional line removalRegex match fix string in hostnameExtract multiple lines if match
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have this output.
[root@linux ~]# cat /tmp/file.txt
virt-top time 11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.0 0.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.0 0.0 95:44:07 instance-00000372
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
I have tried sed/awk/grep and all possible way but not get close to desire result.
linux awk sed command-line regular-expression
add a comment |
I have this output.
[root@linux ~]# cat /tmp/file.txt
virt-top time 11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.0 0.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.0 0.0 95:44:07 instance-00000372
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
I have tried sed/awk/grep and all possible way but not get close to desire result.
linux awk sed command-line regular-expression
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
I think i am closedsed '1,/instance/p' /tmp/foo.txt
still its printing header line which isID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with| grep -v ID
but i believe there must be better way :)
– Satish
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?
– Philippos
12 hours ago
add a comment |
I have this output.
[root@linux ~]# cat /tmp/file.txt
virt-top time 11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.0 0.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.0 0.0 95:44:07 instance-00000372
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
I have tried sed/awk/grep and all possible way but not get close to desire result.
linux awk sed command-line regular-expression
I have this output.
[root@linux ~]# cat /tmp/file.txt
virt-top time 11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.0 0.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.0 0.0 95:44:07 instance-00000372
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
I have tried sed/awk/grep and all possible way but not get close to desire result.
linux awk sed command-line regular-expression
linux awk sed command-line regular-expression
edited 9 hours ago
Jeff Schaller♦
49.7k11 gold badges73 silver badges165 bronze badges
49.7k11 gold badges73 silver badges165 bronze badges
asked 12 hours ago
SatishSatish
7031 gold badge14 silver badges38 bronze badges
7031 gold badge14 silver badges38 bronze badges
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
I think i am closedsed '1,/instance/p' /tmp/foo.txt
still its printing header line which isID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with| grep -v ID
but i believe there must be better way :)
– Satish
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?
– Philippos
12 hours ago
add a comment |
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
I think i am closedsed '1,/instance/p' /tmp/foo.txt
still its printing header line which isID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with| grep -v ID
but i believe there must be better way :)
– Satish
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?
– Philippos
12 hours ago
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
I think i am closed
sed '1,/instance/p' /tmp/foo.txt
still its printing header line which is ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with | grep -v ID
but i believe there must be better way :)– Satish
12 hours ago
I think i am closed
sed '1,/instance/p' /tmp/foo.txt
still its printing header line which is ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with | grep -v ID
but i believe there must be better way :)– Satish
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)
grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?– Philippos
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)
grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?– Philippos
12 hours ago
add a comment |
5 Answers
5
active
oldest
votes
This feels a bit silly, but:
$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
GNU tac
reverses the file (many non-GNU systems have tail -r
instead), the sed
picks lines until the first that starts with virt-top
. You can add sed 1,2d
or tail -n +3
to remove the headers.
Or in awk:
$ awk '/^virt-top/ a = "" a = a $0 ORS END printf "%s", a' file.txt
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
It just collects all the lines to a variable, and clears that variable on a line starting with virt-top
.
If the file is very large, the tac
+sed
solution is bound to be faster since it only needs to read the tail end of the file while the awk
solution reads the full file from the top.
1
Only thetac
solution is practical iffile.txt
is very large and the match is (as expected) towards its end. Also, it could be better written astac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really liketac
first time i heard :D
– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
add a comment |
With ed
you can regex-search upward using ?pattern?
in place of the usual /pattern/
(which searches from above the current position). So for example:
$ printf '%sn' '?ID?+1,$p' q | ed -s file.txt
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
very simple and sweet, didn't thought aboutprintf
can run the show :)
– Satish
12 hours ago
2
ed
sucks with very large files.
– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
@Ed, I suppose it depends on the implementation, but at least GNUed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.
– Stéphane Chazelas
10 hours ago
add a comment |
If your input has a fixed number of blocks you could also do something like:
awk '/^virt-top/ && ++n == 2, 0' <your-file
To output the lines from the 2nd occurrence of virt-top
to the end of the file (0 meaning false, means the end of that first,last range is never found).
add a comment |
Getting the last record from the line that starts with the string virt-top
to the end using ed
(suitable for dealing with files of the size comparable with what you show, not several megabytes big):
$ printf '%sn' '?^virt-top?,$p' | ed -s file
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Or, just the lines after the last virt-top
line that contain the substring instance
:
$ printf '%sn' '?^virt-top?,$g/instance/p' | ed -s file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
That last ed
command, ?^virt-top?,$g/instance/p
, first designates a range of lines from the last line that starts with virt-top
(last, because ed
starts you off on the last line of the buffer and ?re?
searches backwards) to the end of the buffer ($
) and applies the command g/instance/p
to these lines. The g/re/p
command prints all lines within the range that matches the given regular expression (that's where grep
got its name from, by the way).
Using awk
:
$ awk '/^virt-top/ lines = "" /instance/ lines = (lines == "" ? $0 : lines ORS $0) END print lines ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
This saves lines of input in lines
whenever there's a line that contains the substring instance
. These lines are printed at the end. Whenever a line that starts with virt-top
is found, the saved lines are discarded.
Virtually the same thing with sed
, using the hold space to serve as the equivalent of the lines
variable in the awk
code:
$ sed -n '/^virt-top.*/ s///; x; d; ; /instance/H; $ x; s/n//; p; ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
add a comment |
Here's yet another method to deal with it:
$ sed -e '
/n/q
/virt-top/h;d;
H;$!d;g
s/n//;D
' file.txt
Results
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f540029%2fextract-lines-from-bottom-until-regex-match%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
This feels a bit silly, but:
$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
GNU tac
reverses the file (many non-GNU systems have tail -r
instead), the sed
picks lines until the first that starts with virt-top
. You can add sed 1,2d
or tail -n +3
to remove the headers.
Or in awk:
$ awk '/^virt-top/ a = "" a = a $0 ORS END printf "%s", a' file.txt
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
It just collects all the lines to a variable, and clears that variable on a line starting with virt-top
.
If the file is very large, the tac
+sed
solution is bound to be faster since it only needs to read the tail end of the file while the awk
solution reads the full file from the top.
1
Only thetac
solution is practical iffile.txt
is very large and the match is (as expected) towards its end. Also, it could be better written astac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really liketac
first time i heard :D
– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
add a comment |
This feels a bit silly, but:
$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
GNU tac
reverses the file (many non-GNU systems have tail -r
instead), the sed
picks lines until the first that starts with virt-top
. You can add sed 1,2d
or tail -n +3
to remove the headers.
Or in awk:
$ awk '/^virt-top/ a = "" a = a $0 ORS END printf "%s", a' file.txt
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
It just collects all the lines to a variable, and clears that variable on a line starting with virt-top
.
If the file is very large, the tac
+sed
solution is bound to be faster since it only needs to read the tail end of the file while the awk
solution reads the full file from the top.
1
Only thetac
solution is practical iffile.txt
is very large and the match is (as expected) towards its end. Also, it could be better written astac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really liketac
first time i heard :D
– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
add a comment |
This feels a bit silly, but:
$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
GNU tac
reverses the file (many non-GNU systems have tail -r
instead), the sed
picks lines until the first that starts with virt-top
. You can add sed 1,2d
or tail -n +3
to remove the headers.
Or in awk:
$ awk '/^virt-top/ a = "" a = a $0 ORS END printf "%s", a' file.txt
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
It just collects all the lines to a variable, and clears that variable on a line starting with virt-top
.
If the file is very large, the tac
+sed
solution is bound to be faster since it only needs to read the tail end of the file while the awk
solution reads the full file from the top.
This feels a bit silly, but:
$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
GNU tac
reverses the file (many non-GNU systems have tail -r
instead), the sed
picks lines until the first that starts with virt-top
. You can add sed 1,2d
or tail -n +3
to remove the headers.
Or in awk:
$ awk '/^virt-top/ a = "" a = a $0 ORS END printf "%s", a' file.txt
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
It just collects all the lines to a variable, and clears that variable on a line starting with virt-top
.
If the file is very large, the tac
+sed
solution is bound to be faster since it only needs to read the tail end of the file while the awk
solution reads the full file from the top.
edited 11 hours ago
Stéphane Chazelas
333k58 gold badges652 silver badges1023 bronze badges
333k58 gold badges652 silver badges1023 bronze badges
answered 12 hours ago
ilkkachuilkkachu
68.4k10 gold badges113 silver badges197 bronze badges
68.4k10 gold badges113 silver badges197 bronze badges
1
Only thetac
solution is practical iffile.txt
is very large and the match is (as expected) towards its end. Also, it could be better written astac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really liketac
first time i heard :D
– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
add a comment |
1
Only thetac
solution is practical iffile.txt
is very large and the match is (as expected) towards its end. Also, it could be better written astac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really liketac
first time i heard :D
– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
1
1
Only the
tac
solution is practical if file.txt
is very large and the match is (as expected) towards its end. Also, it could be better written as tac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
Only the
tac
solution is practical if file.txt
is very large and the match is (as expected) towards its end. Also, it could be better written as tac file.txt | sed '/^virt-top/q' | tac
– mosvy
11 hours ago
I really like
tac
first time i heard :D– Satish
11 hours ago
I really like
tac
first time i heard :D– Satish
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
@mosvy, excellent points, thank you!
– ilkkachu
11 hours ago
add a comment |
With ed
you can regex-search upward using ?pattern?
in place of the usual /pattern/
(which searches from above the current position). So for example:
$ printf '%sn' '?ID?+1,$p' q | ed -s file.txt
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
very simple and sweet, didn't thought aboutprintf
can run the show :)
– Satish
12 hours ago
2
ed
sucks with very large files.
– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
@Ed, I suppose it depends on the implementation, but at least GNUed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.
– Stéphane Chazelas
10 hours ago
add a comment |
With ed
you can regex-search upward using ?pattern?
in place of the usual /pattern/
(which searches from above the current position). So for example:
$ printf '%sn' '?ID?+1,$p' q | ed -s file.txt
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
very simple and sweet, didn't thought aboutprintf
can run the show :)
– Satish
12 hours ago
2
ed
sucks with very large files.
– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
@Ed, I suppose it depends on the implementation, but at least GNUed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.
– Stéphane Chazelas
10 hours ago
add a comment |
With ed
you can regex-search upward using ?pattern?
in place of the usual /pattern/
(which searches from above the current position). So for example:
$ printf '%sn' '?ID?+1,$p' q | ed -s file.txt
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
With ed
you can regex-search upward using ?pattern?
in place of the usual /pattern/
(which searches from above the current position). So for example:
$ printf '%sn' '?ID?+1,$p' q | ed -s file.txt
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
answered 12 hours ago
steeldriversteeldriver
43.1k5 gold badges56 silver badges95 bronze badges
43.1k5 gold badges56 silver badges95 bronze badges
very simple and sweet, didn't thought aboutprintf
can run the show :)
– Satish
12 hours ago
2
ed
sucks with very large files.
– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
@Ed, I suppose it depends on the implementation, but at least GNUed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.
– Stéphane Chazelas
10 hours ago
add a comment |
very simple and sweet, didn't thought aboutprintf
can run the show :)
– Satish
12 hours ago
2
ed
sucks with very large files.
– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
@Ed, I suppose it depends on the implementation, but at least GNUed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.
– Stéphane Chazelas
10 hours ago
very simple and sweet, didn't thought about
printf
can run the show :)– Satish
12 hours ago
very simple and sweet, didn't thought about
printf
can run the show :)– Satish
12 hours ago
2
2
ed
sucks with very large files.– mosvy
11 hours ago
ed
sucks with very large files.– mosvy
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
... because it reads the whole file into a buffer in memory before operating on it.
– Ed Morton
11 hours ago
2
2
@Ed, I suppose it depends on the implementation, but at least GNU
ed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.– Stéphane Chazelas
10 hours ago
@Ed, I suppose it depends on the implementation, but at least GNU
ed
, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though.– Stéphane Chazelas
10 hours ago
add a comment |
If your input has a fixed number of blocks you could also do something like:
awk '/^virt-top/ && ++n == 2, 0' <your-file
To output the lines from the 2nd occurrence of virt-top
to the end of the file (0 meaning false, means the end of that first,last range is never found).
add a comment |
If your input has a fixed number of blocks you could also do something like:
awk '/^virt-top/ && ++n == 2, 0' <your-file
To output the lines from the 2nd occurrence of virt-top
to the end of the file (0 meaning false, means the end of that first,last range is never found).
add a comment |
If your input has a fixed number of blocks you could also do something like:
awk '/^virt-top/ && ++n == 2, 0' <your-file
To output the lines from the 2nd occurrence of virt-top
to the end of the file (0 meaning false, means the end of that first,last range is never found).
If your input has a fixed number of blocks you could also do something like:
awk '/^virt-top/ && ++n == 2, 0' <your-file
To output the lines from the 2nd occurrence of virt-top
to the end of the file (0 meaning false, means the end of that first,last range is never found).
answered 10 hours ago
Stéphane ChazelasStéphane Chazelas
333k58 gold badges652 silver badges1023 bronze badges
333k58 gold badges652 silver badges1023 bronze badges
add a comment |
add a comment |
Getting the last record from the line that starts with the string virt-top
to the end using ed
(suitable for dealing with files of the size comparable with what you show, not several megabytes big):
$ printf '%sn' '?^virt-top?,$p' | ed -s file
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Or, just the lines after the last virt-top
line that contain the substring instance
:
$ printf '%sn' '?^virt-top?,$g/instance/p' | ed -s file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
That last ed
command, ?^virt-top?,$g/instance/p
, first designates a range of lines from the last line that starts with virt-top
(last, because ed
starts you off on the last line of the buffer and ?re?
searches backwards) to the end of the buffer ($
) and applies the command g/instance/p
to these lines. The g/re/p
command prints all lines within the range that matches the given regular expression (that's where grep
got its name from, by the way).
Using awk
:
$ awk '/^virt-top/ lines = "" /instance/ lines = (lines == "" ? $0 : lines ORS $0) END print lines ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
This saves lines of input in lines
whenever there's a line that contains the substring instance
. These lines are printed at the end. Whenever a line that starts with virt-top
is found, the saved lines are discarded.
Virtually the same thing with sed
, using the hold space to serve as the equivalent of the lines
variable in the awk
code:
$ sed -n '/^virt-top.*/ s///; x; d; ; /instance/H; $ x; s/n//; p; ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
add a comment |
Getting the last record from the line that starts with the string virt-top
to the end using ed
(suitable for dealing with files of the size comparable with what you show, not several megabytes big):
$ printf '%sn' '?^virt-top?,$p' | ed -s file
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Or, just the lines after the last virt-top
line that contain the substring instance
:
$ printf '%sn' '?^virt-top?,$g/instance/p' | ed -s file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
That last ed
command, ?^virt-top?,$g/instance/p
, first designates a range of lines from the last line that starts with virt-top
(last, because ed
starts you off on the last line of the buffer and ?re?
searches backwards) to the end of the buffer ($
) and applies the command g/instance/p
to these lines. The g/re/p
command prints all lines within the range that matches the given regular expression (that's where grep
got its name from, by the way).
Using awk
:
$ awk '/^virt-top/ lines = "" /instance/ lines = (lines == "" ? $0 : lines ORS $0) END print lines ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
This saves lines of input in lines
whenever there's a line that contains the substring instance
. These lines are printed at the end. Whenever a line that starts with virt-top
is found, the saved lines are discarded.
Virtually the same thing with sed
, using the hold space to serve as the equivalent of the lines
variable in the awk
code:
$ sed -n '/^virt-top.*/ s///; x; d; ; /instance/H; $ x; s/n//; p; ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
add a comment |
Getting the last record from the line that starts with the string virt-top
to the end using ed
(suitable for dealing with files of the size comparable with what you show, not several megabytes big):
$ printf '%sn' '?^virt-top?,$p' | ed -s file
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Or, just the lines after the last virt-top
line that contain the substring instance
:
$ printf '%sn' '?^virt-top?,$g/instance/p' | ed -s file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
That last ed
command, ?^virt-top?,$g/instance/p
, first designates a range of lines from the last line that starts with virt-top
(last, because ed
starts you off on the last line of the buffer and ?re?
searches backwards) to the end of the buffer ($
) and applies the command g/instance/p
to these lines. The g/re/p
command prints all lines within the range that matches the given regular expression (that's where grep
got its name from, by the way).
Using awk
:
$ awk '/^virt-top/ lines = "" /instance/ lines = (lines == "" ? $0 : lines ORS $0) END print lines ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
This saves lines of input in lines
whenever there's a line that contains the substring instance
. These lines are printed at the end. Whenever a line that starts with virt-top
is found, the saved lines are discarded.
Virtually the same thing with sed
, using the hold space to serve as the equivalent of the lines
variable in the awk
code:
$ sed -n '/^virt-top.*/ s///; x; d; ; /instance/H; $ x; s/n//; p; ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Getting the last record from the line that starts with the string virt-top
to the end using ed
(suitable for dealing with files of the size comparable with what you show, not several megabytes big):
$ printf '%sn' '?^virt-top?,$p' | ed -s file
virt-top time 11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Or, just the lines after the last virt-top
line that contain the substring instance
:
$ printf '%sn' '?^virt-top?,$g/instance/p' | ed -s file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
That last ed
command, ?^virt-top?,$g/instance/p
, first designates a range of lines from the last line that starts with virt-top
(last, because ed
starts you off on the last line of the buffer and ?re?
searches backwards) to the end of the buffer ($
) and applies the command g/instance/p
to these lines. The g/re/p
command prints all lines within the range that matches the given regular expression (that's where grep
got its name from, by the way).
Using awk
:
$ awk '/^virt-top/ lines = "" /instance/ lines = (lines == "" ? $0 : lines ORS $0) END print lines ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
This saves lines of input in lines
whenever there's a line that contains the substring instance
. These lines are printed at the end. Whenever a line that starts with virt-top
is found, the saved lines are discarded.
Virtually the same thing with sed
, using the hold space to serve as the equivalent of the lines
variable in the awk
code:
$ sed -n '/^virt-top.*/ s///; x; d; ; /instance/H; $ x; s/n//; p; ' file
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
edited 11 hours ago
answered 12 hours ago
Kusalananda♦Kusalananda
163k19 gold badges321 silver badges508 bronze badges
163k19 gold badges321 silver badges508 bronze badges
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
add a comment |
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
unix.stackexchange.com/questions/540029/…
– mosvy
11 hours ago
add a comment |
Here's yet another method to deal with it:
$ sed -e '
/n/q
/virt-top/h;d;
H;$!d;g
s/n//;D
' file.txt
Results
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
add a comment |
Here's yet another method to deal with it:
$ sed -e '
/n/q
/virt-top/h;d;
H;$!d;g
s/n//;D
' file.txt
Results
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
add a comment |
Here's yet another method to deal with it:
$ sed -e '
/n/q
/virt-top/h;d;
H;$!d;g
s/n//;D
' file.txt
Results
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
Here's yet another method to deal with it:
$ sed -e '
/n/q
/virt-top/h;d;
H;$!d;g
s/n//;D
' file.txt
Results
1 R 0 0 0 0 0.6 12.0 96:02:53 instance-0000036f
2 R 0 0 0 0 0.2 12.0 95:44:08 instance-00000372
answered 3 hours ago
Rakesh SharmaRakesh Sharma
1961 silver badge3 bronze badges
1961 silver badge3 bronze badges
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f540029%2fextract-lines-from-bottom-until-regex-match%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It can help us help you if you provide some of your "all possible ways" you tried
– Philippos
12 hours ago
I think i am closed
sed '1,/instance/p' /tmp/foo.txt
still its printing header line which isID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME
i can trim it down with| grep -v ID
but i believe there must be better way :)– Satish
12 hours ago
How about extracting all lines with non-zero CPU-Uage like (untested)
grep -vE -e virt-top -e RDRQ -e "([^ ]* *)60.0"
?– Philippos
12 hours ago