Text file - replace a pattern with part of the previous line? -SOLVED

Question

Hello, I'm stuck with sed/awk/grep...
So I have a file with lines like this:
Nice bunch of words <STUFF> <STUFF> <IMPORTANT_DELIMITER_TYPE_1> <STUFF> <IMPORTANT_DELIMITER_TYPE_2> <STUFF>
Even Nicer bunch of Words <STUFF> <IMPORTANT_DELIMITER_TYPE_1> <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF><IMPORTANT_DELIMITER_TYPE_1><STUFF>
Then, I want to move the "important delimiters" to new lines (might be better not to do this in fact...)
Nice bunch of words <STUFF> <STUFF>
<IMPORTANT_DELIMITER_TYPE_1> <STUFF>
<IMPORTANT_DELIMITER_TYPE_2> <STUFF>
Even Nicer bunch of Words <STUFF>
<IMPORTANT_DELIMITER_TYPE_1> <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF>
<IMPORTANT_DELIMITER_TYPE_1><STUFF>
And finally, I want to replace the important delimiters with the content of the line they came from originally, up to the first angle bracket:
Nice bunch of words <STUFF> <STUFF>
Nice bunch of words <STUFF>
Nice bunch of words <STUFF>
Even Nicer bunch of Words <STUFF>
Even Nicer bunch of Words <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words<STUFF>
How can I accomplish this using absolutely anything at all that doesn't involve too much manual effort (the file is about 30,000 lines of this stuff)
Thanks !
Last edited by darkbeanies (2013-09-01 19:18:29)

Answer 1

"<STUFF>" is not irrelevant. In your original example
Nice bunch of words <STUFF> <STUFF> <IMPORTANT_DELIMITER_TYPE_1> <STUFF> <IMPORTANT_DELIMITER_TYPE_2> <STUFF>
Even Nicer bunch of Words <STUFF> <IMPORTANT_DELIMITER_TYPE_1> <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF><IMPORTANT_DELIMITER_TYPE_1><STUFF>
you want to break the lines along the delimiters:
Nice bunch of words <STUFF> <STUFF>
<IMPORTANT_DELIMITER_TYPE_1> <STUFF>
<IMPORTANT_DELIMITER_TYPE_2> <STUFF>
Even Nicer bunch of Words <STUFF>
<IMPORTANT_DELIMITER_TYPE_1> <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF>
<IMPORTANT_DELIMITER_TYPE_1><STUFF>
and then you say that you want to replace the delimiters with the contents before the first angle bracket:
Nice bunch of words <STUFF> <STUFF>
Nice bunch of words <STUFF>
Nice bunch of words <STUFF>
Even Nicer bunch of Words <STUFF>
Even Nicer bunch of Words <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words <STUFF> <STUFF> <STUFF>
Wonderful bunch of Words<STUFF>
but you have clearly made a distinction between the "nice bunch of words" and "<STUFF>", otherwise the output would have been
Nice bunch of words <STUFF> <STUFF>
Nice bunch of words <STUFF> <STUFF> <STUFF>
So, do you want everything up to  the first delimiter, or do you want everything up to <STUFF> in the replacement? If you only want the "nicer words" then you need some way to distinguish between them and "stuff" programmatically.
edit
Here's a trivial script that will split the lines along the delimiters and replace them with the contents of the line before the first delimiter:
#!/usr/bin/env python3
import re
import sys
def main(args=None):
for line in sys.stdin:
# Trim trailing newline.
line = line.rstrip('\n')
# Split by delimiters.
parts = re.split(r'<[^>]+>', line)
print(parts[0])
for p in parts[1:]:
print(parts[0] + p)
if __name__ == '__main__':
try:
main()
except (KeyboardInterrupt, BrokenPipeError):
pass
Usage
path/to/script < /path/to/input file
Last edited by Xyne (2013-09-01 18:34:55)