APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

sed

Sed is the "stream editor", and while it is truly ancient, it's still something most Unix folk use daily. That's because sed is quick, simple, and efficient. If you learn how to use it, you get a bonus: the same editing commands can be used in ed and vi.

O'Reilly's Sed & Awk is not something you need for day to day use, but if you really want to learn advanced sed, that's the place to go.

A more gentle approach is Definitive Guide to sed: Tutorial and Reference.

Most of us just use it to do things like this, which removes directories that I DON'T want to backup from a list:



find . -depth | sed '/Virtual PC/d;/iTunes/d;/mybackup.cpio/d;\
/Tony/d;/Downloaded/d;/Trash/d;/Desktop/d;/Library/d' | \
cpio -ocv > ~/mybackup.cpio
 

or similar trivial tasks.

However, many of the historical uses for sed have been usurped by better shells. It used to be quite common to use code like this:

# strip off ".html"
file=`echo $file | sed s/.html$//`
 

Today you are probably using Bash and would instead do:

file=${file%.html}
 

The venerable sed is also threatened by Perl, which is ubiquitous on all but the oldest Unixish boxes and of course even runs on Windows. Often sed syntax translates directly to Perl:

echo foo.html | sed 's/html/back/'
echo foo.html | perl -pe 's/html/back/'
echo foo.html | psed 's/html/back/'
 

("psed" is what it sounds like: a Perl script that acts like sed)

Those provide identical results (though for that trivial example, Perl is much slower).

As we saw yesterday when looking at the "ls" command, reading the documentation can be important. As an example, most versions now include a "-i" flag for "in-place" editing, which eliminates the need to send the edits to a new file and then copy them back. Some versions include a "-c" that works with "-i":

-c, --copy

              use  copy  instead  of  rename  when
              shuffling files in -i mode (avoids change
              of input file ownership)
 

Note that the equivalent Perl "-i" does NOT offer any modifier to avoid loss of ownership! Score +1 for sed, perhaps.

Your "sed" may have a "-s" option also. Normally "sed" treats multiple file names as one long stream. which is often just what you want, but might not be. Take the simplistic case of two files:

$ cat a
1
2
3
4
end
$ cat b
5
6
end
7
8
 

Note the difference with and without "-s":

$ sed  '2,/end/d' a b
1
5
6
end
7
8
$ sed -s '2,/end/d' a b
1
5
7
8
 

Again, I can't think of a simple Perl "one liner" to match that - you can easily process the files individually in Perl, but not so succinctly. Well, you can use "psed":

$ psed  '2,/end/d' a b
1
5
7
8
 

These are unusual cases. Generally, Perl is a better choice overall. Now and then, it's quicker to dash off a sed line (as I'll do when I want to delete multiple things from a pipeline), but in the long run, Perl is more flexible and powerful. You can translate sed to Perl with "s2p" (which is actually just "psed" using a different name).

An interesting note from "info sed":


GNU `sed' has no built-in limit on line length; as long as it can `malloc()' more (virtual) memory, you can feed or construct lines as long as you like.

They do go on to note that recursion (a subroutine calling itself) is used in some conditions, so you may run out of stack space even if you have not run out of ram!

As to whether "sed" is dead, well, maybe not quite yet, but it's hard to imagine that the new generation of Unix/Linux folk are finding much to use it for.


Got something to add? Send me email.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> sed is truly ancient, yet many Unix folk use it daily.




Increase ad revenue 50-250% with Ezoic


More Articles by

Find me on Google+

© Tony Lawrence



Kerio Samepage


Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us





The only problem with the cloud is that at some point it will rain. (Reinhard Posch)

If we define Futurism as an exploration beyond accepted limits, then the nature of limiting systems becomes the first object of exploration. (Frank Herbert)












This post tagged: