APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed


© April 2011 Tony Lawrence

Sed is the "stream editor", and while it is truly ancient, it's still something most Unix folk use daily. That's because sed is quick, simple, and efficient. If you learn how to use it, you get a bonus: the same editing commands can be used in ed and vi.

O'Reilly's Sed & Awk is not something you need for day to day use, but if you really want to learn advanced sed, that's the place to go.

A more gentle approach is Definitive Guide to sed: Tutorial and Reference.

Most of us just use it to do things like this, which removes directories that I DON'T want to backup from a list:

find . -depth | sed '/Virtual PC/d;/iTunes/d;/mybackup.cpio/d;\
/Tony/d;/Downloaded/d;/Trash/d;/Desktop/d;/Library/d' | \
cpio -ocv > ~/mybackup.cpio

or similar trivial tasks.

However, many of the historical uses for sed have been usurped by better shells. It used to be quite common to use code like this:

# strip off ".html"
file=`echo $file | sed s/.html$//`

Today you are probably using Bash and would instead do:


The venerable sed is also threatened by Perl, which is ubiquitous on all but the oldest Unixish boxes and of course even runs on Windows. Often sed syntax translates directly to Perl:

echo foo.html | sed 's/html/back/'
echo foo.html | perl -pe 's/html/back/'
echo foo.html | psed 's/html/back/'

("psed" is what it sounds like: a Perl script that acts like sed)

Those provide identical results (though for that trivial example, Perl is much slower).

As we saw yesterday when looking at the "ls" command, reading the documentation can be important. As an example, most versions now include a "-i" flag for "in-place" editing, which eliminates the need to send the edits to a new file and then copy them back. Some versions include a "-c" that works with "-i":

-c, --copy

              use  copy  instead  of  rename  when
              shuffling files in -i mode (avoids change
              of input file ownership)

Note that the equivalent Perl "-i" does NOT offer any modifier to avoid loss of ownership! Score +1 for sed, perhaps.

Your "sed" may have a "-s" option also. Normally "sed" treats multiple file names as one long stream. which is often just what you want, but might not be. Take the simplistic case of two files:

$ cat a
$ cat b

Note the difference with and without "-s":

$ sed  '2,/end/d' a b
$ sed -s '2,/end/d' a b

Again, I can't think of a simple Perl "one liner" to match that - you can easily process the files individually in Perl, but not so succinctly. Well, you can use "psed":

$ psed  '2,/end/d' a b

These are unusual cases. Generally, Perl is a better choice overall. Now and then, it's quicker to dash off a sed line (as I'll do when I want to delete multiple things from a pipeline), but in the long run, Perl is more flexible and powerful. You can translate sed to Perl with "s2p" (which is actually just "psed" using a different name).

An interesting note from "info sed":

GNU `sed' has no built-in limit on line length; as long as it can `malloc()' more (virtual) memory, you can feed or construct lines as long as you like.

They do go on to note that recursion (a subroutine calling itself) is used in some conditions, so you may run out of stack space even if you have not run out of ram!

As to whether "sed" is dead, well, maybe not quite yet, but it's hard to imagine that the new generation of Unix/Linux folk are finding much to use it for.

Got something to add? Send me email.

(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> sed is truly ancient, yet many Unix folk use it daily.

Inexpensive and informative Apple related e-books:

Take Control of Apple Mail, Third Edition

Take Control of Numbers

Take Control of the Mac Command Line with Terminal, Second Edition

Digital Sharing Crash Course

Take Control of Pages

More Articles by © Tony Lawrence

Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us

Printer Friendly Version

FORTRAN—the "infantile disorder"—, by now nearly 20 years old, is hopelessly inadequate for whatever computer application you have in mind today: it is now too clumsy, too risky, and too expensive to use. (Edsger W. Dijkstra)

Linux posts

Troubleshooting posts

This post tagged:





Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode