Improve your scripts argument processing wih getopt and getopts.
Both "getopt" and getopts are tools to use for processing and validating shell script arguments. They are similar, but not identical. More confusingly, functionality may vary from place to place, so you need to read the man pages carefully if your usage is more than casual.
Properly handling command line arguments is difficult if you want the usage to be flexible. It's easy to write a script that demands arguments in a specific order; much harder to allow any order at all. It's also hard to allow bunched together arguments or spaced out to be equivalent:
foo -a -b -c foo -abc foo -a -c +b foo -ac +b
These are the problems that getopt(s) are designed to handle. They ease the job considerably, but introduce their own little quirks that your script will need to deal with.
This is a standalone executable that has been around a long time. Older versions lack the ability to handle quoted arguments (foo a "this won't work" c) and the versions that can, do so clumsily. If you are running a recent Linux version, your "getopt" can do that; SCO OSR5, Mac OS X 10.2.6 and FreeBSD 4.4 has an older version that does not.
The simple use of "getopt" is shown in this mini-script:
#!/bin/bash echo "Before getopt" for i do echo $i done args=`getopt abc:d $*` set -- $args echo "After getopt" for i do echo "-->$i" done
What we have said is that any of -a, -b, -c or -d will be allowed, but that -c is followed by an argument (the "c:" says that).
If we call this "g" and try it out:
bash-2.05a$ ./g -abc foo Before getopt -abc foo After getopt -->-a -->-b -->-c -->foo -->--
We start with two arguments, and "getopt" breaks apart the options and puts each in its own argument. It also added "--".
Of course "getopt" doesn't care that we didn't use "-d"; if that were important, it would be up to your script to notice and complain. However, "getopt" will notice if we try to use a flag that wasn't specified:
bash-2.05a$ ./g -abc foo -d -f Before getopt -abc foo -d -f getopt: illegal option -- f After getopt -->-a -->-b -->-c -->foo -->-d -->--
However, if you preface the option string with a colon:
args=`getopt :abc:d $*`
"getopt" will be silent about the unwanted flag.
As noted at the beginning, if we give "getopt" arguments containing spaces, it breaks:
bash-2.05a$ ./g -abc "foo bar" Before getopt -abc foo bar After getopt -->-a -->-b -->-c -->foo -->-- -->bar
Not only has "foo bar" become two arguments, but they have been separated. This will be true whether you have the newer version that is capable of handling those arguments or not, because it requires different syntax to handle them. If you do have the newer "getopt", you'd need to write the script differently:
#!/bin/bash echo "Before getopt" for i do echo $i done args=`getopt -o abc: -- "$@"` eval set -- "$args" echo "After getopt" for i do echo "-->$i" done
We've added a "-o", changed $* to $@ in quotes, and used an "eval" for the set. With the newer (as is on Linux) version, that works:
bash-2.05a$ ./g -abc "foo bar" bash-2.05a$ ./g -abc "foo bar" Before getopt -abc foo bar After getopt -->-a -->-b -->-c -->foo bar -->--
However, if you use that script with the older getopt, you get a useless result:
bash-2.05a$ ./gg -abc "foo bar" Before getopt -abc foo bar After getopt -->-- -->abc: -->-- -->-abc -->foo -->bar
It's unfortunately easy to get bad results from "getopt" by misquoting or using the wrong syntax. Whenever I've had to use this, I make sure to print out the arguments as I did in the "After getopt" while testing. Once you get it right, using it is easy:
#!/bin/bash # (old version) args=`getopt abc: $*` if test $? != 0 then echo 'Usage: -a -b -c file' exit 1 fi set -- $args for i do case "$i" in -c) shift;echo "flag c set to $1";shift;; -a) shift;echo "flag a set";; -b) shift;echo "flag b set";; esac done
and the results are as expected.
bash-2.05a$ ./g -abc "foo" flag a set flag b set flag c set to foo bash-2.05a$
However, note the "Usage" section which prints if "getopt" doesn't like what you gave it: an extra flag, or not giving an argument to a flag that requires one. Using the this newest script, we can test some of that:
bash-2.05a$ ./g -ab -c getopt: option requires an argument -- c Usage: -a -b -c file Bash-2.05a$ ./g -abj foo getopt: illegal option -- j Usage: -a -b -c file
But "getopt" is easily fooled:
bash-2.05a$ ./g -a -c -b foo flag a set flag c set to -b flag b set
You'd have to deal with that nastiness yourself.
sh and bash builtin. Easier to use and generally better than getopt, though of course not available in csh-like shells. You shouldn't be using those anyway.
This works rather differently than "getopt". First, because it's a built-in, you usually won't find a separate man page for it, though "help getopts" may give you what you need.
The old "getopt" is called once, and it modifies the environment as we saw above. The builtin "getopts" is called each time you want to process an argument, and it doesn't change the original arguments . A simple script to test with:
#!/bin/bash while getopts "abc:" flag do echo "$flag" $OPTIND $OPTARG done
Trying this produces good results:
bash-2.05a$ ./g -abc "foo" a 1 b 1 c 3 foo
The "$OPTIND" will contain the index of the argument that will be examined next. If you really needed to, you could tell from that whether arguments were bunched together or given separately, but the real point of it is to let you reset it to re-process the arguments. Try this slightly more complicated version (we'll call it "gg"):
#!/bin/bash while getopts "abc:def:ghi" flag do echo "$flag" $OPTIND $OPTARG done echo "Resetting" OPTIND=1 while getopts "abc:def:ghi" flag do echo "$flag" $OPTIND $OPTARG done
We'll give it more arguments so that you can observe it at work:
bash-2.05a$ ./gg -a -bc foo -f "foo bar" -h -gde a 2 b 2 c 4 foo f 6 foo bar h 7 g 7 d 7 e 8 Resetting a 2 b 2 c 4 foo f 6 foo bar h 7 g 7 d 7 e 8
The leading ":" works like it does in "getopt" to suppress errors, but "getopt" gives you more help. Back to our first simple version:
sh-2.05a$ ./g -a -c -b foo a 2 c 4 -b
The builtin "getopts" doesn't get fooled: the "-b" is the argument to c, but it doesn't think that b is set also.
If "getopts" encounters an unwanted argument, and hasn't been silenced by a leading ":", the "$flag" in our script above will be set to "?":
bash-2.05a$ ./g -a -c foo -l a 2 c 4 foo ./g: illegal option -- l ? 4 bash-2.05a$ ./g -a -c a 2 ./g: option requires an argument -- c ? 3
With a leading ":" (while getopts ":abc:d" flag), things are different:
bash-2.05a$ ./g -a -c a 2 : 3 c bash-2.05a$ ./g -a -c foo -l a 2 c 4 foo ? 4 l bash-2.05a$ ./g -a -c a 2 : 3 c
If an argument is not given for a flag that needs one, "$flag" gets set to ":" and OPTARG has the misused flag. If an unknown argument is given, a "?" is put in "$flag", and OPTARG again has the unrecognized flag.
See also Perl Getopts
© September 2003 Tony Lawrence All rights reservedGot something to add? Send me email.
More Articles by Tony Lawrence © 2013-06-29 Tony Lawrence
The difference between theory and practice is that in theory, there is no difference between theory and practice. (Richard Moore)
Good article! It should also be noted that getopts is part of the Korn shell (ksh) shipped with SCO OSR5 and Unixware.
--BigDumbDinosaur
Ahh.. forgot about ksh! Thanks..
--TonyLawrence
Mon May 16 00:54:44 2005: 504 anonymous
Great! thank's a lot!
Tue Jun 7 15:17:15 2005: 623 anonymous
Of course, getopts doesn't handle long option names, while getopt does.
Tue Jun 7 18:03:59 2005: 629 BigDumbDinosaur
Of course, getopts doesn't handle long option names, while getopt does.
So don't use long option names. Besides, typing long option names gets annoying in a big hurry. If you need to specify more than 52 options (that 26 LC and 26 UC for those who are not familiar with UNIX shell case sensitivity), how about using a config file? <Grin>
Mon Jun 20 23:42:37 2005: 675 dannyman
Okay, but after I loop through getopts, how do I reset the $@ stuff in bash so that I can read in subsequent arguments? For a thing like:
./foo.sh -a -b -c file.conf arg1 arg2 arg3 ...
Wed Jun 22 20:02:16 2005: 683 TonyLawrence
By doing OPTIND=1
Fri Aug 5 22:51:11 2005: 934 anonymous
or maybe :
Fri Sep 23 20:34:42 2005: 1117 anonymous
While I understand it's never going away, getopt() is a really bad, bad, bad piece of code. This is the part of it in C I hate:
extern int optind;
extern char *optarg;
I detest functions like this that use hard coded externs that make programmers not older than dirt scratch their head and say , what the heck?
Fri Sep 23 22:36:23 2005: 1118 BigDumbDinosaur
I detest functions like this that use hard coded externs that make programmers not older than dirt scratch their head and say, "what the heck?"
Whaddya mean "older than dirt?" I completely resemble that remark.
Seriously, library functions like getopt are part of the glue that holds a coherent UNIX API together. I have a big problem with some of the programming youngsters who, rather than understand and work with standard library functions that we old dinosaurs have used for decades, want to come up with something different just because they don't understand the existing function or the associated variable names (optind makes perfect sense to me -- it's option index). It's that sort of narrow thinking that has resulted in some of the language abortions we have today, such as Java and C++.
BTW, just how would you implement a library function like getopt and provide identical functionality, without any use of "hard coded externs"?
Fri Oct 28 17:36:01 2005: 1256 anonymous
The comments above asked about getting the rest of the args from getopts and suggested setting OPTIND. I hacked out something, and I think it's a better way, something like this:
Thu Nov 17 14:21:40 2005: 1362 anonymous
In your case statement, just use after you've processed a certain option (and it's argument).
bartX
Thu Dec 14 20:12:02 2006: 2738 Dave
Found this very useful, thanks!
Thu Dec 14 23:45:42 2006: 2739 BigDumbDinosaur
Naturally, all of this getopts maneuvering is thoroughly documented in a 1000 places (including here), and buried somewhere in that documentation (What!!!??? I have to actually read this crap???) is the little part about using to reset the option index to a new "ground zero" setting. Obviously, the anonymous person who pointed this out *has* read the documentation. <Grin>
Wed Jan 23 16:59:10 2008: 3514 anonymous
Thanks for this article -- it's a shame that the bash info file is so example free. I could have never figured out what was meant from that alone.
Fri Feb 8 06:56:52 2008: 3602 anonymous
Great article, very useful! Thank you a lot!
Wed May 21 13:51:22 2008: 4225 anonymous
while getopts fdagrACi: option
do
case $option in
f )
#echo "f"
FLAG_FIH=Y
;;
d )
#echo "d"
FLAG_DH=Y
;;
a )
#echo "a"
FLAG_AH=Y
;;
g )
#echo "g"
FLAG_GH=Y
;;
r )
#echo "r"
FLAG_RP=Y
;;
esac
done
This works, but now I want to add the options r3), r4) and r5) instead only option r). I tried to add new case options, but (obviously?) it won't work, it gives me the message for the command 'teste_rating.sh -r3' :
"teste_rating.sh[540]: getopts: 3 bad option(s)"
Anyone gives me some help? I'm new at this shell scripts programming (has you have already noticed)
Best regards
Wed May 21 14:05:01 2008: 4226 anonymous
I think you want to look at the ":" options above to say that -r takes an argument, and then parse that argument.
Fri Jun 27 06:19:23 2008: 4378 thierryescola
très bon article qui m'a bien aidé.
merci beaucoup
Thierry
Mon Oct 19 23:28:44 2009: 7311 hb6381
I found this article. Here are my 2 cents.
#!/bin/bash
optstr=`getopt ab:c: $*` ## a w/o param, b and c with param
eval set -- "$optstr"
for i
do
case "$i" in
-a) shift; echo "A is set" ;;
-b) shift; echo "B is $1"; shift;;
-c) shift; echo "C is $1"; shift;;
esac
done
##Conclusion:
##No good. Try running this w/o parameter to option b or c where that option is not the last one in the list.
##I would prefer getopts, over this. The only drawback of getopts I see, now, is that it cannot do long named
##options. I will have to stay with that, rather than this bug with getopt.
So as I put in the comments, I will probably just stick to getopts for now, (though i dont like the idea of looping as much)
Mon Mar 29 13:00:21 2010: 8295 anonymous
how can -a and -b have the optind value 1 in the first getopts example's output itsd have been 1 & 2 right?
Mon Mar 29 13:12:13 2010: 8296 TonyLawrence
Try it and you'll see.
Mon Mar 29 16:44:29 2010: 8297 piyush
how to use "set" while using getopts as after that its not processing further arguments
Mon Mar 29 17:42:22 2010: 8298 TonyLawrence
Of course it does - how could it not?
What is it you are actually trying to accomplish?
Thu Apr 22 22:42:22 2010: 8458 SumitroPalit
getopts does handle long options - you just need to create an alias for the short option and the use -- instead of -. At least worked on Sun Solaris 5.10
Check out the man to see if long options are supported on your system:
getopts supports both traditional single-character short
options and long options defined by Sun's Command Line
Interface Paradigm (CLIP).
Each long option is an alias for a short option and is
specified in parentheses following its equivalent short
option. For example, you can specify the long option file
as an alias for the short option f using the following
script line:
getopts "f(file)" opt
Precede long options on the command line with -- or ++. In
the example above, --file on the command line would be the
equivalent of -f, and ++file on the command line would be
the equivalent of +f.
Each short option can have multiple long option equivalents,
although this is in violation of the CLIP specification and
should be used with caution. You must enclose each long
option equivalent parentheses, as follows:
getopts "f:(file)(input-file)o:(output-file)"
In the above example, both --file and --input-file are the
equivalent of -f, and --output-file is the equivalent of -o.
The variable name is always set to a short option. When a
long option is specified on the command line, name is set to
the short-option equivalent.
Fri Apr 23 15:30:19 2010: 8463 anonymous
this is a very useful article,but i'm realising a project where i'm obliged to use long option but i can't.please help me ? tank you..
Fri Apr 23 16:01:16 2010: 8464 TonyLawrence
Did you read the comment just above yours?
Sat Apr 24 08:16:06 2010: 8471 piyush
@tony
i am getting problem if i use the script like this...its not the full script but the problem is as soon as shell is executing set command, it stops processing further getopts options.
while getopts n:s:v OPTION; do
case "$OPTION" in
n) echo "N";set `who` ;;
s) echo "S";;
v) echo "V";;
esac
done
Sat Apr 24 10:18:52 2010: 8473 TonyLawrence
Well, of course it does. The "set" changes the value of the arguments.
I don't know what you need to do with "who", but don't "set" it. Put it in a variable or multiple variables and work with that.
Sat Apr 24 11:51:11 2010: 8474 piyush
its just a example...actual script is something else..How can i store the output of who ( or any command ) to variable (or array of variable), using getopts at same time...
Sat Apr 24 11:58:28 2010: 8475 TonyLawrence
MyVar=`who`
or
MyVar=$(who)
Sat Apr 24 12:15:02 2010: 8477 TonyLawrence
May I suggest you buy a book on basic scripting?
Bash:
Myarray=($(who))
echo ${Myarray[0]}
echo ${Myarray[2]}
Thu Apr 29 16:49:51 2010: 8484 anonymous
Thanks for this usefull post,
i had problem with arguments given after options (to use them with $1, $2...) so i did:
until [ $OPTIND -le 1 ]
do
shift;
OPTIND=`expr $OPTIND - 1`;
done
i hope it's usefull
Thu Dec 23 03:16:20 2010: 9176 anonymous
hi !!I have a problem
you say:
sh-2.05a$ ./g -a -c -b foo
a 2
c 4 -b
The builtin "getopts" doesn't get fooled: the "-b" is the argument to c, but it doesn't think that b is set also.
means the "getopts" can avoid this?
but i can't
#!/bin/bash
while getopts ":abc:def:ghi" flag
do
echo "$flag" $OPTIND $OPTARG
done
./gg -c -d
c 3 -d
it's wrong,how can i get this?
c 2
d 3
Thu Dec 23 11:48:25 2010: 9177 TonyLawrence
You are misunderstanding the example.
If you put a ":", that says an argument will follow. So you've said c requires an argument and with "./gg -c -d ", you gave it one: "-d"
Wed Dec 29 00:33:23 2010: 9185 bsquared
My question is similar to above:
# f is an optional flag, o requires a path
getops "fo:" flag
how can I best invalidate a flag passed as an argument to o?
either as "test -foj" or as "test -f -o -j".
I wonder what is the best practice in this scenario?
Thanks.
Wed Dec 29 00:33:23 2010: 9185 bsquared
My question is similar to above:
# f is an optional flag, o requires a path
getops "fo:" flag
how can I best invalidate a flag passed as an argument to o?
either as "test -foj" or as "test -f -o -j".
I wonder what is the best practice in this scenario?
Thanks.
Wed Dec 29 11:57:29 2010: 9186 TonyLawrence
You look at what was passed and decide if it is valid.
When it starts to get complicated, I shuck Bash and move to Perl.
Wed Dec 29 17:37:11 2010: 9187 bsquared
Oddly, when I type man getopts I get a perl man page (Getopt::Std(3perl)) have to check my packages, but looking online I was able to sort it out with the man page. Thanks for the info. very helpful.
Wed Dec 29 17:43:00 2010: 9188 TonyLawrence
I do have another page here specifically about the Perl methods (Getopt and GetOptions) : (link)
Be sure to look at GetOptions::Long especially.
Mon Mar 14 17:18:51 2011: 9369 ravi
Thank you, Lawrence!
I understand a bit more about getopts after reading your article.
Mon May 23 09:06:15 2011: 9500 anonymous
Will getopts work fine if we use source <command line>?
Mon May 23 10:36:47 2011: 9501 TonyLawrence
Will getopts work fine if we use source <command line>?
If you mean are the variables available in the script that used "source", no.
Fri Oct 7 11:56:06 2011: 9935 anonymous
The Bash builtin getopts function can be used to parse long options by putting a dash character followed by a colon into the optspec. For an example script see:
(link)
Fri Mar 2 06:09:10 2012: 10681 NB
Hi!
I tried using getopts in while with case like:
function dspusage
{
echo "###"
echo "### Usage: $1 [-u username] [-d dirname] [-v VAH]"
exit
}
while getopts "d:u:v:h:*" arguments
do
case $arguments in
d) PDIR=$OPTARG;;
u) USR=$OPTARG;;
v) VAH=$OPTARG;;
h) dspusage;;
*) dspusage;;
\?) shift;;
esac
done
I am actually trying to call dspusage if the script is being called with wrong arguments or with -h.
It works fine wiht -h but not wiht h or anything without '-'.
Can you please tell me whats wrong with this?
Thanks.
NB
Fri Mar 2 12:58:58 2012: 10687 TonyLawrence
Why would you want it to work without a "-" ??
Fri Mar 2 16:06:18 2012: 10690 BigDumbDinosaur
Can you please tell me whats wrong with this?
There shouldn't be a colon after the 'h' option in your getopts argument list. What you are doing is telling getopts that the 'h' option itself should have an argument -- which it shouldn't.
getopts is not complicated to use. Reread Tony's article and you will see that it's not rocket science or brain surgery.
Fri Mar 2 16:38:26 2012: 10692 TonyLawrence
I wasn't sure if he wanted -h to have an argument or not - not too clear..
Wed Dec 19 06:20:54 2012: 11573 felipe1982
This advice is really old! The part of the article about what occurs when it's silenced (`:') is not up to date.
Wed Dec 19 11:41:26 2012: 11574 TonyLawrence
I am tempted to delete your useless comment, Felipe, but I'll give you one chance to redeem it.
If you think there is something I neglected to mention about the use of ":" to silence warnings, tell us what it is. Making empty complaints as you did is of no use to anyone.
As to "really old", yes. So what?
Sat Jun 29 00:58:34 2013: 12176 younggun
Tony --Thank you so, much for this article! Just got home and will test this all out asap.
BigDumbDinosaur -- Your name made me laugh... but please don't assume that all of us young folk just don't understand. I believe that you too were once a youngster too and as green as the grass on the other side.
Regardless if we understand it or not redoing it ourselves kick starts new innovations and sometimes(ok most of the time) we re-invent the wheel, but some of those innovations can lead to fundamental improvements in ways we could never have thought had we stuck to the beaten path. Being "older than dirt", as it was so eloquently put earlier, I am positive that you have been a trailblazer in the niche that you call home in the world of computers, and you did not follow the ways of 'old' because of whatever reason. While I am not one of those dirty hippie's that's all about peace and love and what have you I do see that we all(as programmers) have our place to fill... New and OLD.
TO: us 'Youngsters': If we do not learn from our past we are doomed to repeat it.
Sat Jun 29 01:14:37 2013: 12177 TonyLawrence
Aww, youngun, don't mind us - we get grumpy, but our time is over.. just pat us on the head and put us to bed :-)
------------------------
Printer Friendly Version
Getopt and getopts Copyright © September 2003 Tony Lawrence
Have you tried Searching this site?
This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.
Contact us
Printer Friendly Version