Using GNU Grep for Fun and Profit!

February 11, 2009

Shell Script: GetCluster – a shell script I happen to have written today; uses a few features of GNU Grep, whilst being tolerant of non-GNU greps (at the cost of some functionality).

This simple shell script is an example of the kind of thing that can easily be done with a few simple commands. As it is intended for the Solaris platform, which does not necessarily include GNU’s grep, some additional complexity is required to see what the platform’s grep utility is capable of.

This is achieved by:
1) Set the PATH to pick up the GNU tools first: PATH=/usr/sfw/bin:$PATH
2) Find grep (ggrep if possible; the existence of ggrep suggests that grep itself is not GNU’s grep):
type ggrep >/dev/null 2>&1
if [ "$?" -eq "0" ]; then
GREP=`which ggrep`
GREP=`which grep`

It then finds a file, which could be in one of many places:
CLUSTERTOC=`ls clustertoc .clustertoc /cdrom/cdrom0/Solaris*/Product/.clustertoc /cdrom/Solaris*/Product/.clustertoc /mnt/Solaris*/Product/.clustertoc 2>/dev/null|head -1`
What this does, is identify the first succesful match (the rest go to /dev/null). If none is found, the CLUSTERTOC will not be a valid file (note: this gets us the added bonus that it will complain if clustertoc is a device driver, pipe, etc).

It then looks for “SUNW_CSRMEMBER” in the clustertoc, using “grep -n” to get the line number, then passing that linenumber on to head so as to limit the output to only contain those lines. Any lines below that point would refer to other clusters, and be irrelevant. We want to get the last of the relevant part of the file, so that the tail command below will give us what we need.

GNU Grep has a “-A n” and “-B n” facility to say “include n lines After the matching line” or “include n lines Before the matching line”. We use “-A” here to get the following lines which describe the pacakge in more detail:
head -${linenum} $CLUSTERTOC |$GREP -A4 "^CLUSTER="|cut -d"=" -f2-|tail -4

See the readme file for specific output – the Solaris (non-GNU) version has notably less detail than the GNU version.

Hopefully the script ( is in other ways sufficiently self-explanatory.

The input file ( is relatively simple, and should explain any other queries.

Still – this simple script may provoke a few questions; please do play with it and ask away, if anything is unclear.

Happy Second Birthday!

January 21, 2009

I have missed the second birthday of this blog; starting with Hello World on 17th January 2007, this blog has seen a small growth, but the comments posted show that people have found it to be useful, which is good.





I hope that you continue to find it useful; I really will try to post a bit more often in 2009!


A very happy Christmas to all readers

December 24, 2008

echo unccl puevfgznf | tr A-Za-z N-ZA-Mn-za-m

This is a very simplistic thing called “rot13”, whereby “a” becomes “n”, “b” becomes “o”, “c” becomes “p”, and so on. The tr utility is ideal for simple string manipulation like this.

Happy Christmas to all!

ssh port forwarding without starting a new session

December 10, 2008

You can forward ports with ssh like this:

$ ssh -L 8080:localhost:80 user@remotehost

This will log you in to remotehost as user, and port 8080 on your local machine will be tunnelled to port 80 on remotehost. If remotehost can see a machine that you can’t (for example, if it’s on an internal network), you can even do this:

$ ssh -L 8080:internalhost:80 user@borderhost

This will log you in to borderhost, and localhost:8080 will be directed to internalhost:80, even though you may not be able to see internalhost directly yourself.

What I didn’t know until I read Nico Golde’s blog today, is that you can do this interactively, with an existing session. Tilde (~) is the default escape character, and ~C (note that’s an uppercase C) gets you a shell session within ssh itself:

$ ssh user@remotehost
user@remotehost$ ~C
ssh> -L 8080:localhost:80
Forwarding port.

Emptying deleted files

November 8, 2008

Mike Hommey has a nice article on clearing down deleted (but still-open) files: He explains the problem clearly, and goes on to provide the solution, too. Well worth bookmarking, for that 2am emergency!

someone had to free some space on a 1GB filesystem, and thought a good idea would be to delete that 860MB log file that nobody cares about. Except that it didn’t really remove it, but he didn’t really check.

Later, the “filesystem full” problem came back at someone else, who came to ask me what files from a small list he could remove. But the files were pretty small, and that wouldn’t have freed enough space. That gave me the feeling that we probably were in this typical case I introduced this post with, which du -sk confirmed: 970MB used on the filesystem according to df, but only 110MB worth of data…

Mike has solutions to this for Solaris and Linux; lsof is also useful for this kind of thing, on systems which have it.

more (or less) vi

October 17, 2008

When using the more tool (this works with less, also, and less is generally more useful; you can scroll backwards and forwards with less), you can press “v” to start editing the file, in vi, at the line you are currently viewing.

This works on Linux and Solaris; please confirm on other OSes as appropriate, though I would assume that what works on more on Solaris, will work on more or less on just about any OS.

This makes using more or less far more powerful than before, and a new way into vi.

chmod syntax… be careful!

September 9, 2008

Ben Hutchings has noted a quirk in the chmod syntax: If you use the “chmod o-x” syntax, but omit the “o” (Others) (or the “u” (User) or “g” (Group)), it will default to “a” (All).

So “chmod -x foo” becomes “chmod a-x foo“; similarly, “chmod +x foo” becomes “chmod a+x foo