Timestamps for Log Files

March 11, 2007

There are two common occasions when you might want to get a timestamp

  • If you want to create a logfile called “myapp_log.11.Mar.2007″
  • If you want to write to a logfile with “myapp: 11 Mar 2007 22:14:44: Something Happened”

Either way, you want to get the current date, in the format you prefer – for example, it’s easier if a filename doesn’t include spaces.

For the purposes of this article, though for no particular reason, I am assuming that the current time is 10:14:44 PM on Sunday the 11th March 2007.

The tool to use is, naturally enough, called “date“. It has a bucket-load of switches, but first, we’ll deal with how to use them. For the full list, see the man page (“man date“), though I’ll cover some of the more generally useful ones below.

Setting the Date/Time

The first thing to note, is that date has two aspects: It can set the system clock:

# date 031122142007.44

will set the clock to 03 11 22 14 2007 44 – that is, 03=March, 11=11th day, 22 = 10pm, 14 = 14 minutes past the hour, 2007 = year 2007, 44 = 44 seconds past the minute.

Heck, I don’t even know why I bothered to spell it out, it’s obvious. Of course the year should come between the minutes and the seconds (ahem).

Getting the Date/Time

The more often used feature of the date command, is to find the current system date / time, and that is what we shall focus on here. It doesn’t follow tradition, in that it uses the “+” and “%” symbols, instead of the “-” symbol, for its switches.

H = Hours, M = Minutes, S = Seconds, so:

$ date +%H:%M:%S

Which means that you can name a logfile like this:

LOGFILE=/tmp/log_`date +%H%M%S`.log
echo Starting work > $LOGFILE
do_stuff >> $LOGFILE
do_more_stuff >> $LOGFILE
echo Finished >> $LOGFILE

This will create a logfile called /tmp/log_221444.log

You can also put useful information to the logfile:

LOGFILE=/tmp/log_`date +%H%M%S`.log
echo `date +%H:%M:%S : Starting work > $LOGFILE
do_stuff >> $LOGFILE
echo "`date +%H:%M:%S : Done do_stuff" >> $LOGFILE
do_more_stuff >> $LOGFILE
echo "`date +%H:%M:%S : Done do_more_stuff" >> $LOGFILE
echo Finished >> $LOGFILE

This will produce a logfile along the lines of:

$ cat /tmp/log_221444.log
22:14:44: Starting work
do_stuff : Doing stuff, takes a short while
22:14:53: Done do_stuff
do_more_stuff : Doing more stuff, this is quite time consuming.
22:18:35: Done do_more_stuff

Counting the Seconds

UNIX has 1st Jan 1970 as a “special” date, the start of the system clock; GNU date will tell you how many seconds have elapsed since midnight on 1st Jan 1970:

$ date +%s

Whilst this information is not very useful in itself, it may be useful to know how many seconds have elapsed between two events:

$ cat list.sh
start=`date +%s`
ls -R $1 > /dev/null 2>&1
end=`date +%s`

diff=`expr $end - $start`
echo "Started at $start : Ended at $end"
echo "Elapsed time = $diff seconds"
$ ./list.sh /usr/share
Started at 1173651284 : Ended at 1173651290
Elapsed time = 6 seconds

For more useful switches, see the man page, but here are a few handy ones:

$ date "+%a %b %d" # (in the local language)
Sun Mar 11
$ date +%D         # (show the full date)
$ date +%F         # (In another format)
$ date +%j         # (how many days into the year)
$ date +%u         # (day of the week)

Looping in the shell (for and while)

March 7, 2007

In programming, the two most common types of loop are “for” and “while” loops. We can do both (and “repeat” loops, too, because that’s just a special case of the “while” loop) with the *nix shell. I’ve got some more detail in the tutorial.

for loops

Some languages have a “foreach” command; if you are used to such a language, then treat the shell’s for command as equivalent to foreach. If not, then don’t worry about it, just watch the examples, it’s about as simple as it can be.

$ for artist in Queen "Elvis Costello" Metallica
> do
> echo "I like $artist"
> done
I like Queen
I like Elvis Costello
I like Metallica

The for loop will simply go through whatever text it gets passed, and do the same stuff with each item.

Note that for “Elvis Costello” to be marked as one artist, not “Elvis” and “Costello”, I had to put quotes around the words. However, if these were files, then the following would suffice:

$ ls -l
total 0
-rw-r--r-- 1 steve steve 0 2007-03-07 00:45 Elvis Costello
-rw-r--r-- 1 steve steve 0 2007-03-07 00:45 Metallica
-rw-r--r-- 1 steve steve 0 2007-03-07 00:45 Queen
$ for artist in *
> do
> echo "I like $artist"
> done
I like Elvis Costello
I like Metallica
I like Queen

This time around, because we passed them via the shell’s interpreter, they are parsed in alphabetical order.

Many languages can do better than “foreach”, though: “for i=1 to 99 step 3“, for example, to step through 1,4,7,10 .. 91, 94, 97.

We can do this with a while loop.

while loops

While loops are not quite as simple as for loops; they have some kind of condition to match; when the condition does not match, the loop will exit.

The examples above, stepping through from 1 to 100 in increments of 3 (1, 4, 7, 10, … 91, 94, 97), can easily enough be done with a while loop:

$ i=1
$ while [ "$i" -lt "100" ]
> do
>   echo $i
>   i=`expr $i + 3`
> done

The “i=`expr $i + 3`” means “increment ‘i’ by 3″ (“i = i + 3” in most other languages).

The “-lt” means “is less than” (“-le” means “is less than or equal too; see “man test”, or http://steve-parker.org/sh/test.shtml and http://steve-parker.org/sh/quickref.shtml)

Tool Tip: “ls”

February 26, 2007

Yeah yeah, we know ls already.

But how much of ls‘s functionality do you actually use? There are so many switches to ls, that when Sun added extended attributes (does anyone use that?) they found that there were no letters left, so they had to use “-@” !

So, here are a couple of handy ls options, in no particular order; either for interactive or scripting use. I’m assuming GNU ls; Solaris ls supports most GNU-style features, but the “nice-to-have” features, like ls -h aren’t in historical UNIX ls implementations. I’ll split these into two categories: Sort ‘em and Show ‘em. What are your favourites?

Sort ‘em

When sorting, I tend to use the “-l (long listing)” and “-r (reverse order)” switches:

Sort ‘em by Size:

ls -lSr

Sort ‘em by Date:

ls -ltr

Show ‘em

There are a number of ways to show different attributes of the files you are listing; “-l” is probably the obvious example. However, there are a few more:

Show ‘em in columns

ls -C

Useful if you’re not seeing as many as you’d expect.

Show ‘em one by one

ls -1

That’s the number 1 (one) there, not the letter l (ell). Forces one-file-per-line. Particularly useful for dealing with strange filenames with whitespace in them.

Show ‘em as they are

ls -F

To append symbols (“*” for executables, “/” for directories, etc) to the filename to show further information about them.

Show ‘em so I can read it

ls -lh

Human-readable filesizes, so “12567166” is shown as “12M”, and “21418” is “21K”. This is handy for people, but of course, if you’re writing a script which wants to know file sizes, you’re better off without this (21Mb is bigger than 22Kb, after all!)

Show ‘em with numbers

ls -n

This is equivalent to ls -l, except that UID and GID are not looked up, so:

$ ls -l foo.txt
-rw-r--r-- 1 steve steve 46210 2006-11-25 00:33 foo.txt
$ ls -n foo.txt
-rw-r--r-- 1 1000 1000 46210 2006-11-25 00:33 foo.txt

This can be useful in a number of ways; particularly if your NIS (or other) naming service is down, or if you’ve imported a filesystem from another system.

What’s your favourite?

What are your most-used switches for the trusty old ls tool?

Harnessing the flexibility of Regular Expressions with Grep

February 14, 2007

There are two sides to grep – like any command, there’s the learning of syntax, the beginning
of which I covered in the grep tool tip. I’ll
come back to the syntax later, because there is a lot of it.

However, the more powerful side is grep‘s use of regular expressions. Again, there’s not room
here to provide a complete rundown, but it should be enough to cover 90% of usage. Once I’ve got a library
of grep-related stuff, I’ll post an entry with links to them all, with some covering text.

This or That

Without being totally case-insensitive (which -i) does,
we can search for “Hello” or “hello” by specifying the optional
characters in square brackets:

$ grep [Hh]ello *.txt
test1.txt:Hello. This is  test file.
test3.txt:Why, hello there!

If we’re not bothered what the third letter is, then we can say “grep [Hh]e.o *.txt“, because the dot (“.”) will match any single character.

If we don’t care what the third and fourth letters are, so long as it’s “he..o”, then we say exactly that: “grep he..o” will match “hello”, hecko”, heolo”, so long as it is “he” + 1 character + “lo”.

If we want to find anything like that, other than “hello”, we can do that, too:

$ grep he[^l]lo *.txt

Notice how it doesn’t pick up any of the “Hello” variations which have a “llo” in them?

How many?!

We can specify how many times a character can repeat, too. We have to put the expression we’re talking about in [square brackets]:

  • “?” means “it might be there”
  • “+” means “it’s there, but there might be loads of them”
  • “*” means “lots (or none) might be there”

So, we can match “he”, followed by as many “l”s as you like (even none), followed by an “o” with “grep he[l]*o *.txt“:

$ grep he[l]*o *.txt
test3.txt:Why, hello there!


February 14, 2007

The great thing about Unix, and the Bourne shell, when it was introduced back in the day, was multitasking. It’s such an overused buzzword these days, but at the time, it was really a new thing. If you’ve only got one connection in to a machine, you can get it doing as much as you want.

The shell command to “do this in the background, then give me a new prompt to provide the next command” is the ampersand (“&”):

$  # I need to trawl the filesystem for files called "*dodgy*"
    #  (should have installed slocate
    #  (http://packages.debian.org/stable/source/slocate), 
    # but it's too late for that)
$ find / -name "*dodgy" -print
    (wait for a very very very long time)

Well, that’s a good hour of my life wasted.

Chuck it into a script, and run it in the background. If you want the outcome, direct it to a file:

$ cd /tmp
$ cat myfindscript.sh
find / -name "*dodgy*"
$ chmod u+rx myfindscript.sh
$ ./myfindscript.sh > /tmp/mysuspectfiles.txt &
$ # wow, didja see that? It'll take ages, but I've got 
   # control back. "4402" is the Process ID (PID), 
   # so I can run "ps -fp 4402" to check on its
   # progress, but it's happening, in the backrgound.

You don’t get a lot of job control here; the “ps” mentioned above is about your lot, but you can spawn a child process and let it run, whilst you get on with the stuff you need.

This is known as “backgrounding” a task; if you know it will take a long time, just background it. Of course, if the next thing you need to do is to read the entire file, then you won’t get away with it, you’ll have to wait for it to finish. However, you could background it and then “tail -f /tmp/mysuspectfiles.txt” to check on the status.

Search and Replace

February 6, 2007

Two great tools for search-and-replace are tr and sed.

tr – Translate (or Delete)

tr can translate a single character into another. For example, “tr 'a' 'b'” will convert all instances of “a” into “b”. Although it works on single characters, it also understands blocks, so “tr '[A-Z]' '[a-z]'” will convert uppercase to lowercase: ‘A’ becomes ‘a’, ‘B’ becomes ‘b’, and so on.

The GNU version of tr (which comes with most Linux distributions) has some handy keywords, too: “tr [:lower:] [:upper:]” is a more readable opposite of the above [A-Z] convention.

You can also specify your own set of characters, so if you want to convert ‘l’ to ‘1’, ‘o’ to ‘0’, and ‘e’ to ‘3’, then this will do the job:

$  echo welcome | tr 'loe' '103'

You could extend it to a more complete “l33t sp33k”:

$ echo abcdefghijlkmnopqrstuvwxyz | tr 'aeilost' '@3!1057'

tr can also just delete – this deletes the letter ‘l’ :

$ echo hello and welcome | tr -d 'l'
heo and wecome

sed – Stream Editor

UNIX uses a few metaphors, one being a water metaphor, which we use with pipes (|), redirects (<, >), and a few other places. Sed gets its name from what it does… much like tr, you stream data into it, and slightly modified data comes out the other end.

sed isn’t limited to single-character operations; it can cope with whole phrases, as well as regular expressions. I’ll keep it simple(ish) for now, I plan to do a more complete post on sed and another on regular expressions soon, though. For today, I’ll stick to the sed s/from/to/count syntax.

With the s/from/to/count syntax, sed will convert “from” to “to”, as many times (per line of text) as you specify. The special “/g” converts every instance.

I like to get stuck in with a few examples, so here goes:

$ cat text.txt
Fedora Core is my favourite distribution.
It's got just the right level of ease-of-use
along with regular updates, whilst remaining
a stable, supportable Operating System. In fact,
I'd go so far as to say that Fedora Core is definitely
the best Linux distribution for home users.
Fedora Core is certainly my favourite distribution.
$ sed s/"Fedora Core"/"Ubuntu"/g text.txt
Ubuntu is my favourite distribution.
It's got just the right level of ease-of-use
along with regular updates, whilst remaining
a stable, supportable Operating System. In fact,
I'd go so far as to say that Ubuntu is definitely
the best Linux distribution for home users.
Ubuntu is certainly my favourite distribution.

The syntax there was sed s/from/to/count, so it replaces “Fedora Core” with “Ubuntu” in this example. If we specified “/1” at the end, it would only convert the first instance on each line. Similarly, “/2” would convert the first two instances. “/g” is probably the most-used, it converts everything (the “g” stands for “global”).

sed can emulate tr‘s tr -d functionality by having the “to” part being an empty string; here we refer to “Fedora Core” simply as “Fedora” (note the leading space: it’s ” Core”, not “Core”) :

$ sed s/" Core"//g text.txt
Fedora is my favourite distribution.
It's got just the right level of ease-of-use
along with regular updates, whilst remaining
a stable, supportable Operating System. In fact,
I'd go so far as to say that Fedora is the
best Linux distribution for home users.
Fedora is certainly my favourite distribution.

Notice also that we can cat stuff into sed, as “cat text.txt | sed s/src/dest/g“, or we can pass the file directly to sed, like this: sed s/src/dest/g text.txt. The same applies to most *nix commands.

To get in to the rest, we’ll need to get into regexp (Regular Expressions, the stuff like “the * brown * jumped over the * dog” which result in $1 = “quick”, $2 = “fox”, $3 = “lazy”). That’s for another day, though.


January 31, 2007

The cut utility is a great example of the UNIX philosophy of “do one thing, and do it well.” cut just cuts lines of text, based on a (single-character) delimiter.

There are two basic forms in which cut is generally used:

1. Grab These Columns (cut -c)

One form of cut gets certain characters, or columns of characters, out of a file. This is done with the cut -c command. So we can get the 5th character of the string:

$ echo "Hello, World" | cut -c 5

Or we can get the first 5 characters:

$ echo "Hello, World" | cut -c -5

Or from character #5 onwards:

$ echo "Hello, World" | cut -c 5-
o, World

Or even just characters 2-5:

$ echo "Hello, World" | cut -c 2-5

Or maybe just select characters 5, 8 and 9:

$ echo "Hello, World" | cut -c 5,8,9

2. Delimited (cut -d)

The other method is to use a delimiter. For example, the /etc/passwd file looks something like thism with the fields delimited by the “:” (colon) symbol:

steve:x:1000:1000:Steve Parker,,,:/home/steve:/bin/bash

In this example, field 1 is “root” (or “steve”), field 2 is “x”, and so on…. the last field (7) is “/bin/bash” for both accounts, in this case.

So we just have to specify the delimiter (:) and the field number(s). Field 1 is the account name, 6 is the home directory, field 3 is the UserID…

$ cut -d: -f1 /etc/passwd
$ cut -d: -f6 /etc/passwd
$ grep "^root:" /etc/passwd | cut -d: -f3

(In the last example, we got *just* the root account, with a grep which searches for a line starting with “root:”, which could only be the root account, and not (for example) “Fred Troot” which would also match a search for “root”)

cut is one of those really simple, but really useful utilities. And because it’s very simple, it’s nice and quick (which matters a lot if you’re looping through a few hundred times.


January 29, 2007

A lot of the power of the *nix shell is in redirection. The model is called “streams”, and we even have “pipes” for these “streams”. It’s not the best metaphor ever, but it’s good enough, I suppose. There are 3 streams associated with a process: the standard input (stdin), standard output (stdout) and standard error output (stderr). As you might expect, you get user input on stdin, output to stderr, and send errors to stderr.

stdout : Redirecting Output

Let’s start with stdout, the output stream, since that is the most common:

$ ls > /tmp/foo.txt

This will run the “ls” (list files) command, but instead of showing you the results, it will write to the file you specify (/tmp/foo.txt, in this example). If there is already a /tmp/foo.txt file, then it gets replaced.

$ ls >> /tmp/foo.txt

By using >> instead of the single >, we can append to the file instead of overwriting it.

If we want to see the output like we normally would, but log it to a file as well, we can use the “tee” utility:

$ ls | tee /tmp/foo.txt
[ the usual ls output shown here ]
$ ls | tee -a /tmp/foo.txt
[ the usual ls output shown here ]

The first command will write to /tmp/foo.txt; the second (with “-a“) will append to /tmp/foo.txt.

We have used a different redirector here; “|” (pipe) instead of “>” and “>>” . This is because we’re not funnelling the output to a different place (/tmp/foo.txt), but passing it to a program (tee) which does some more funky redirection.

Another common use of the pipe (“|“) is to go to the more (or less) utility. If a command would produce loads of output, faster than you can read it, then more will pause after the screen is full, and prompt you with a “— more” prompt (hence the name). SPACE will show the next screen; RETURN will show the next line. less is just like more, but it can also go backwards (PgUp and PgDn):

$ ls | more
--- more  (PRESS SPACE)
--- more (PRESS SPACE)

stderr : Redirecting Errors

Well-written programs will send their errors to a different “device” than they send their normal output to. Both stdout and stderr are usually the same place (your terminal), but they can be treated separately:

$ ls
$ ls fo.txt
ls: fo.txt: No such file or directory
$ ls fo.txt > output.txt 2>errors.txt
$ ls
foo.txt    output.txt   errors.txt
$ cat output.txt
$ cat errors.txt
ls: fo.txt: No such file or directory

What happened there? Well, we asked for “ls fo.txt“, but fo.txt doesn’t exist (foo.txt does). So we see an error from ls. If we direct stdout to “output.txt” and stderr to “errors.txt”, then we can see the difference. What ls actually did, was that it wrote *nothing* to stdout and the error message was sent to stderr. (stderr has file descriptor #2, so we say “2>” to direct stderr.) So when we “cat output.txt“, we get nothing (there was no output), but when we “cat errors.txt“, we see the error.

stdin: Redirecting Input

This is most commonly done by system utilities, but many shell scripts use the functionality.
The simple way is to use the “<” director:

$ mycommand < myfile.txt

This will take input from “myfile.txt” instead of from the keyboard.

The second way is probably more common. The more example above had more redirecting its input, via the pipe (“|“). We can create entire pipelines this way:

$ find . -print | grep foo | more

This will attach the stdout of the find command to the stdin of the grep command, and attach the stdout of grep to the stdin of more. Got that?! So what the *nix kernel will do, is that it will start more first, and then start grep telling it that its stdout is more‘s stdin. It will then start find, and tell it that its stdout is grep‘s stdin.

There’s actually more to it than that, but that’s got the basics of stdin, stdout and stderr covered briefly.

Simple Maths in the Unix Shell

January 29, 2007

One thing that often confuses new users to the Unix / Linux shell, is how to do (even very simple) maths. In most languages, x = x + 1 (or even x++) does exactly what you would expect. The Unix shell is different, however. It doesn’t have any built-in mathematical operators for variables. It can do comparisons, but maths isn’t supported, not even simple addition.

Following the Unix tradition (“do one thing, and do it well”) to the extreme, because the expr and bc utilites can do maths, there is absolutely no need for sh to re-invent the wheel.

Yes, I agree. This is frustrating. If I’ve got one gripe against shell programming, then this is it.

Addition and Subtraction

So how do we cope? There are basically two ways, depending on whether we choose expr or bc:


echo "Give me a number: "
read x
echo "Give me another number: "
read y

######  Here's where we have the two options:
# The expr method:
exprans=`expr $x + $y`

# The bc method:
bcans=`echo $x + $y | bc`
###### Did you see the difference?

echo "According to expr, $x + $y = $exprans"
echo "According to bc,   $x + $y = $bcans"

As you can see, the language is slightly different for the two commands; expr parses an expression passed to it as arguments: expr something function something whereas bc takes the expression as its input (stdin): echo something function something | bc. Also, for expr, you must put spaces around the arguments: “expr 1+2” doesn’t work. “expr 1 + 2” works.


Multiplication is a little awkward, too; the * asterisk, which traditionally denotes multiplication, is a special character to the shell; it means “every file in the current directory”, so we have to delimit it with a backslash. “*” becomes “\*“:


echo "Give me a number: "
read x
echo "Give me another number: "
read y

######  Here's where we have the two options:
# The expr method:
exprans=`expr $x \\* $y`

# The bc method:
bcans=`echo $x \\* $y | bc`
###### Did you see the difference?

echo "According to expr, $x * $y = $exprans"
echo "According to bc,   $x * $y = $bcans"

The other thing to note here, is the backtick (`). This grabs the output of the command it surrounds, and passes it back to the caller. So a command like

x=`expr 1 + 2`

means that, while if you type expr 1 + 2 at the command line, you’d get “3” back:

steve@nixshell$ expr 1 + 2

If you enclose it with backticks, then the variable $x becomes set to the output of the command. Therefore,

x=`expr 1 + 2`

Is equivalent to (but of course more flexible than):


One last thing about assigning values to variables: Whitespace MATTERS. Don’t put spaces around the = sign. “x = 3” won’t work. “x=3” works.

Update: 17 Feb 2007 : Division, and Base Conversion

As noted by Constantin, the “scale=x” function can be useful for defining precision (bc sometimes seems to downgrade your precision: “echo 5121 / 1024 | bc” claims that the answer is “5”, which isn’t quite true; 5120/1024=5. echo "scale = 5 ; 5121 / 1024" | bc produces an answer to 5 decimal places (5.00097)).

Another important note I would like to add, is that bc is great at converting between bases:

Convert Decimal to Hexadecimal

steve@nixshell$ bc

This tells us that 12345 is represented, in Hex (Base 16) as “0x3039″.
Similarly, we can convert back to decimal (well, we can use bc to convert any base to any other base)…

steve@nixshell$ bc

Or we can convert from Binary:

steve@nixshell$ bc

… which tells us that 01010110 (Binary) is 86 in decimal. We can get that in hex, like this:

steve@nixshell$ bc

Which tells bc that the input base is 2 (Binary), and the output base is 16 (Hex). So, 01010110 (base 2) = 56 (hex) = 86 (decimal).

Note that the order does matter a lot; if we’d have said “ibase=2; obase=16″, that would be interpreted differently from “obase=16; ibase=2″.

I hope that this article will help some people out with some of the more frustrating aspects of shell programming. Please, let me know what you think.

File Permissions

January 25, 2007

The Unix file permissions model doesn’t seem to get explained very clearly, very often. It’s really quite simple, though some of the more advanced stuff isn’t so widely known. The key commands are ls -l and chmod. chmod has two ways of working; we’ll deal with the easy one first.

When you look at a file, there are lots of fields. (I’m using Linux with an ext3 filesystem for these examples, but it’s the same across the board for Unix and Linux, and just about any filesystem.)

$ ls -l myfile.txt
-rw-r--r-- 1 steve users 4 2007-01-25 20:37 myfile.txt

So what does it all mean? Going through the fields in order, it’s:

-rw-r--r--    1    steve   users  4    2007-01-25 20:37 myfile.txt
permission  links  owner   group  size  last-modified   filename

We’re dealing with the permission stuff here, but I’ll quickly run through the others. “Links” tells you how many “hard links” there are to the file. That’s probably for another post, but if you type “ln myfile.txt yourfile.txt“, then the link count will go up from 1 to 2. “owner” tells you what user owns the file, and “group” tells you what group is associated with the file. “size” is pretty obvious; it’s in bytes, (this file’s 4 bytes are “f”, “o”, “o” and a newline character). “last-modified” tells you when the file was last changed (not necessarily when it was created), and finally, the filename.

For our purposes, the important stuff is the permission, owner and group. That’s “-rw-r–r–“, “steve” and “users” in this example.

Looking at the “-rw-r–r–“, it seems almost random. Once you know the structure, it’s very informative. There are 10 characters, or fields, grouped with the first character by itself, then three sets of three, like so:

File Type Owner Group Other
- rw- r– r–

The initial “-” for File Type, tells you what kind of file it is. In this case, “-” means it’s a regular file. “d” indicates a directory, “c” means a character-special device, and “b” means a block-special device. Run “ls -l /dev” to see some “c” and “b” files. They’re device drivers; a character-device (eg, /dev/lp0, the printer) is accessed with characters; you tend to chuck text at it. A block-device (eg, /dev/hda1, the hard disk) is accessed in blocks, not single characters. We’re not kernel developers, so we don’t need to worry about that too much.

The Meat Of It

The main part of the -rw-r–r– information is the three sets of three characters: “rw-“, “r–” and “r–” in this example. Of the block, the first character is either “r” to indicate that you can Read the file, or “-” to indicate that you can’t read it. The second is “w” if you can Write to the file, and “-” if you can’t. The third is usually “x” if you can eXecute (run) the file, or “-” if you can’t. (the third can also be “t” or “s”; we’ll come to that in a minute).

So in this case, the file’s owner (“steve”) can read and write, but not execute (rw-). Members of the group (“users”) can read the file, but can’t change it or run it (r–). Anybody who’s not “steve” and not in the “user” group can do the same in this example (r–).

Common Uses

Common sets of permissions are:

600 -rw——- I can read and write it, but nobody else can. (Private files)
640 -rw-r—– I can read and write it, my group can only read it. Others can do nothing. (Semi-shared files)
755 -rwxr-xr-x I can read, write, execute; everyone else can read and execute it, but not change it. (Shared programs)
644 -rw-r–r– I can read and write it, the rest of the world can read it (Shared files)

What’s that left-hand column I threw in there? That’s the other way of thinking about permissions. if “r” is 4, “w” is 2, and “x” is 1, then “rwx” is “4+2+1=7″, “r–” is 4, “rw-” is 4+2=6, and so on. It’s a kind of shorthand.


We set permissions with the chmod command. The first set of three is “u” for “User”, the second is “g” for “Group”, and the last is “o” for “Other”. There’s also “a” for “All”. So “chmod g+rwx” means “add rwx to the second block”, while “chmod a-x” means “take off the x flags for everybody”.

This is easiest to show with examples:

$ ls -l myfile.txt
-rw-r--r-- 1 steve steve 4 2007-01-25 20:39 myfile.txt

#                                           Allow me to eXecute the file: User + eXecute = u+x:
$ chmod u+x myfile.txt
$ ls -l myfile.txt
-rwxr--r-- 1 steve steve 4 2007-01-25 20:39 myfile.txt

#                                           Don't let Others Read the file: Others - Read = o-r:
$ chmod o-r myfile.txt
$ ls -l myfile.txt
-rwxr----- 1 steve steve 4 2007-01-25 20:39 myfile.txt

#                                           Don't the Group Read the file: Group - Read = g-r:
$ chmod g-r myfile.txt
$ ls -l myfile.txt
-rwx------ 1 steve steve 4 2007-01-25 20:39 myfile.txt

#                                           Be specific with numbers: 600 = -rw-------
$ chmod 600 myfile.txt
$ ls -l myfile.txt
-rw------- 1 steve steve 4 2007-01-25 20:39 myfile.txt

#                                           Be specific with numbers: 755 = rwxr-xr-x
$ chmod 755 myfile.txt
$ ls -l myfile.txt
-rwxr-xr-x 1 steve steve 4 2007-01-25 20:39 myfile.txt


Get every new post delivered to your Inbox.