Concurrent writing to a log file from many processes

In mimicing MacOS's terminal's open command or Window's terminal's start , the comments of this answer suggest appending stdout and stderr to ~/.xsession-errors , something like ( bash ):

alias open='&>>~/.xsession-errors xdg-open'

The issue I forsee with this is race conditions. lsof ~/.xsession-errors shows 22 processes have the file open for writing. How does one prevent two processes writing to the same offset in ~/.xsession-errors ?

asked Feb 19, 2017 at 5:35 31.5k 38 38 gold badges 154 154 silver badges 247 247 bronze badges

What exactly do you mean by "race conditions"? The order stdout and stderr are written? Or are you afraid both may be written "at once", and the output will be swallowed? That doesn't happen, AFAIK.

Commented Feb 19, 2017 at 7:16

I aim to prevent two processes both having an error to write, opening the flie, seeking to the same offset (EOF), and then writing at the same position.

Commented Feb 19, 2017 at 13:03

Hm, lsof shows that on my system, .xsession-errors is already opened by a dozen or so processes. No idea how and if they handle this .

Commented Feb 19, 2017 at 16:14 @dirkt It's handled poorly based on the However section of this answer. Commented Feb 20, 2017 at 7:09

2 Answers 2

When a file is opened in append mode, the OS guarantees that all writes take place at the end. So the data from one writer will not overwrite the data from another writer.

This only applies if the file is opened in append mode, i.e. with >> in the shell. If the creator of the file opens it with > then that guarantee won't apply and it's possible to have this sequence:

Process 1: >out ; now at position 0
Process 2: >>out ; now at position 0
Process 1: write hello , now at position 6
Process 2: write world , this is written at position 6 and process 2 is now at position 12
Process 1: write oops , this is written at position 6 because process 1's file position has not changed.

On Debian (since 2001 or thereabouts), the file .xsession-errors is created by `/etc/X11/Xsession and it is opened in append mode, so everything is fine:

 exec >>"$ERRFILE" 2>&1

I don't know if that's the case on all distributions that log to ~/.xsession-errors .

As long as everybody opens the file in append mode, all output will be present. The output may be fragmented however. In practice, small enough writes to a regular file are atomic. Anything less than 512B should be small enough everywhere, and I think Linux guarantees more than that¹. So each log line should appear intact, even with multiple concurrent writers, assuming that the writers use line-buffered output and that the lines are not overly long.

¹ _{Note that POSIX does not guarantee anything except for pipes.}