Picjumbo com HNCK4432

Problem:

There are certain aspects of programming that lay dormant or un-touched for sometime, some problem areas that evade your daily grasp until a suite of side-affects in your environment appear to have an inexplicable randomness. One of these problems is that of concurrent file access, that is shared files.

I saw the worst of this come to light one day, when a partially migrated legacy application was running concurrently to a new one (let’s call it version 1). The aspect that was missed here, was that while locking was happening within each individual application, both applications were manipulating the same file. That is the maintainers and developers did not for see that individual applicaiton threads were protected from concurrent access, across application there was no such fail-safe..

So to better understand the problem, let’s try out some scenarios.

File Locking Work Across Applications

A common misconception is that an Exclusive Lock will lock across applications (aka. globally across the OS). This is not true. One might assume a Operating System Level lock would do so, but imagine the sequence of issues that would then ensure including starvation / deadlocks. Here is a simple proof of concept:

Snip #1:

filename = "virtualdomains"
File.open(filename, 'r') do |f|
  success = f.flock(File::LOCK_EX)
  puts "Do I have a lock? #{success == 0 ? true : false}" 
  puts "Holding lock..."
  sleep(500)
end

 

In the snippet above, I am placing an exclusive lock on the “virtual domains” file (File::LOCK_EX) and thereafter calling sleep. While that is running I open another terminal and run the following piece of code:

Snip #2

echo "Hello World" >> virtual domains

That echo would have absolutely no problem appending to our exclusively locked virtual domains file, and thus proving to us, that file system locks (at least in Linux) does not work. But that is not a problem, it’s design. Remember, an operating system is not a database, it is not managing records or rows, nor should it “generally” speaking.

Another valuable lesson here, is that opening a file with the attribute of ‘w’ or ‘w+’ will lead to truncation before the file can be written too. 

File Locking Work Within The Same Application Across Threads

If however you wanted to lock within an application, then so much the easier – provided again, you retain the knowledge of nothing external to application (or process) that will be manipulating your resource (with a similar or likely timing).

Snip #3

filename = "/virtualdomains"
File.open(filename, 'w') do |f|
  old_pos = 0
  f.each do |line|
    f.pos = old_pos
    f.print line.gsub(/^dane/, "#dane")
    old_pos = f.pos
  end
  sleep(500)
end

In the snippet above if we execute this code and then open another terminal and try again, we will immediately see that our exclusive locks now holds true and our code will block, till the sleep has expired.

An important lesson here, is that Ruby provides LOCK_NB (non-blocking) which will return false for such a scenario, as demonstrated below.

Snip #4

def read_to_write(filename)
  wrote_contents = true
  tries ||= 3
  File.open(filename, 'r+') do |file|
    begin
      locked = file.flock(File::LOCK_NB | File::LOCK_EX)
      raise "Failed to get exclusive lock on #{filename}" if !locked
      entries = File.readlines(filename).collect(&:strip)
      entries.collect! { |entry| entry.gsub(/^my$/, "#my") }
      file << entries.join("\n")
    rescue Exception => e
      retry unless (tries -= 1).zero?
      wrote_contents = false
    end
  end
  wrote_contents
end

How does File Locking Work in Regards to Reads?

As a last point of discussion it’s important to realise that no matter the lock or how you would implement it (r+ attribute -> read then write), you  cannot block a read from happening, and as such, one should be well aware prior to assuming it’s state before a write (without a good concurrency/locking mechanism at play).