open(): add O_DIRECTORY flag

This patch adds the O_DIRECTORY flag to sys_open(), which causes the open
to fail with ENOTDIR if the given file is any type of file but a directory.

We need this flag as part of a correct implementation of chdir() (which
should fail on a non-directory file), and it is also required for Linux
compatibility (the O_DIRECTORY flag exists since Linux 2.1.126).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

fs/vfs/vfs_syscalls.cc

@@ -124,6 +124,12 @@ sys_open(char *path, int flags, mode_t mode, struct file **fpp)
 			if (vp->v_type == VDIR)
 				goto out_drele;
 		}
+		if (flags & O_DIRECTORY) {
+		    if (vp->v_type != VDIR) {
+		        error = ENOTDIR;
+		        goto out_drele;
+		    }
+		}
 	}
 
 	vn_lock(vp);