i-nodes and hard links
Unix-like operating systems store file information in a data structure called i-node table. File names are just a way to reference an i-node and the same i-node may be referenced by more than one name. These associations between file names and i-nodes are called hard links.
When a file is deleted, the corresponding hard link is removed and only if the number of hard links pointing to that i-node is zero, the i-node and the corresponding data blocks are deallocated.
The command ls -li shows the i-node and the number of hard links as the following terminal session shows:
19:36:56 filesystem % echo "hello world" > myfile
19:37:01 filesystem % ls -li myfile
33355238 -rw-r--r-- 1 franciscoalvarez staff 12 17 Nov 19:37 myfile
19:37:12 filesystem % ln myfile other_file
19:37:17 filesystem % ls -li myfile
33355238 -rw-r--r-- 2 franciscoalvarez staff 12 17 Nov 19:37 myfile
19:37:20 filesystem % cat other_file
hello world
19:37:31 filesystem % ls -li other_file
33355238 -rw-r--r-- 2 franciscoalvarez staff 12 17 Nov 19:37 other_file
19:38:17 filesystem % rm myfile
19:38:53 filesystem % ls -li other_file
33355238 -rw-r--r-- 1 franciscoalvarez staff 12 17 Nov 19:37 other_fileHere’s a schematic representation of the filesystem organisation that displays the relationship between i-nodes, hard links and file descriptors
Unlinking files
When a process unlinks (deletes) a file, its mapping is removed from the corresponding folder but the i-node remains allocated until all descriptors pointing to that file are closed. Removing the mapping makes the file invisible for any other process and yet, the original process still can read/write to it.
Here’s a toy example that relies on that behaviour to create a file only visible to two processes. The example features a “producer” writing to a file and a “consumer” reading from the same file and writing the content to standard output. The consumer process unlinks the file soon after opening it, making it invisible to any other potential consumer.
/**
* Create and write random data to a file passed as argument to the application.
*
* Data is taken from a random block of stack memory. The same chunk of data is written multiple times at
* intervals of fixed time
*
*/
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define BUF_SIZE 1024
#define NUM_REPETITIONS 5
#define WRITE_DELAY 3 //seconds
int
main(int argc, char *argv[])
{
char *filename = argv[1];
char buf[BUF_SIZE]; // chunk of random bytes to write to file
// open file
int fd = open(filename, O_WRONLY | O_CREAT | O_EXCL, S_IRUSR | S_IWUSR);
if (fd == -1) {
perror("open error");
exit(EXIT_FAILURE);
}
// write to file
for (int j = 0; j < NUM_REPETITIONS; j++) {
if (write(fd, buf, BUF_SIZE) != BUF_SIZE) {
perror("partial/failed write");
exit(EXIT_FAILURE);
}
printf("blocks written: %d\n", j);
sleep(WRITE_DELAY);
}
// close file desciptor
if (close(fd) == -1) {
perror("close error");
exit(EXIT_FAILURE);
}
exit(EXIT_SUCCESS);
}
/**
* Read data from the file file created by the consumer and write it to standard output.
*
* File is unlinked soon after being opened so that no other consumer can read it
*/
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define BUF_SIZE 1024
#define READ_DELAY 3 //seconds
int
main(int argc, char *argv[])
{
char *filename = argv[1];
char buf[BUF_SIZE];
// open read-only file
int fd = open(filename, O_RDONLY, S_IRUSR | S_IWUSR);
if (fd == -1) {
perror("open error");
exit(EXIT_FAILURE);
}
// remove filename so that no other process can read it
if (unlink(filename) == -1) {
perror("unlink error");
exit(EXIT_FAILURE);
}
// data is read in chunks at intervals of 6 seconds
ssize_t numRead;
while ((numRead = read(fd, buf, BUF_SIZE))) {
for (size_t i = 0; i < numRead; i++) {
// printable chars
if (buf[i] >= 33 && buf[i] < 127) {
printf("%c", buf[i]);
}
}
printf("\n");
sleep(READ_DELAY);
}
if (numRead == -1) {
perror("read error");
exit(EXIT_FAILURE);
}
// close desciptor and file is destroyed
if (close(fd) == -1) {
perror("close error");
exit(EXIT_FAILURE);
}
printf("file descriptor closed\n");
sleep(5);
exit(EXIT_SUCCESS);
}
The following terminal session shows the interaction between producer and consumer through the “invisible file”. The producer creates the file and writes to it. The file is visible until the consumer runs, then the file can’t be displayed by ls but still appears when examining the list of files held open by the process lsof -p $(ps -ef | awk '/out\/consumer/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'.
If any other consumer tried to read the file, it’d get an error: open error: No such file or directory.
What’s more, despite opening the file with the flag O_EXCL (“open for exclusive creation”), another producer could get started and would create a new “invisiblefile”, even though the original “invisiblefile” still exists.
Once both processes end, the i-node and the data blocks corresponding to the file are finally deallocated.
18:51:58 % gcc producer.c -o out/producer
18:52:04 % gcc consumer.c -o out/consumer
18:52:06 % ./out/producer invisiblefile
blocks written: 1
blocks written: 2
^Z
zsh: suspended ./out/producer invisiblefile
18:52:17 % ls invisiblefile
invisiblefile
18:52:23 % ./out/consumer invisiblefile
(5@hvm(5(5hvm(5PP5P5PSP(k(555hvmP55!h(55@5p!0(55@5555555555558555p5`phvm555p`hvm5p55
^Z
zsh: suspended ./out/consumer invisiblefile
18:52:38 % ls invisiblefile
ls: invisiblefile: No such file or directory
18:52:41 % lsof -p $(ps -ef | awk '/out\/consumer/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'
size:2048
inode:33738110
18:52:45 % fg %1
[1] - continued ./out/producer invisiblefile
blocks written: 3
blocks written: 4
blocks written: 5
18:53:05 % lsof -p $(ps -ef | awk '/out\/consumer/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'
size:5120
inode:33738110
18:53:29 % fg %2
[2] - continued ./out/consumer invisiblefile
(5@hvm(5(5hvm(5PP5P5PSP(k(555hvmP55!h(55@5p!0(55@5555555555558555p5`phvm555p`hvm5p55
(5@hvm(5(5hvm(5PP5P5PSP(k(555hvmP55!h(55@5p!0(55@5555555555558555p5`phvm555p`hvm5p55
(5@hvm(5(5hvm(5PP5P5PSP(k(555hvmP55!h(55@5p!0(55@5555555555558555p5`phvm555p`hvm5p55
(5@hvm(5(5hvm(5PP5P5PSP(k(555hvmP55!h(55@5p!0(55@5555555555558555p5`phvm555p`hvm5p55
file descriptor closed
^Z
zsh: suspended ./out/consumer invisiblefile
18:53:47 % lsof -p $(ps -ef | awk '/out\/consumer/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'
18:53:52 % fg %2
[2] - continued ./out/consumer invisiblefile
18:53:56 %Other examples
The same behaviour can be observed with Python and Java applications running on Unix as, ultimately, high level languages rely on the OS facilities to handle files:
import os
import time
path = "invisiblefile"
# need to assign the result to a variable, otherwise file is closed just after calling 'open'
# this behaviour can be verified by calling lsof
f = open(path, 'x')
print(f"file {path} created")
time.sleep(5)
os.unlink(path)
f.write("hello")
f.flush()
print(f"file {path} unlinked")
time.sleep(5)
# ls returns nothing
# BUT
# file is listed in the list of files held open by the process
# lsof -p $(ps -ef | awk '/[u]nlink_file_python/ {print $2}')
# lsof -p $(ps -ef | awk '/[u]nlink_file_python/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'
f.close()
print(f"file {path} closed")
time.sleep(5)
print("end")
import java.io.*;
public class Main {
public static void main(String args[]) throws InterruptedException, IOException {
File invisibleFile = new File("invisiblefile");
invisibleFile.createNewFile();
FileWriter fw = new FileWriter(invisibleFile);
System.out.println("file created");
Thread.sleep(5000);
boolean deleted = invisibleFile.delete();
if(!deleted) {
System.out.println("error deleting file");
System.exit(1);
}
fw.write("hello");
fw.flush();
System.out.println("file deleted");
// 'ls invisiblefile' does not return anything
// but the file is still held open by the process:
// lsof -p $(ps -ef | awk '/[I]nvisibleFile.java/ {print $2}') | awk '/invisiblefile/ {print "size:" $7; print "inode:" $8}'
Thread.sleep(5000);
fw.close();
System.out.println("file closed");
Thread.sleep(5000);
// after closing file, it is completely gone
System.out.println("end");
}
}