File Descriptors

Introduction

A File Descriptor (FD) is a low-level integer handle used by operating systems—particularly Unix-like systems such as Linux and macOS—to uniquely identify an open file or input/output resource. These descriptors serve as references to open files, sockets, pipes, character devices, and even anonymous memory regions.

In most modern operating systems, everything is treated as a file, and file descriptors are the mechanism through which programs interact with the OS-level I/O system.

Core Concept

When a program opens a file (or any stream-like resource), the OS returns a non-negative integer that represents that open file. This integer is the file descriptor, and it’s used in subsequent system calls like read(), write(), or close().

int fd = open("data.txt", O_RDONLY); // fd might be 3
read(fd, buffer, 128);               // read from file using fd
close(fd);                           // release descriptor

The descriptor acts as a pointer to a data structure maintained by the OS, which stores metadata like file position, access mode, and buffer state.

Standard File Descriptors

At program start, Unix-like operating systems automatically allocate three file descriptors:

FD	Name	Purpose
0	`stdin`	Standard input
1	`stdout`	Standard output
2	`stderr`	Standard error

These are commonly redirected or piped in shell environments.

./program > output.txt 2> error.log < input.txt

File Descriptor Table

Each process has its own file descriptor table, which maps integers to file table entries in the kernel. The kernel tracks all open files globally, while the descriptor table gives each process private handles to those resources.

Diagram

User Space (Process)
  ┌────────────┐
  │ FD Table   │
  ├────────────┤
  │ 0 → stdin  │
  │ 1 → stdout │
  │ 2 → stderr │
  │ 3 → socket │
  └────────────┘
         ↓
Kernel Space (Global File Table)

System Calls Using File Descriptors

Call	Description
`open()`	Opens file, returns file descriptor
`read()`	Reads bytes from descriptor
`write()`	Writes bytes to descriptor
`close()`	Closes descriptor
`dup()`	Duplicates a descriptor
`fcntl()`	Manipulates descriptor flags
`poll()` / `select()` / `epoll()`	Waits for I/O readiness on descriptors

Example: Using File Descriptors in C

#include 
#include 

int main() {
    int fd = open("log.txt", O_WRONLY | O_CREAT, 0644);
    write(fd, "Hello, FD!\n", 11);
    close(fd);
    return 0;
}

O_WRONLY: open for writing
O_CREAT: create if it doesn’t exist
0644: file permission bits

File Descriptors vs File Handles

File Descriptors: Used in POSIX/Unix systems (integer values)
File Handles: Used in Windows, more abstracted (often pointers or structures)

File Descriptor Limits

Each process has a limited number of file descriptors it can open simultaneously.

Check Current Limit:

ulimit -n

Increase Limit (Temporary):

ulimit -n 4096

Increase Limit (Permanent):

Edit /etc/security/limits.conf and reboot.

Exceeding this limit can cause:

Too many open files error
Unresponsiveness in I/O-heavy applications (e.g., web servers)

File Descriptor Leak

A file descriptor leak occurs when an application fails to close descriptors after use. Over time, this can exhaust available descriptors, leading to errors.

Example:

def leak():
    while True:
        open('temp.txt')  # never closes

Always use proper closing mechanisms like with statements (Python) or close() in C.

Monitoring File Descriptors

Linux Tools:

lsof – lists open files for processes

lsof -p

procfs – file descriptors in /proc//fd/

ls -l /proc/1234/fd/

strace – trace system calls, including file operations

File Descriptors in Networking

When you create a socket, the OS treats it like a file and assigns a descriptor:

int sockfd = socket(AF_INET, SOCK_STREAM, 0);

This socket can be read() and write() just like a file. This abstraction allows polling tools (select, poll, epoll) to uniformly monitor sockets and files.

Duplicating File Descriptors

`dup()` and `dup2()`

int new_fd = dup(old_fd);       // duplicates fd
dup2(old_fd, 1);                // redirects stdout to old_fd

These are useful in shell-like applications for I/O redirection.

Advanced Topics

1. Non-blocking Descriptors

Use fcntl() to set O_NONBLOCK:

int flags = fcntl(fd, F_GETFL);
fcntl(fd, F_SETFL, flags | O_NONBLOCK);

This allows read() or write() calls to return immediately if no data is available—critical in event-driven systems.

2. Select and Poll

Monitor multiple file descriptors:

fd_set readfds;
FD_ZERO(&readfds);
FD_SET(fd, &readfds);
select(fd+1, &readfds, NULL, NULL, NULL);

Useful for asynchronous I/O across many streams.

Summary

File Descriptors are the backbone of Unix-like I/O systems, providing an elegant and efficient way to abstract input and output—whether it’s reading from a file, writing to a socket, or listening for events.

Understanding how they work is essential for system-level programming, performance tuning, and building scalable applications that interact with files, networks, and external devices.

By managing them properly and avoiding leaks, developers can ensure robust and reliable resource handling across their software.