Open more than RLIMIT_NOFILE file descriptors

DOVES Vulnerability V-00103

DOVES Project
Computer Security Laboratory
Department of Computer Science
University of California at Davis



Description

Brief summary: The check for the maximum number of file descriptors allocated to a process is incorrect.

Detailed description: The semantics of the UNIX operating system require that file descriptors be allocated in increasing order, beginning with 0. Some kernels use this to simplify the check for the maximum number of open file descriptors. Call this limit RLIMIT_NOFILE. When the file descriptor RLIMIT_NOFILE-1 is assigned, the kernel concludes that the maximum number of file descriptors has been opened. (Note that in general, more than RLIMIT_NOFILE can be opened. The hard limit is different. But RLIMIT_NOFILE is intended to be the limit for a single process.)

The problem is that the dup2()(2) does not restrict the naming of file descriptors to be underRLIMIT_NOFILE . So, the following sequence of code opens a file descriptor that does not count in the number of open file descriptors that the kernel uses:

/* assume fd is an open file descriptor */
if (dup2(fd, RLIMIT_NOFILE+1) < 0)
        perror("dup2");
	return;
}
close(fd);

Components: kernel

Operating system(s): Linux kernels 2.2,14 and pre 2.4;The attacker can open more file descriptors than the system specifications allow.

How to detect:

  1. Look in the kernel code for the dup2() routine. If the check for the number of the new file descriptor is
    if (newfd >= NR_OPEN)
    you are vulnerable. (NR_OPEN is the hard limit described earlier.)

How to fix:

  1. Change the kernel code check to
    if (newfd >= current->rlim[RLIMIT_NOFILE].rlim_cur)
    This checks the new file descriptor against the process resource limit, not the hard limit.

Other information:


Keywords

file descriptor, violable limit, kernel, resource allocation

Cataloguing Information

PA Classification:

RISOS Classification:

Davis Classification:


Exploits

Attacks: See Doves exploit #104.


Related Information


History

Who reported it: Olaf Kirch in Letter titled "circumvent RLIMIT_NOFILE" on Mon, 24 Jul 2000 16:50:03 +0200: reported the problem and gave an exploit; Linus Torvalds in response to Olaf's letter on Mon, 24 Jul 2000 18:57:39 PDT: suggested the fix


Revision #1

  1. Matt Bishop on 7/29/2000
    Initial entry


Send email to doves@cs.ucdavis.edu

Department of Computer Science
University of California at Davis
One Shields Ave.
Davis, CA 95616-8562


Page created August 28, 2000 at 16:41:45 GMT

Dove images © 1999-2000 www.barrysclipart.com