life of a NPTL pthread

这是2013年写的一篇旧文，放在gegahost.net上面 http://raison.gegahost.net/?p=91

March 7, 2013

life of a NPTL pthread

Filed under: concurrency,linux,posix — Tags: NPTL, pthread — Raison @ 12:52 pm

(Original Work by Peixu Zhu)

NPTL pthread is the default pthread implementation under current Linux distributions. In concurrent/parallel programming, multiple threading is the basic technology, and knowledge of the NPTL thread’s life will make us clear on using pthread.

1. The nature of the NPTL pthread.

Naturally a NPTL pthread is a light weight pseudo process in Linux. Thus, the maximum of allowed pthread number is limited by the limitation of allowed processes to create in the system.  For a specific user, the number of processes to create is also limited. In system scope, we may get/set the maximum number of process at `/proc/sys/kernel/pid_max`, or by administration command `sysctl`, and get/set maximum number of threads at `/proc/sys/kernel/threads-max`.  For a specific user, we may get/set the maximum number of processes to running with by the command `ulimit   -u`.

Since they’re processes in nature, thus, they are scheduled by the kernel all.

Keep in mind that each thread exhaust a Process ID.

2. The creation of NPTL pthread

A NPTL thread is created by the system routine `__clone`, which allow child process to share parts of its  execution context with the calling process, including memory space, file descriptors, signal handler table, etc. .

The NPTL pthread library offers library function `pthread_create` to create a thread. As calling `pthread_create`, user provide it with necessary arguments like the attribute, the executor function, the argument for executor function. In `pthread_create`,  below preparation works are performed in sequence:

• thread attributes are initialized,  with the argument attribute, and then a stack and thread context are allocated,   the thread context  including TCB, internal locks  etc., are initialized. In fact, the new thread id is the address of allocated stack. Thus, the count of allowed threads in a process is also limited by the memory for the process.
• calling process/thread thread attribute flags and scheduling parameters are copied to the new thread, the join ID which required by `pthread_join` is initialized.
• new thread scheduling policy is determined by the calling process scheduling parameters and argument attribute.
• calling internal routine `create_thread` to create the thread, with the thread context, new thread stack, and attributes as arguments.

In internal function `create_thread`,  system routine `__clone` is called, with demanded arguments to create a thread(in fact a child process). When calling `__clone`,  a fixed helper routine `start_thread` is feed instead of the thread executor function, when `__clone` successfully create the new thread (child process), the helper routine `start_thread` is executed, when `start_thread` finished, it returns the error code as the exiting code of the thread. The new thread executor function is executed in `start_thread`.

3. Cleanup of NPTL pthread

In helper routine `start_thread`, after the thread executor function is executed, the return value is stored in the thread context, then, the routine runs step by step as below:

• run the destructor for the thread-local data.
• if the thread is the last thread in process, terminate the process.
• set the thread to be EXITING.
• if the system support robust mutex, and if there’s robust mutex hooked on the thread, make all of them dead, thus, any sequent access of them will be signalled with EOWNERDEAD.
• recycle the stack of the thread, leaving TCB not freed.
• if the thread is detached,  TCB is freed.
• call system call to terminate the thread.

4. pthread_join and pthread_detach

`pthread_join` is supposed to join the target thread, provided that the target thread is joinable. If the target thread is not terminated yet, the calling thread will waiting for the target thread to be terminated, after the target thread is terminated, it will clean up the TCB of the target thread which is not recycled when the target thread finish running, and then return to the calling thread. If the target thread is already terminated, the calling thread returns from `pthread_join` soon.

`pthread_detach` is supposed to detach a joinable target thread. If the target thread is detached ready, error returned. The routine does not terminate the target thread, it just change `joinid` of the thread context.

A detached thread will recycle its TCB when it is terminated, and a joinable thread will remain its TCB when it is terminated, until a calling of `pthread_join` is called to recycle it. A thread is default to be joinable, unless it is explicitly set to be detached when it is created.