1The cgroup freezer is useful to batch job management system which start 2and stop sets of tasks in order to schedule the resources of a machine 3according to the desires of a system administrator. This sort of program 4is often used on HPC clusters to schedule access to the cluster as a 5whole. The cgroup freezer uses cgroups to describe the set of tasks to 6be started/stopped by the batch job management system. It also provides 7a means to start and stop the tasks composing the job. 8 9The cgroup freezer will also be useful for checkpointing running groups 10of tasks. The freezer allows the checkpoint code to obtain a consistent 11image of the tasks by attempting to force the tasks in a cgroup into a 12quiescent state. Once the tasks are quiescent another task can 13walk /proc or invoke a kernel interface to gather information about the 14quiesced tasks. Checkpointed tasks can be restarted later should a 15recoverable error occur. This also allows the checkpointed tasks to be 16migrated between nodes in a cluster by copying the gathered information 17to another node and restarting the tasks there. 18 19Sequences of SIGSTOP and SIGCONT are not always sufficient for stopping 20and resuming tasks in userspace. Both of these signals are observable 21from within the tasks we wish to freeze. While SIGSTOP cannot be caught, 22blocked, or ignored it can be seen by waiting or ptracing parent tasks. 23SIGCONT is especially unsuitable since it can be caught by the task. Any 24programs designed to watch for SIGSTOP and SIGCONT could be broken by 25attempting to use SIGSTOP and SIGCONT to stop and resume tasks. We can 26demonstrate this problem using nested bash shells: 27 28 $ echo $$ 29 16644 30 $ bash 31 $ echo $$ 32 16690 33 34 From a second, unrelated bash shell: 35 $ kill -SIGSTOP 16690 36 $ kill -SIGCONT 16690 37 38 <at this point 16690 exits and causes 16644 to exit too> 39 40This happens because bash can observe both signals and choose how it 41responds to them. 42 43Another example of a program which catches and responds to these 44signals is gdb. In fact any program designed to use ptrace is likely to 45have a problem with this method of stopping and resuming tasks. 46 47In contrast, the cgroup freezer uses the kernel freezer code to 48prevent the freeze/unfreeze cycle from becoming visible to the tasks 49being frozen. This allows the bash example above and gdb to run as 50expected. 51 52The cgroup freezer is hierarchical. Freezing a cgroup freezes all 53tasks beloning to the cgroup and all its descendant cgroups. Each 54cgroup has its own state (self-state) and the state inherited from the 55parent (parent-state). Iff both states are THAWED, the cgroup is 56THAWED. 57 58The following cgroupfs files are created by cgroup freezer. 59 60* freezer.state: Read-write. 61 62 When read, returns the effective state of the cgroup - "THAWED", 63 "FREEZING" or "FROZEN". This is the combined self and parent-states. 64 If any is freezing, the cgroup is freezing (FREEZING or FROZEN). 65 66 FREEZING cgroup transitions into FROZEN state when all tasks 67 belonging to the cgroup and its descendants become frozen. Note that 68 a cgroup reverts to FREEZING from FROZEN after a new task is added 69 to the cgroup or one of its descendant cgroups until the new task is 70 frozen. 71 72 When written, sets the self-state of the cgroup. Two values are 73 allowed - "FROZEN" and "THAWED". If FROZEN is written, the cgroup, 74 if not already freezing, enters FREEZING state along with all its 75 descendant cgroups. 76 77 If THAWED is written, the self-state of the cgroup is changed to 78 THAWED. Note that the effective state may not change to THAWED if 79 the parent-state is still freezing. If a cgroup's effective state 80 becomes THAWED, all its descendants which are freezing because of 81 the cgroup also leave the freezing state. 82 83* freezer.self_freezing: Read only. 84 85 Shows the self-state. 0 if the self-state is THAWED; otherwise, 1. 86 This value is 1 iff the last write to freezer.state was "FROZEN". 87 88* freezer.parent_freezing: Read only. 89 90 Shows the parent-state. 0 if none of the cgroup's ancestors is 91 frozen; otherwise, 1. 92 93The root cgroup is non-freezable and the above interface files don't 94exist. 95 96* Examples of usage : 97 98 # mkdir /sys/fs/cgroup/freezer 99 # mount -t cgroup -ofreezer freezer /sys/fs/cgroup/freezer 100 # mkdir /sys/fs/cgroup/freezer/0 101 # echo $some_pid > /sys/fs/cgroup/freezer/0/tasks 102 103to get status of the freezer subsystem : 104 105 # cat /sys/fs/cgroup/freezer/0/freezer.state 106 THAWED 107 108to freeze all tasks in the container : 109 110 # echo FROZEN > /sys/fs/cgroup/freezer/0/freezer.state 111 # cat /sys/fs/cgroup/freezer/0/freezer.state 112 FREEZING 113 # cat /sys/fs/cgroup/freezer/0/freezer.state 114 FROZEN 115 116to unfreeze all tasks in the container : 117 118 # echo THAWED > /sys/fs/cgroup/freezer/0/freezer.state 119 # cat /sys/fs/cgroup/freezer/0/freezer.state 120 THAWED 121 122This is the basic mechanism which should do the right thing for user space task 123in a simple scenario. 124