13: EARLY-STAGE PLANNING 2 3When contemplating a Linux kernel development project, it can be tempting 4to jump right in and start coding. As with any significant project, 5though, much of the groundwork for success is best laid before the first 6line of code is written. Some time spent in early planning and 7communication can save far more time later on. 8 9 103.1: SPECIFYING THE PROBLEM 11 12Like any engineering project, a successful kernel enhancement starts with a 13clear description of the problem to be solved. In some cases, this step is 14easy: when a driver is needed for a specific piece of hardware, for 15example. In others, though, it is tempting to confuse the real problem 16with the proposed solution, and that can lead to difficulties. 17 18Consider an example: some years ago, developers working with Linux audio 19sought a way to run applications without dropouts or other artifacts caused 20by excessive latency in the system. The solution they arrived at was a 21kernel module intended to hook into the Linux Security Module (LSM) 22framework; this module could be configured to give specific applications 23access to the realtime scheduler. This module was implemented and sent to 24the linux-kernel mailing list, where it immediately ran into problems. 25 26To the audio developers, this security module was sufficient to solve their 27immediate problem. To the wider kernel community, though, it was seen as a 28misuse of the LSM framework (which is not intended to confer privileges 29onto processes which they would not otherwise have) and a risk to system 30stability. Their preferred solutions involved realtime scheduling access 31via the rlimit mechanism for the short term, and ongoing latency reduction 32work in the long term. 33 34The audio community, however, could not see past the particular solution 35they had implemented; they were unwilling to accept alternatives. The 36resulting disagreement left those developers feeling disillusioned with the 37entire kernel development process; one of them went back to an audio list 38and posted this: 39 40 There are a number of very good Linux kernel developers, but they 41 tend to get outshouted by a large crowd of arrogant fools. Trying 42 to communicate user requirements to these people is a waste of 43 time. They are much too "intelligent" to listen to lesser mortals. 44 45(http://lwn.net/Articles/131776/). 46 47The reality of the situation was different; the kernel developers were far 48more concerned about system stability, long-term maintenance, and finding 49the right solution to the problem than they were with a specific module. 50The moral of the story is to focus on the problem - not a specific solution 51- and to discuss it with the development community before investing in the 52creation of a body of code. 53 54So, when contemplating a kernel development project, one should obtain 55answers to a short set of questions: 56 57 - What, exactly, is the problem which needs to be solved? 58 59 - Who are the users affected by this problem? Which use cases should the 60 solution address? 61 62 - How does the kernel fall short in addressing that problem now? 63 64Only then does it make sense to start considering possible solutions. 65 66 673.2: EARLY DISCUSSION 68 69When planning a kernel development project, it makes great sense to hold 70discussions with the community before launching into implementation. Early 71communication can save time and trouble in a number of ways: 72 73 - It may well be that the problem is addressed by the kernel in ways which 74 you have not understood. The Linux kernel is large and has a number of 75 features and capabilities which are not immediately obvious. Not all 76 kernel capabilities are documented as well as one might like, and it is 77 easy to miss things. Your author has seen the posting of a complete 78 driver which duplicated an existing driver that the new author had been 79 unaware of. Code which reinvents existing wheels is not only wasteful; 80 it will also not be accepted into the mainline kernel. 81 82 - There may be elements of the proposed solution which will not be 83 acceptable for mainline merging. It is better to find out about 84 problems like this before writing the code. 85 86 - It's entirely possible that other developers have thought about the 87 problem; they may have ideas for a better solution, and may be willing 88 to help in the creation of that solution. 89 90Years of experience with the kernel development community have taught a 91clear lesson: kernel code which is designed and developed behind closed 92doors invariably has problems which are only revealed when the code is 93released into the community. Sometimes these problems are severe, 94requiring months or years of effort before the code can be brought up to 95the kernel community's standards. Some examples include: 96 97 - The Devicescape network stack was designed and implemented for 98 single-processor systems. It could not be merged into the mainline 99 until it was made suitable for multiprocessor systems. Retrofitting 100 locking and such into code is a difficult task; as a result, the merging 101 of this code (now called mac80211) was delayed for over a year. 102 103 - The Reiser4 filesystem included a number of capabilities which, in the 104 core kernel developers' opinion, should have been implemented in the 105 virtual filesystem layer instead. It also included features which could 106 not easily be implemented without exposing the system to user-caused 107 deadlocks. The late revelation of these problems - and refusal to 108 address some of them - has caused Reiser4 to stay out of the mainline 109 kernel. 110 111 - The AppArmor security module made use of internal virtual filesystem 112 data structures in ways which were considered to be unsafe and 113 unreliable. This concern (among others) kept AppArmor out of the 114 mainline for years. 115 116In each of these cases, a great deal of pain and extra work could have been 117avoided with some early discussion with the kernel developers. 118 119 1203.3: WHO DO YOU TALK TO? 121 122When developers decide to take their plans public, the next question will 123be: where do we start? The answer is to find the right mailing list(s) and 124the right maintainer. For mailing lists, the best approach is to look in 125the MAINTAINERS file for a relevant place to post. If there is a suitable 126subsystem list, posting there is often preferable to posting on 127linux-kernel; you are more likely to reach developers with expertise in the 128relevant subsystem and the environment may be more supportive. 129 130Finding maintainers can be a bit harder. Again, the MAINTAINERS file is 131the place to start. That file tends to not always be up to date, though, 132and not all subsystems are represented there. The person listed in the 133MAINTAINERS file may, in fact, not be the person who is actually acting in 134that role currently. So, when there is doubt about who to contact, a 135useful trick is to use git (and "git log" in particular) to see who is 136currently active within the subsystem of interest. Look at who is writing 137patches, and who, if anybody, is attaching Signed-off-by lines to those 138patches. Those are the people who will be best placed to help with a new 139development project. 140 141The task of finding the right maintainer is sometimes challenging enough 142that the kernel developers have added a script to ease the process: 143 144 .../scripts/get_maintainer.pl 145 146This script will return the current maintainer(s) for a given file or 147directory when given the "-f" option. If passed a patch on the 148command line, it will list the maintainers who should probably receive 149copies of the patch. There are a number of options regulating how hard 150get_maintainer.pl will search for maintainers; please be careful about 151using the more aggressive options as you may end up including developers 152who have no real interest in the code you are modifying. 153 154If all else fails, talking to Andrew Morton can be an effective way to 155track down a maintainer for a specific piece of code. 156 157 1583.4: WHEN TO POST? 159 160If possible, posting your plans during the early stages can only be 161helpful. Describe the problem being solved and any plans that have been 162made on how the implementation will be done. Any information you can 163provide can help the development community provide useful input on the 164project. 165 166One discouraging thing which can happen at this stage is not a hostile 167reaction, but, instead, little or no reaction at all. The sad truth of the 168matter is (1) kernel developers tend to be busy, (2) there is no shortage 169of people with grand plans and little code (or even prospect of code) to 170back them up, and (3) nobody is obligated to review or comment on ideas 171posted by others. Beyond that, high-level designs often hide problems 172which are only reviewed when somebody actually tries to implement those 173designs; for that reason, kernel developers would rather see the code. 174 175If a request-for-comments posting yields little in the way of comments, do 176not assume that it means there is no interest in the project. 177Unfortunately, you also cannot assume that there are no problems with your 178idea. The best thing to do in this situation is to proceed, keeping the 179community informed as you go. 180 181 1823.5: GETTING OFFICIAL BUY-IN 183 184If your work is being done in a corporate environment - as most Linux 185kernel work is - you must, obviously, have permission from suitably 186empowered managers before you can post your company's plans or code to a 187public mailing list. The posting of code which has not been cleared for 188release under a GPL-compatible license can be especially problematic; the 189sooner that a company's management and legal staff can agree on the posting 190of a kernel development project, the better off everybody involved will be. 191 192Some readers may be thinking at this point that their kernel work is 193intended to support a product which does not yet have an officially 194acknowledged existence. Revealing their employer's plans on a public 195mailing list may not be a viable option. In cases like this, it is worth 196considering whether the secrecy is really necessary; there is often no real 197need to keep development plans behind closed doors. 198 199That said, there are also cases where a company legitimately cannot 200disclose its plans early in the development process. Companies with 201experienced kernel developers may choose to proceed in an open-loop manner 202on the assumption that they will be able to avoid serious integration 203problems later. For companies without that sort of in-house expertise, the 204best option is often to hire an outside developer to review the plans under 205a non-disclosure agreement. The Linux Foundation operates an NDA program 206designed to help with this sort of situation; more information can be found 207at: 208 209 http://www.linuxfoundation.org/en/NDA_program 210 211This kind of review is often enough to avoid serious problems later on 212without requiring public disclosure of the project. 213