Lots of kerneldoc entries use "example:" or "note:" as section headers.
Until such a time as we can make them use proper markup, make them work as
intended.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The meaning of "leak" can be both "untracked resource allocation" and
"memory content disclosure". This document's use was entirely of the
latter meaning, so avoid the confusion by using the Common Weakness
Enumeration name for this: Information Exposure (CWE-200). Additionally
adds a section on structure randomization.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jani Nikula says:
Jon, this is v2 of [1] and [2], with a considerable amount of polish and
fixes added. We started dogfooding this within drm-intel, and Daniel has
reviewed the lot and contributed a number of fixes, most notably
accurate file and line number references from Sphinx build
errors/warnings to the kernel-doc comments in source code.
We believe this is now in good shape for merging for v4.8. It's all in
my sphinx-for-docs-next branch that you've already looked at; pull
details below.
When this lands in docs-next and we can backmerge to drm, we'll plunge
ahead and convert gpu.tmpl to rst, and have that ready for v4.8. We
think it's best to contribute that via the drm tree, as it'll involve
splitting up the documentation and likely numerous updates to kernel-doc
comments.
I plan to update Documentation/kernel-doc-nano-HOWTO.txt for Sphinx and
rst, obviously converting it to rst while at it.
Design is pretty simple: kernel-doc inserts breadcrumbs with line
numbers, and sphinx picks them up. At first I went with a sphinx
comment, but inserting those at random places seriously upsets the
parser, and must be filtered. Hence why this version now uses "#define
LINEO " since one of these ever escape into output it's pretty clear
there is a bug.
It seems to work well, and at least the 2-3 errors where sphinx
complained about something that was not correct in kernel-doc text the
line numbers matched up perfectly.
v2: Instead of noodling around in the parser state machine, create
a ViewList and parse it ourselves. This seems to be the recommended
way, per Jani's suggestion.
v3:
- Split out ViewList pach. Splitting the kernel-doc changes from the
sphinx ones isn't possible, since emitting the LINENO lines wreaks
havoc with the rst formatting. We must filter them.
- Improve the regex per Jani's suggestions, and compile it just once
for speed.
- Now that LINENO lines are eaten, also add them to function parameter
descriptions. Much less content and offset than for in-line struct
member descriptions, but still nice to know which exact continuation
line upsets sphinx.
- Simplify/clarify the line +/-1 business a bit.
v4: Split out the scripts/kernel-doc changes and make line-numbers
opt-in, as suggested by Jani.
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Opt-in since this wreaks the rst output and must be removed
by consumers again. This is useful to adjust the linenumbers
for included kernel-doc snippets in shinx. With that sphinx
error message will be accurate when there's issues with the
rst-ness of the kernel-doc comments.
Especially when transitioning a new docbook .tmpl to .rst this
is extremely useful, since you can just use your editors compilation
quickfix list to accurately jump from error to error.
v2:
- Also make sure that we filter the LINENO for purpose/at declaration
start so it only shows for selected blocks, not all of them (Jani).
While at it make it a notch more accurate.
- Avoid undefined $lineno issues. I tried filtering these out at the
callsite, but Jani spotted more when linting the entire kernel.
Unamed unions and similar things aren't stored consistently and end
up with an undefined line number (but also no kernel-doc text, just
the parameter type). Simplify things and filter undefined line
numbers in print_lineno() to catch them all.
v3: Fix LINENO 0 issue for kernel-doc comments without @param: lines
or any other special sections that directly jump to the description
after the "name - purpose" line. Only really possible for functions
without parameters. Noticed by Jani.
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Chinese version CodingStyle is a little outdate, it should be updated.
This patch sync with the latest CodingStyle of all changes,
new chapters (chapter 19 and chapter 20) have been translated.
Signed-off-by: Andy Deng <theandy.deng@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
It took me browsing through the source code to determine that I was,
indeed, using the wrong delimiter in my command lines. So I might as
well document it for the next person.
Signed-off-by: Brian Norris <computersforpeace@gmail.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The compilation emits a warning in function ‘snprintf’,
inlined from ‘set_cmdline’ at
../Documentation/mic/mpssd/mpssd.c:1541:9:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:64:10:
warning: call to __builtin___snprintf_chk will always overflow
destination buffer
This was introduced in commit f4a66c2044 ("misc: mic: Update MIC host
daemon with COSM changes") and is fixed by reverting the changes to the
size argument of these snprintf statements.
Cc: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Mike Danese <mikedanese@google.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
There are two sentences in the Sync File documentation where the
english is a little off. This patch is an attempt to fix these.
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
state3 = prototype parsing, so name them accordingly.
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Instead of just forcefully inserting our kernel-doc input and letting
the state machine stumble over it the recommended way is to create
ViewList, parse that and then return the list of parsed nodes.
Suggested by Jani.
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Further up in the state machinery we switch from STATE_NAME to
STATE_DOCBLOCK when we match /$doc_block/. Which means this block of
code here is entirely unreachable, unless there are multiple DOC:
sections within a single kernel-doc comment.
Getting a list of all the files with more than one DOC: section using
$ git grep -c " * DOC:" | grep -v ":1$"
and then doing a full audit of them reveals there are no such comment
blocks in the kernel.
Supporting multiple DOC: sections in a single kernel-doc comment does
not seem like a recommended way of doing things anyway, so nuke the code
for simplicity.
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
[Jani: amended the commit message]
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
With this error output becomes almost readable. The line numbers are
still totally bonghits, but that's a lot harder to pull out of
kerneldoc. We'd essentially have to insert some special markers in the
kernel-doc output, split the output along these markers and then
insert each block separately using
state_machine.insert_input(block, source, first_line)
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reconcile differences between python2 and python3 on dealing with
stdout, stderr from Popen. This fixes "name 'unicode' is not defined"
errors on python3. We'll need to try to keep the extension working on
both python-sphinx and python3-sphinx so we don't need two copies.
Reported-and-tested-by: Marius Vlad <marius.c.vlad@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
If the documentation comment does not have params or sections, the
section heading may leak from the previous documentation comment.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
If there are multiple sections with the same section name, the current
implementation results in several sections by the same heading, with the
content duplicated from the last section to all. Even if there's the
error message, a more graceful approach is to combine all the
identically named sections into one, with concatenated contents.
With the supported sections already limited to select few, there are
massively fewer collisions than there used to be, but this is still
useful for e.g. when function parameters are documented in the middle of
a documentation comment, with description spread out above and
below. (This is not a recommended documentation style, but used in the
kernel nonetheless.)
We can now also demote the error to a warning.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
kernel-doc currently identifies anything matching "section header:"
(specifically a string of word characters and spaces followed by a
colon) as a new section in the documentation comment, and renders the
section header accordingly.
Unfortunately, this turns all uses of colon into sections, mostly
unintentionally. Considering the output, erroneously creating sections
when not intended is always worse than erroneously not creating sections
when intended. For example, a line with "http://example.com" turns into
a "http" heading followed by "//example.com" in normal text style, which
is quite ugly. OTOH, "WARNING: Beware of the Leopard" is just fine even
if "WARNING" does not turn into a heading.
It is virtually impossible to change all the kernel-doc comments, either
way. The compromise is to pick the most commonly used and depended on
section headers (with variants) and accept them as section headers.
The accepted section headers are, case insensitive:
* description:
* context:
* return:
* returns:
Additionally, case sensitive:
* @return:
All of the above are commonly used in the kernel-doc comments, and will
result in worse output if not identified as section headers. Also,
kernel-doc already has some special handling for all of them, so there's
nothing particularly controversial in adding more special treatment for
them.
While at it, improve the whitespace handling surrounding section
names. Do not consider the whitespace as part of the name.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
If a param description spans multiple lines, check any leading
whitespace in the first continuation line, and remove same amount of
whitespace from following lines.
This allows indentation in the multi-line parameter descriptions for
aesthetical reasons while not causing accidentally significant
indentation in the rst output.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Handle whitespace on the first line of param text as if it was the empty
string. There is no need to add the newline in this case. This improves
the rst output in particular, where blank lines may be problematic in
parameter lists.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Move away from field lists, and simply use **strong emphasis** for
section headings on lines of their own. Do not use rst section headings,
because their nesting depth depends on the surrounding context, which
kernel-doc has no knowledge of. Also, they do not need to end up in any
table of contexts or indexes.
There are two related immediate benefits. Field lists are typically
rendered in two columns, while the new style uses the horizontal width
better. With no extra indent on the left, there's no need to be as fussy
about it. Field lists are more susceptible to indentation problems than
the new style.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
The inline member markup allows whitespace lines before the actual
documentation starts. Strip the leading blank lines. This improves the
rst output.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
The use of these is confusing in the script, and per this grep, they're
not used anywhere anyway:
$ git grep " \* [%$&][a-zA-Z0-9_]*:" -- *.[ch] | grep -v "\$\(Id\|Revision\|Date\)"
While at it, throw out the constants array, nothing is ever put there
again.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link "&foo->bar", "&foo->bar()", "&foo.bar", and "&foo.bar()" to the
struct/union/enum foo definition. The members themselves do not
currently have anchors to link to, but this is better than nothing, and
promotes a universal notation.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Let the user use "&union foo" and "&typedef foo" to reference foo. The
difference to using "union &foo", "typedef &foo", or just "&foo" (which
are valid too) is that "union" and "typedef" become part of the link
text.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
It's possible to use &foo to reference structs, enums, typedefs, etc. in
the Sphinx C domain. Thus do not prefix the links with "struct".
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
If the user requests a specific DOC: section by name, do not output its
section title. In these cases, the surrounding context already has a
heading, and the DOC: section title is only used as an identifier and a
heading for clarity in the source file.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Make the output selection a bit more readable by adding constants for
the various types of output selection. While at it, actually call the
variable for choosing what to output $output_selection.
No functional changes.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Make the state machine a bit more readable by adding constants for
parser states and inline member documentation parser substates. While at
it, rename the "split" documentation to "inline" documentation.
No functional changes.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
This script uses pandoc to convert existing DocBook template files to RST
templates. A couple of sed scripts are need to massage things both before
and after the conversion, but the result is then usable with no hand
editing.
[Jani: Change usage to tmplcvt <in> <out>. Fix escaping for docproc
directives. Add support the new kernel-doc extension.]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Read the version and release from the top level Makefile (for use when
Sphinx is invoked directly, by e.g. Read the Docs), but override them
via Sphinx command line arguments in a normal documentation build.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Tell Sphinx where to find the extension, and pass on the kernel src tree
and kernel-doc paths to the extension.
With this, any .rst files under Documentation may contain the kernel-doc
rst directive to include kernel-doc documentation from any source file.
While building, it may be handy to pass kernel-doc extension
configuration on the command line. For example, 'make SPHINXOPTS="-D
kerneldoc_verbosity=0" htmldocs' silences all stderr output from
kernel-doc when the kernel-doc exit code is 0. (The stderr will be
logged unconditionally when the exit code is non-zero.)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.
The syntax for the kernel-doc directive is:
.. kernel-doc:: FILENAME
:export:
:internal:
:functions: FUNCTION [FUNCTION ...]
:doc: SECTION TITLE
Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.
The FILENAME is relative from the kernel source tree root.
The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Add basic configuration and makefile to build documentation from any
.rst files under Documentation using Sphinx. For starters, there's just
the placeholder index.rst.
At the top level Makefile, hook Sphinx documentation targets alongside
(but independent of) the DocBook toolchain, having both be run on the
various 'make *docs' targets.
All Sphinx processing is placed into Documentation/Makefile.sphinx. Both
that and the Documentation/DocBook/Makefile are now expected to handle
all the documentation targets, explicitly ignoring them if they're not
relevant for that particular toolchain. The changes to the existing
DocBook Makefile are kept minimal.
There is graceful handling of missing Sphinx and rst2pdf (which is
needed for pdf output) by checking for the tool and python module,
respectively, with informative messages to the user.
If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but
otherwise gracefully fall back to the Sphinx default theme, with an
informative message to the user, and slightly less pretty HTML output.
Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available),
epubdocs and xmldocs targets. The output documents are written into per
output type subdirectories under Documentation/output.
Finally, you can pass options to sphinx-build using the SPHINXBUILD make
variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose
output from Sphinx.
This is based on the original work by Jonathan Corbet, but he probably
wouldn't recognize this as his own anymore.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Currently we use docproc to figure out which symbols are exported, and
then docproc calls kernel-doc on specific functions, to get
documentation on exported functions. According to git blame and docproc
comments, this is due to historical reasons, as functions and their
corresponding EXPORT_SYMBOL* may have been in different files. However
for more than ten years the recommendation in CodingStyle has been to
place the EXPORT_SYMBOL* immediately after the closing function brace
line.
Additionally, the kernel-doc comments for functions are generally placed
above the function definition in the .c files (i.e. where the
EXPORT_SYMBOL* is) rather than above the declaration in the .h
files. There are some exceptions to this, but AFAICT none of these are
included in DocBook documentation using the "!E" docproc directive.
Therefore, assuming the EXPORT_SYMBOL* and kernel-doc are with the
function definition, kernel-doc can extract the exported vs. not
information by making two passes on the input file. Add support for that
via the new -export and -internal parameters.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
I'm not quite sure why the errors below are happening, but this fixes
them.
Use of uninitialized value in string ne at ./scripts/kernel-doc line 1819, <IN> line 6494.
Use of uninitialized value $_[0] in join or string at ./scripts/kernel-doc line 1759, <IN> line 6494.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
The self-test was updated to cover zero-length strings; the function
needs to be updated, too.
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Fixes: fcfd2fbf22 ("fs/namei.c: Add hashlen_string() function")
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The original name was simply hash_string(), but that conflicted with a
function with that name in drivers/base/power/trace.c, and I decided
that calling it "hashlen_" was better anyway.
But you have to do it in two places.
[ This caused build errors for architectures that don't define
CONFIG_DCACHE_WORD_ACCESS - Linus ]
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Reported-by: Guenter Roeck <linux@roeck-us.net>
Fixes: fcfd2fbf22 ("fs/namei.c: Add hashlen_string() function")
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The HPFS filesystem used generic_show_options to produce string that is
displayed in /proc/mounts. However, there is a problem that the options
may disappear after remount. If we mount the filesystem with option1
and then remount it with option2, /proc/mounts should show both option1
and option2, however it only shows option2 because the whole option
string is replaced with replace_mount_options in hpfs_remount_fs.
To fix this bug, implement the hpfs_show_options function that prints
options that are currently selected.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
