init: frontmatter support

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## Version 3.1.0
+
+Added frontmatter feature:
+
+- `doc_location` keyword allows to externalize documentation.
+
+See our [full specification](./doc/frontmatter.md).
+
+by @hsjobeki;
+
+in https://github.com/nix-community/nixdoc/pull/114.
+
 ## Version 3.0.2
 
 Avoid displaying arguments when a doc-comment is already in place.

Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "nixdoc"
-version = "3.0.2"
-authors = ["Vincent Ambo <[email protected]>", "asymmetric"]
+version = "3.1.0"
+authors = ["Vincent Ambo <[email protected]>", "asymmetric", "hsjobeki"]
 edition = "2021"
 description = "Generate CommonMark from Nix library functions"
 
@@ -12,6 +12,9 @@ serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 textwrap = "0.16"
 clap = { version = "4.4.4", features = ["derive"] }
+serde_yaml = "0.9.33"
+gray_matter = "0.2.6"
+relative-path = "1.9.2"
 
 [dev-dependencies]
 insta = "1.36.1"

README.md

@@ -18,17 +18,20 @@ It is important to start doc-comments with the additional asterisk (`*`) -> `/**
 
 The content of the doc-comment should conform to the [Commonmark](https://spec.commonmark.org/0.30/) specification.
 
-### Example
+### Examples
+
+#### Example: simple but complete example
 
 The following is an example of markdown documentation for new and current users of nixdoc.
 
-> Sidenote: Indentation is automatically detected and should be consistent across the content. 
-> 
+> Sidenote: Indentation is automatically detected and should be consistent across the content.
+>
 > If you are used to multiline-strings (`''`) in nix this should be intuitive to follow.
 
+`simple.nix`
 ````nix
 {
-  /** 
+  /**
     This function adds two numbers
 
     # Example
@@ -60,10 +63,39 @@ The following is an example of markdown documentation for new and current users
 
 > Note: Within nixpkgs the convention of using [definition-lists](https://www.markdownguide.org/extended-syntax/#definition-lists) for documenting arguments has been established.
 
+## Example: using frontmatter
+
+In some cases, it may be desirable to store the documentation in a separate Markdown file.
+Nixdoc supports frontmatter, which allows a separate Markdown file to be referenced within the doc comment.
+
+`fontmatter.nix`
+````nix
+{
+  /**
+    ---
+    doc_location: ./docs.md
+    ---
+  */
+  add = a: b: a + b;
+}
+````
+
+`./docs.md`
+````markdown
+# Add
+
+A simple function that adds two numbers.
+````
+
+> Note: Frontmatter works only with the newer doc-comments `/** */`.
+
+See our [frontmatter specification](./doc/frontmatter.md) for further information.
+
+## Legacy support
 
-## Custom nixdoc format (Legacy)
+### Custom nixdoc format
 
-You should consider migrating to the newer format described above.
+You should consider migrating to the newer format described above. The format described in this section will be removed with the next major release.
 
 See [Migration guide](./doc/migration.md).
 

doc/frontmatter.md

@@ -0,0 +1,120 @@
+# Frontmatter
+
+This document is the specification for custom metadata within doc-comments.
+
+It should only apply to doc-comments (`/** */`) and not to regular code comments to ensure a limited scope.
+
+## Why frontmatter is needed (sometimes)
+
+Sometimes it is desireable to extend the native doc-comment functionality. For that user-scenario, frontmatter can be optionally used.
+
+Frontmatter is the de-facto standard of adding document specific meta tags to markdown. [1](https://docs.github.com/en/contributing/writing-for-github-docs/using-yaml-frontmatter) [2](https://jekyllrb.com/docs/front-matter/) [3](https://starlight.astro.build/reference/frontmatter/)
+
+it can be useful for:
+
+- Enriching documentation generation.
+- Maintaining References.
+- Metadata inclusion.
+- Allow better Handling of documentation edge cases.
+
+## Detailed design
+
+Frontmatter is defined using `key`-`value` pairs, encapsulated within triple-dashed lines (---).
+
+While there is no strict specification for frontmatter formats, YAML is commonly preferred.
+Although JSON could also be used alternatively.
+
+Only `key`s from the list of available keywords can be used.
+
+The `value` is any valid yaml value. Its type and semantics is specified by each keyword individually.
+
+Example:
+
+```nix
+{
+  /** 
+    ---
+    tags:
+      - foo
+      - bar
+    ---
+  */
+  foo = x: x;
+}
+```
+
+In this example `key` is `tags` and `value` is `List("foo", "bar")`.
+
+See also: [yaml specification](https://yaml.org/spec/1.2.2/) for how to use YAML.
+
+## Keywords
+
+### `doc_location`
+
+Rules:
+
+1. The `doc_location` keyword can be used to use content from another file INSTEAD of the doc-comments content.
+
+2. The value must be given as a path relative to the current file.
+
+3. Using this directive does not process any content inside the file, using it as-is.
+
+```nix
+{
+  /** 
+  ---
+  doc_location: ./path.md
+  ---
+  */
+  foo = x: x;
+}
+```
+
+`path.md`
+```md
+some nice docs!
+```
+
+In this example, the `doc_location` directive fetches content from `./path.md` treating it as if it were directly within the doc-comment.
+This allows tracking the reference between the source position and the markdown file, in case of external documentation.
+
+#### Design decision: Relative vs Absolute paths
+
+This section justifies the previous decision why absolute paths are not allowed.
+
+<details>
+<summary>Should absolute paths be allowed?</summary>
+
+- (+) When the docs are entirely elsewhere, e.g. `doc/manual/..`, a relative path would have to be `../../..`, very ugly
+  - (-) If only relative paths are allowed, encourages moving docs closer to the source,
+    Makes changing documentation easier.
+    - For the nix-build, adjustments which files are included in the derivation source may be needed.
+  - (-) With only relative paths, it's more similar to NixOS module docs
+- (-) Makes it very confusing with where absolute paths are relative to (build root, git root, `.nix` location, etc.)
+- (-) We can still allow absolute paths later on if necessary
+- (-) Relies on a Git repository and git installed
+- (-) It's unclear where absolute paths are rooted
+  - (+) Could use a syntax like `$GIT_ROOt/foo/bar`
+    - (-) Not a fan of more custom syntax
+
+**Decision**: Not supported by now.
+
+This outcome was discussed in the nix documentation team: https://discourse.nixos.org/t/2024-03-21-documentation-team-meeting-notes-114/41957.
+
+</details>
+
+## Error handling
+
+Any issues encountered during the processing of frontmatter — be it syntax errors, invalid paths, or unsupported keywords—should result in clear, actionable error messages to the user.
+
+## Extensibility
+
+The initial set of keywords is intentionally minimalistic, focusing on immediate and broadly applicable needs.
+
+When extending this document we ask our contributors to be tooling agnostic, such that documentation wont't rely on any implementation details.
+This approach ensures that the documentation remains independent of specific implementation details. By adhering to this principle, we aim to create a resource that is universally applicable
+
+## Future work
+
+This proposal represents a foundational step towards more versatile and extendable reference documentation.
+As we move forward, we'll remain open to adapting and expanding this specification to meet emerging needs and leverage community insights.