jjkiers/tzk
1
0
Fork 0
Code Issues Pull Requests Releases Activity

fully document builders

Browse Source
This commit is contained in:
Soren I. Bjornstad
2021-08-27 12:52:58 -05:00
parent 85b836144c
commit 5b836b3cff
9 changed files with 254 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

137
docs/builders.rst Normal file
View File

@@ -0,0 +1,137 @@
========
Builders
========
.. currentmodule:: tzk.builders
*Builders* are small executable chunks that together can be linked into a useful build process.
Products are built by applying builders in sequence.
Please see the existing ``products`` dictionary and associated comments
in the :ref:`config file <Configuring tzk>` for how these are specified.
Builders included with tzk
==========================
You can use the following builders in your configuration file out of the box:
.. autofunction:: check_for_kill_phrases
.. autofunction:: compile_html_file
.. autofunction:: export_public_tiddlers
.. autofunction:: new_output_folder
.. autofunction:: publish_wiki_to_github
.. autofunction:: replace_private_people
.. autofunction:: require_branch
.. autofunction:: require_clean_working_tree
.. autofunction:: save_attachments_externally
.. autofunction:: say_hi
.. autofunction:: set_tiddler_values
.. autofunction:: shell
Builder helper functions
========================
These helper functions, also defined in :mod:`tzk.builders`,
are intended for use with any custom builders you create.
.. autofunction:: tzk.builders::info
.. autofunction:: tzk.builders::stop
.. autodecorator:: tzk.builders::tzk_builder
Custom builders
===============
If the existing builders don't cover something you're hoping to do to build a product,
you can write your own directly within your config file.
As an example, let's suppose that we want to publish our wiki to an S3 bucket
fronted by CloudFront on Amazon Web Services.
We can work with AWS using the ``aws`` CLI,
if we've set that up on our computer.
We first write a builder function in our ``tzk_config.py``,
anywhere above the ``products`` dictionary:
.. code-block:: python
import subprocess
@builders.tzk_builder
def publish_to_aws(target_uri: str, cloudfront_distribution_id: str):
source_folder = Path(builders.build_state['public_wiki_folder']) / "output"
# Sync the output folder to S3, deleting any files that have been removed.
subprocess.call(("aws", "s3", "sync", "--delete", source_folder, target_uri))
# Clear the CDN cache so the new version is available immediately.
subprocess.call(("aws", "cloudfront", "create-invalidation",
"--distribution-id", cloudfront_distribution_id, "--paths", "/*"))
``builders.build_state`` is a dictionary that is preserved across all build steps.
The :func:`new_output_folder()` builder
populates this ``public_wiki_folder`` attribute early in the default build process,
so that it contains the path to the temporary wiki that build steps happen within.
Then we add a call to this builder within the list for this product,
with whatever parameters we like:
.. code-block:: python
:emphasize-lines: 5
products = {
'public': [
[...]
builders.compile_html_file(externalize_attachments=True),
publish_to_aws("s3://my_target_uri", "MY_DISTRIBUTION_ID"),
],
}
Since we've parameterized this builder,
we can easily use it multiple times if we want,
for instance within different products.
Note that we say just ``publish_to_aws``,
not ``builders.publish_to_aws``,
since this builder is located directly within the config file
rather than in the external ``tzk.builders`` module that comes with tzk.
Shell commands
==============
If a builder seems like overkill for your use case,
you can also run simple shell commands using the :func:`shell()` builder.
Our AWS example would look like this:
.. code-block:: python
:emphasize-lines: 5-7
products = {
'public': [
[...]
builders.compile_html_file(externalize_attachments=True,
output_folder="output/public_site/"),
builders.shell("aws s3 sync --delete output/public_site s3://my_target_uri"),
builders.shell("aws cloudfront create-invalidation --distribution-id MY_DISTRIBUTION_ID --paths '/*'"),
],
}
Notice the need to include quotes within the string in :func:`builders.shell`;
the same quoting rules as when running shell commands directly apply.
Also notice that we had to access the compiled HTML file from
``output/public_site``, since we can no longer use the ``build_state`` dictionary.
Paths are relative to the private wiki's root directory
(the directory containing the ``tiddlywiki.info`` file)
while builders are running.

65
docs/configuration.rst
View File

@@ -2,14 +2,61 @@
Configuring tzk
===============
Change into a directory you'd like to use as your Zettelkasten repository.
The directory should be empty, so you'll probably want to create a new one, e.g.:
Basic setup
===========
```
$ mkdir my_zettelkasten
$ cd my_zettelkasten
```
1. Change into a directory you'd like to use as your Zettelkasten repository.
The directory should be empty, so you'll probably want to create a new one, e.g.:
::
Run ``tzk init``.
This will install TiddlyWiki,
set up a Git repository,
$ mkdir my_zettelkasten
$ cd my_zettelkasten
2. Run ``tzk init``.
This will create a tzk configuration file,
install TiddlyWiki to this folder,
and set up a Git repository.
3. When ``init`` has completed successfully,
open the ``tzk_config.py`` in your favorite text editor.
Read the comments and make any changes you would like.
See the :ref:`Builders` section of this documentation
for more information about builders --
but you'll most likely want to get started with your wiki now
and worry about builds once you actually have some content to build!
4. Run ``tzk listen`` and confirm that you can access your wiki.
Committing
==========
Many people find that carefully designing atomic Git commits
when editing a TiddlyWiki
is difficult and not all that useful,
so the ``tzk commit`` command is made available
to quickly stage, commit, and (if you wish) push all changes in the repository in one go.
To enable pushes,
add a new Git remote (e.g., ``git remote add origin https://github.com/you/YourRepository``)
and set the ``commit_remote`` option in your tzk config to the remote name
(here, ``origin``).
You can selectively skip pushing for a particular commit
with the ``--local`` switch to ``tzk commit``.
Environment
===========
If you'd like to be able to run ``tzk`` from any directory,
rather than having to change into the directory of your tzk repository,
set the ``TZK_DIRECTORY`` environment variable on your system
to its full path.
If the current directory contains a ``tzk_config.py`` file,
the current directory will still be preferred to the ``TZK_DIRECTORY`` directory.
.. note::
``TZK_DIRECTORY`` is not honored when calling ``tzk init``.
Otherwise tzk would prioritize the ``TZK_DIRECTORY`` over the current directory
since the current directory doesn't contain a config file yet,
and it would be impossible to initialize a second tzk repository.

3
docs/index.rst
View File

@@ -10,7 +10,8 @@ for Soren Bjornstad's Zettelkasten edition of TiddlyWiki.
:caption: Contents
installation
operations
configuration
builders
Indices and tables

29
docs/installation.rst
View File

@@ -7,10 +7,9 @@ tzk itself
==========
On most systems, tzk may be installed directly from pip at a command line:
::
```
$ pip install tzk
```
$ pip install tzk
If you don't have Python 3.6 or greater on your computer,
you'll need to install it first.
@@ -22,11 +21,10 @@ you'll be able to use ``pip`` to install tzk as described above.
To check your work, run ``tzk --version``;
you should see a version number rather than an error,
something like:
::
```
$ tzk --version
1.0.0
```
$ tzk --version
1.0.0
.. _exhaustive guide: https://realpython.com/installing-python/#how-to-install-python-on-macos
@@ -37,23 +35,22 @@ Dependencies
In order to set up your Zettelkasten,
you'll also need ``npm`` and ``git``.
You can check if they're installed like this:
::
```
$ npm --version
7.20.6
$ git --version
git version 2.32.0
```
$ npm --version
7.20.6
$ git --version
git version 2.32.0
Your versions will likely be a little different by the time you read this.
As long as you get a version number, you're good;
As long as you get a version number rather than an error, you're good;
tzk does not use any features of either tool that require bleeding-edge versions.
If you don't have **npm**,
If you don't have **NPM**,
follow step 1 of the Node.js installation instructions in the `TiddlyWiki documentation`_.
You can skip all the remaining steps -- tzk takes care of that part for you.
If you don't have **git**,
If you don't have **Git**,
follow the steps in the `Installing Git`_ section of Pro Git.
.. _TiddlyWiki documentation: https://tiddlywiki.com/#Installing%20TiddlyWiki%20on%20Node.js

20
docs/operations.rst
View File

@@ -1,20 +0,0 @@
========
Builders
========
.. automodule:: tzk.builders
:members: check_for_kill_phrases, compile_html_file, export_public_tiddlers, new_output_folder, publish_wiki_to_github, replace_private_people, require_branch, require_clean_working_tree, save_attachments_externally, say_hi, set_tiddler_values, shell
Builder helper functions
========================
These helper functions, also defined in :mod:`tzk.builders`,
are intended for use with any custom builders you create.
.. autofunction:: tzk.builders::info
.. autofunction:: tzk.builders::stop
.. autodecorator:: tzk.builders::tzk_builder

11
tzk/__main__.py
View File

@@ -45,13 +45,13 @@ class CommitCommand(CliCommand):
"-m", "--message",
metavar="MSG",
help="Commit message to use.",
default=(cm().commit_message or "checkpoint")
default=cm().commit_message,
)
parser.add_argument(
"-r", "--remote",
metavar="REMOTE",
help="Name of the configured Git remote to push to.",
default=(cm().commit_remote or "origin"),
help="Name of the Git remote to push to.",
default=cm().commit_remote,
)
parser.add_argument(
"-l", "--local",
@@ -70,8 +70,7 @@ class CommitCommand(CliCommand):
f"'{cm().commit_require_branch}' branch to commit.")
git.exec("add", "-A")
git.exec("commit", "-m", args.message)
if not args.local:
if git.rc("commit", "-m", args.message) == 0 and args.remote and not args.local:
git.exec("push", args.remote)
@@ -270,7 +269,7 @@ def chdir_to_wiki():
os.chdir(cm().wiki_folder)
except FileNotFoundError:
fail(f"Tried to change directory into the wiki_folder '{cm().wiki_folder}' "
f"specified in your config file, but that directory does not exist.")
f"specified in your config file, but that directory does not exist.")
if not os.path.exists("tiddlywiki.info"):
fail(f"After changing directory into {cm().wiki_folder} per your config file: "

39
tzk/builders.py
View File

@@ -1,19 +1,13 @@
"""
*Builders* are small executable chunks that together can be linked into a useful build
process. Aside from two helper functions (:func:`stop()` and :func:`info()`)
to fail the build and display an informational message, respectively, all other
functions in this module are builders.
process.
Builders are decorated with ``@tzk_builder``, a synonym for
:func:`_lazy_evaluable()`, which causes them to be lazy-evaluated: that is,
when they're initially called in the configuration, instead of running the
function and returning its result, a zero-argument function with all of the
arguments wrapped up is returned, to be run at a later time. This allows the
configuration file to be read at any time to retrieve information about the
defined products without actually running any build steps.
You can write and use custom builders right within your config file
by decorating them with ``@builders.tzk_builder``.
Builders are decorated with :func:`tzk_builder`, which causes them to be
lazy-evaluated: that is, when they're initially called in the configuration,
instead of running the function and returning its result, a zero-argument
function with all of the arguments wrapped up is returned, to be run at a later
time. This allows the configuration file to be read at any time to retrieve
information about the defined products without actually running any build steps.
"""
from contextlib import contextmanager
@@ -139,7 +133,10 @@ def new_output_folder():
assert 'public_wiki_folder' not in build_state
build_state['public_wiki_folder'] = tempfile.mkdtemp()
new_output_folder.cleaner = lambda: shutil.rmtree(build_state['public_wiki_folder'])
def new_output_folder_cleaner():
if 'public_wiki_folder' in build_state:
shutil.rmtree(build_state['public_wiki_folder'])
new_output_folder.cleaner = new_output_folder_cleaner
@tzk_builder
@@ -232,7 +229,7 @@ def save_attachments_externally(attachment_filter: str = "[is[image]]",
on each image tiddler to point to this new location.
For the latter step, use the
``externalize_attachments``, ``attachment_filter``, and ``canonical_uri_template``
parameters to the ``compile_html_file`` step.
parameters to the :func:`compile_html_file` step.
:param attachment_filter: The tiddlers to be saved to the external folder;
by default, ``[is[image]]``.
@@ -429,11 +426,13 @@ def set_tiddler_values(mappings: Dict[str, str]) -> None:
flags or other wikitext solutions within the wiki -- for instance, changing the
subtitle or what buttons are visible. It's also used to implement feature flags
in the first place by changing the ``$:/config/sib/CurrentEditionPublicity``
tiddler to ``public``, so the minimal functional wiki will use:
```python
{'$__config_sib_CurrentEditionPublicity.tid': 'public',}
```
tiddler to ``public``, so at minimum, the build of a public wiki should use:
.. code-block:: python
builders.set_tiddler_values({
'$__config_sib_CurrentEditionPublicity.tid': 'public',
})
:param mappings: A dictionary whose keys are tiddler names
and whose values are the values to be inserted

15
tzk/config.py
View File

@@ -14,6 +14,15 @@ from tzk.util import fail
class ConfigurationManager:
def __init__(self):
self.initialize_cm()
def __getattr__(self, attr):
if self.conf_mod is None:
return None
else:
return getattr(self.conf_mod, attr, None)
def initialize_cm(self):
self.config_path = Path.cwd()
for child in sorted(self.config_path.iterdir()):
@@ -29,12 +38,6 @@ class ConfigurationManager:
# no config file
self.conf_mod = None
def __getattr__(self, attr):
if self.conf_mod is None:
return None
else:
return getattr(self.conf_mod, attr, None)
def has_config(self) -> bool:
return self.conf_mod is not None

18
tzk/default_config.py
View File

@@ -19,10 +19,12 @@ wiki_folder = "wiki"
### COMMITTING ####
# Default commit message to use with 'tzk commit'.
# You can always use 'tzk commit -m' to use a different message on the fly.
commit_message = "daily checkpoint"
commit_message = "checkpoint"
# Git remote to push changes to when you run 'tzk commit'.
commit_remote = "origin"
# If you never want to push changes, set this to the empty string ("").
commit_remote = ""
#commit_remote = "origin"
# Uncomment if you want to abort 'tzk commit' if you're not on a specific branch.
#commit_require_branch = "master"
@@ -67,12 +69,13 @@ _public_export_filt = r"""
# you can write your own, or you can run arbitrary shell commands
# using a 'builders.shell("my shell command here"),' builder.
products = {
# The default configuration contains a single product for building a public wiki.
# The default configuration contains a single product for building a public wiki;
# use 'tzk build public' to build it. You can add as many products as you want.
'public': [
builders.new_output_folder(),
builders.export_public_tiddlers(_public_export_filt),
builders.export_public_tiddlers(export_filter=_public_export_filt),
builders.replace_private_people(),
builders.set_tiddler_values({
builders.set_tiddler_values(mappings={
'$__config_sib_CurrentEditionPublicity.tid': 'public',
'$__config_sib_IsPublicEdition.tid': 'false',
'$__config_DefaultSidebarTab.tid': '$:/sib/SideBar/Explore',
@@ -97,4 +100,9 @@ products = {
builders.save_attachments_externally(),
builders.compile_html_file(externalize_attachments=True),
],
# If you want a second product, add it like this:
#'secondproduct': [
# builders.new_output_folder(),
# ... and so on ...
#],
}