various tweaks

Bad Soren let these sit around for a while, so I don't know what they
are or what they were for anymore!
This commit is contained in:
Soren I. Bjornstad 2021-09-10 10:59:41 -05:00
parent 90ea549c36
commit f4e8888101
5 changed files with 82 additions and 14 deletions

View File

@ -20,6 +20,8 @@ You can use the following builders in your configuration file out of the box:
.. autofunction:: compile_html_file .. autofunction:: compile_html_file
.. autofunction:: editionify
.. autofunction:: export_public_tiddlers .. autofunction:: export_public_tiddlers
.. autofunction:: new_output_folder .. autofunction:: new_output_folder
@ -58,21 +60,28 @@ Custom builders
=============== ===============
If the existing builders don't cover something you're hoping to do to build a product, If the existing builders don't cover something you're hoping to do to build a product,
you can write your own directly within your config file. you can write your own builder as a Python function directly within your config file.
As an example, let's suppose that we want to publish our wiki to an S3 bucket As an example, let's suppose that we want to publish our wiki to an S3 bucket
fronted by CloudFront on Amazon Web Services. fronted by CloudFront on Amazon Web Services.
We can work with AWS using the ``aws`` CLI, We can work with AWS using the ``aws`` CLI,
if we've set that up on our computer. if we've set that up on our computer
We first write a builder function in our ``tzk_config.py``, (we could also use the ``boto3`` Python library,
which is cleaner but slightly longer
and requires us to muck around with ``pip``,
so we'll do it through the CLI in this example).
We first write a builder function decorated with :func:`builders.tzk_builder() <tzk_builder>`
in our ``tzk_config.py``,
anywhere above the ``products`` dictionary: anywhere above the ``products`` dictionary:
.. code-block:: python .. code-block:: python
from pathlib import Path
import subprocess import subprocess
@builders.tzk_builder @builders.tzk_builder
def publish_to_aws(target_uri: str, cloudfront_distribution_id: str): def publish_to_aws(target_uri: str, cloudfront_distribution_id: str):
"Publish output to Amazon S3"
source_folder = Path(builders.build_state['public_wiki_folder']) / "output" source_folder = Path(builders.build_state['public_wiki_folder']) / "output"
# Sync the output folder to S3, deleting any files that have been removed. # Sync the output folder to S3, deleting any files that have been removed.
subprocess.call(("aws", "s3", "sync", "--delete", source_folder, target_uri)) subprocess.call(("aws", "s3", "sync", "--delete", source_folder, target_uri))
@ -85,7 +94,10 @@ The :func:`new_output_folder()` builder
populates this ``public_wiki_folder`` attribute early in the default build process, populates this ``public_wiki_folder`` attribute early in the default build process,
so that it contains the path to the temporary wiki that build steps happen within. so that it contains the path to the temporary wiki that build steps happen within.
Then we add a call to this builder within the list for this product, The first line of the docstring, in this case ``"Publish output to Amazon S3"``,
is used as the description of the step in output, with any period at the end removed.
Then we add a call to this builder within the list of steps for this product,
with whatever parameters we like: with whatever parameters we like:
.. code-block:: python .. code-block:: python
@ -107,11 +119,26 @@ not ``builders.publish_to_aws``,
since this builder is located directly within the config file since this builder is located directly within the config file
rather than in the external ``tzk.builders`` module that comes with tzk. rather than in the external ``tzk.builders`` module that comes with tzk.
.. tip::
If you do something in a builder that needs to be cleaned up later,
like creating a temporary file,
assign a cleanup function to the ``cleaner`` attribute of your builder:
::
def aws_cleanup():
# nothing really needs to be cleaned up, but if it did we'd do it here
pass
publish_to_aws.cleaner = aws_cleanup
Cleanup functions will run after all steps are done,
regardless of whether the build succeeds or fails.
Shell commands Shell commands
============== ==============
If a builder seems like overkill for your use case, If a custom builder seems like overkill
or you're not familiar with Python,
you can also run simple shell commands using the :func:`shell()` builder. you can also run simple shell commands using the :func:`shell()` builder.
Our AWS example would look like this: Our AWS example would look like this:
@ -129,10 +156,12 @@ Our AWS example would look like this:
], ],
} }
Notice the need to include quotes within the string in :func:`builders.shell`; Notice the need to include quotes within the string in :func:`shell`;
the same quoting rules as when running shell commands directly apply. the same quoting rules as when running shell commands directly apply.
Also notice that we had to access the compiled HTML file from Also notice that we had to access the compiled HTML file from
``output/public_site``, since we can no longer use the ``build_state`` dictionary. ``output/public_site``,
since we can no longer refer to the ``build_state`` dictionary
to learn where the temporary output folder is.
Paths are relative to the private wiki's root directory Paths are relative to the private wiki's root directory
(the directory containing the ``tiddlywiki.info`` file) (the directory containing the ``tiddlywiki.info`` file)
while builders are running. while builders are running.

View File

@ -216,7 +216,7 @@ class BuildCommand(CliCommand):
for idx, step in enumerate(steps, 1): for idx, step in enumerate(steps, 1):
# Explain what we're doing. Use first line of the builder's docstring # Explain what we're doing. Use first line of the builder's docstring
# as a summary, if present. # as a summary, if present.
if hasattr(step, '__doc__'): if hasattr(step, '__doc__') and step.__doc__ is not None:
short_description = step.__doc__.strip().split('\n')[0].rstrip('.') short_description = step.__doc__.strip().split('\n')[0].rstrip('.')
print(f"tzk: Step {idx}/{len(steps)}: {short_description}") print(f"tzk: Step {idx}/{len(steps)}: {short_description}")
else: else:

View File

@ -12,6 +12,7 @@ information about the defined products without actually running any build steps.
from contextlib import contextmanager from contextlib import contextmanager
import functools import functools
import json
import os import os
from pathlib import Path from pathlib import Path
import re import re
@ -124,8 +125,8 @@ def new_output_folder():
""" """
Create a new temporary folder to hold intermediate steps of the product being built. Create a new temporary folder to hold intermediate steps of the product being built.
The path to this temporary folder will be stored in the 'public_wiki_folder' The path to this temporary folder will be stored in the ``public_wiki_folder``
key of the builders.build_state dictionary. Future build steps can access key of the ``builders.build_state`` dictionary. Future build steps can access
the work in progress here. A cleaner is registered to delete this folder the work in progress here. A cleaner is registered to delete this folder
when all steps complete, so any finished product should be copied out by a when all steps complete, so any finished product should be copied out by a
later build step once it is complete. later build step once it is complete.
@ -421,7 +422,7 @@ def replace_private_people(initialer: Callable[[str], str] = None) -> None:
@tzk_builder @tzk_builder
def set_tiddler_values(mappings: Dict[str, str]) -> None: def set_tiddler_values(mappings: Dict[str, str]) -> None:
""" """
Set the 'text' field of selected config or other tiddlers to arbitrary new values. Set the ``text`` field of selected config or other tiddlers to arbitrary new values.
This can be used to make customizations that can't easily be done with feature This can be used to make customizations that can't easily be done with feature
flags or other wikitext solutions within the wiki -- for instance, changing the flags or other wikitext solutions within the wiki -- for instance, changing the
@ -524,3 +525,33 @@ def shell(shell_command: str) -> None:
info(f"Command exited with return code 0:\n{output}") info(f"Command exited with return code 0:\n{output}")
else: else:
info(f"Command exited with return code 0 (no output).") info(f"Command exited with return code 0 (no output).")
@tzk_builder
def editionify(target_folder: str, description: str) -> None:
"""
Copy the output folder to a target location and set its edition description.
This generates a TiddlyWiki edition based on the temporary output. By
copying it into an appropriate location or setting the environment variable
:envvar:`TIDDLYWIKI_EDITION_PATH` to the parent directory of the edition's folder,
it's possible to quickly generate new TiddlyWikis based on the edition
"template" (``tiddlywiki --init EDITION_NAME``, where the ``EDITION_NAME`` is the
name of the folder).
:param target_folder: The folder to copy the output folder to.
Note that the output folder will be merged into this folder
if it exists.
:param description: The description of this edition to use in the new edition's
``tiddlywiki.info`` file.
"""
shutil.copytree(
build_state['public_wiki_folder'],
target_folder,
dirs_exist_ok=True
)
with (Path(target_folder) / "tiddlywiki.info").open("r") as f:
tinfo = json.load(f)
tinfo['description'] = description
with (Path(target_folder) / "tiddlywiki.info").open("w") as f:
json.dump(tinfo, f)

View File

@ -53,7 +53,7 @@ _public_export_filt = r"""
-[prefix[$:/temp]] -[prefix[$:/temp]]
-[prefix[$:/state]] -[prefix[$:/state]]
-[prefix[$:/sib/StorySaver/saved]] -[prefix[$:/sib/StorySaver/saved]]
-[prefix[$:/sib/checkify/]] -[prefix[$:/checkify/]]
-[[$:/config/zettelkasten/Build/KillPhrases]] -[[$:/config/zettelkasten/Build/KillPhrases]]
""" """

View File

@ -105,7 +105,13 @@ def _init_tw(wiki_name: str) -> None:
except FileExistsError: except FileExistsError:
pass pass
with pushd(wiki_name): with pushd(wiki_name):
subprocess.check_call((_tw_path(), "--init")) old_edition_path = os.environ.get('TIDDLYWIKI_EDITION_PATH')
os.environ['TIDDLYWIKI_EDITION_PATH'] = str(Path(__file__).parent / "editions")
try:
subprocess.check_call((_tw_path(), "--init", "tzk"))
finally:
if old_edition_path:
os.environ['TIDDLYWIKI_EDITION_PATH'] = old_edition_path
def _add_filesystem_plugins(wiki_name: str) -> None: def _add_filesystem_plugins(wiki_name: str) -> None:
@ -148,7 +154,9 @@ def _initial_commit() -> None:
print("tzk: Committing changes to repository...") print("tzk: Committing changes to repository...")
git.exec("add", "-A") git.exec("add", "-A")
git.exec("commit", "-m", "Initial commit") output = git.read("commit", "-m", "Initial commit")
# Print just a summary since there are going to be a lot of files.
print('\n'.join(output.split('\n')[0:2]))
def install(wiki_name: str, tw_version_spec: str, author: Optional[str]): def install(wiki_name: str, tw_version_spec: str, author: Optional[str]):