./blog.sh

Five posts covered the shape of this release. Here is the rest of it, a paragraph each.

This site's approved comments and its post counters are two small JSON files, refreshed by a cron job every half hour and uploaded with the rest of the site.

Through 1.4, a tick that ran while the archive was not there would narrow both files to nothing and upload them. An absent or empty content directory answers with an empty list and no error, and an empty list reads exactly like "every announced post was deleted". The live site lost every approved comment and every counter at once. Exit 0, nothing said, nothing mailed.

The archive lives on a synced volume. A volume that has not mounted yet, or a working copy whose files are still arriving, is not an unusual Tuesday.

Not being able to tell is a reason to keep everything

That sentence is the fix, and it now runs in two tiers.

If the content directory is not there — or if the archive reads as empty while the previous files held anything at all — nothing is narrowed, and the run says why.

An importer's worst failure is not losing a post. It is publishing one.

1.5 went through all eleven import adapters and the machinery underneath them looking for exactly that, and found it three times over, in three different files, made the same way each time.

An archive is the whole account

A Mastodon export is not your public timeline. The outbox holds your followers-only posts and your direct messages beside the public ones — and every standalone toot in it was being written as published. On the archive this was measured against that is 141 posts of 2,548, of which 132 are direct messages, each given its own page, its own sitemap entry and its own item in the feed. Nothing in the run's summary mentioned it.

The scope was in the export all along, exactly where ActivityPub puts it. Public in to is public. Public in cc is unlisted — a real address, out of the listings. Followers-only and direct become drafts: that keeps the author's words in the archive and off the site, which is the only reading of "not public" that cannot publish something by accident.

A listing card used to be the whole post. The engine rendered every block of it into the page, and the stylesheet clipped the result at 500 pixels.

On the real front page that meant fourteen cards carrying between 794 and 2,616 pixels of content and showing 500. Of the page's 50,659 characters, 49,312 sat inside the clip. Thirty-three of its 154 focusable elements could be reached with the keyboard while being invisible on screen — links a sighted mouse user could not see and a keyboard user could not skip.

1.5 cuts the card before it is written. The stylesheet hides nothing.

That page went from 89,268 bytes of HTML to 41,155. Its largest card, 7,463 characters, is now 2,566. And "read more" appears exactly when something did not fit, rather than sitting under a card that was already showing everything it had.

The budget is in pixels, not characters

Until now this engine could show you ten posts at a time and nothing else. /page/128/ is a real address on a large archive, and it tells you nothing about whether you have landed in 2009 or in 2014 — pagination is anchored from the oldest post, so the number moves under you every time something is published. The only complete list of anything lived in the terminal.

1.5 adds two pages that read the archive instead of walking it.

/archive/ — the shape of the whole thing

A row per year, with a strip of twelve months beside it. A month is shaded in one of four steps rather than simply lit or dark, because "has posts / has none" is not the question worth answering: on the reference archive a month holds anywhere from one post to eighty-seven, and drawing those the same would hide the one thing a map is for. /archive/<year>/ is one line per post, by month.

Two levels and no more. A third would be some 280 pages of individual months — a great deal of building for a question nobody asked.

For most of this project's life, a post without a title had two names. Its own page called it by its date. Everything else — the browser tab, the link card somebody shares, every item in the feed — called it burtiky-opekame-hipstamatic-oggl-jane.

That is not a corner case. On the archive this engine is measured against, 2,754 posts of 4,418 carry no title. Sixty-two per cent of a site, going out into the world under an address.

A slug is made out of the words. It is not the words.

A slug is a machine's answer to "where does this live": lowercased, stripped of diacritics, joined with hyphens, cut to the first eight words. Every property that makes it a good address makes it a bad name. It has no capitals, no punctuation, and no idea where the sentence it was cut from was going.

1.5 takes the name from the same place the slug came from — the post's own opening — and stops in a better place.

Where it stops

1.5 is the release where the site learned to say what it holds. The release is on GitHub.

A post that never had a title used to be called by its address — on this project's own reference archive that was 62 % of it. Now it is called by its own opening words, everywhere except its heading, which stays the date it has been since 1.3.

Two new pages come with it: /archive/, a map of the whole site in two levels, and /tag/, every subject it has ever written about. Both are built from the posts themselves — nothing to configure and nothing to keep up to date.

Two things a post can now decide for itself: where its teaser stops (//--more--//, asked for in issue #35), and how much of it a listing card shows — cards are cut before they are written rather than drawn in full and hidden with CSS. And a code block carries a copy button, asked for by an operator running a blog of terminal how-tos.

1.4 went through four adversarial audit rounds before the tag: fleets of independent reviewers over real archives — real exports, a real 4400-post site, real deploy targets — each told to break the engine rather than confirm it. The findings numbered over a hundred. Every one was reproduced before it was fixed, and every fix is pinned by a test that demonstrably fails on the code it fixed.

The pattern that kept repeating

The uncomfortable lesson: each round's fixes were where the next round found its bugs. A fix for a comment-parsing rule deleted a hundred lines of somebody's config; a fix for that hid a widget's answers; the recovery advice written to save posts could destroy one. Four times in a row, the newest code was the most dangerous code in the tree — which is exactly why the last round audited nothing but the fixes themselves. It caught three regressions of mine before any user could.

Eighteen files

The queue can now swap two scheduled posts that share a slug across two years — with their media and their edit history riding along, because those are keyed by year and slug exactly like the post file. That sentence is short; making it true was most of a release cycle. The interesting part is not the swap. It is what happens when the power goes out in the middle of one.

The budget is zero

A move is several renames, and a crash can land between any two of them. The rule 1.4 holds to: whatever the moment, a crash may cost you the move — never a post. Files step aside under parking names before anything writes; a failed write puts everything back, byte for byte; a hard kill that strands a parked file leaves it where check finds it and says truthfully what it is.

The advice is part of the system

./blog.sh check reads the whole archive and says what is wrong: media a post asks for and hasn't got, links to addresses nothing answers at, two posts that would be served at one address. Through 1.3 that was where it ended — a list of findings and an exit code, with the fixing left to you and a text editor.

--repair walks it with you

check --repair is the checker's other half: for each finding it offers the one repair that finding allows — an old address written into the target post's redirect_from, a relative link rewritten to the address it means, an orphaned file moved to the trash. Nothing is applied without a keypress, nothing is deleted outright, and a finding with no obvious answer — two posts colliding, an image only a human can judge — is shown and passed over rather than guessed at.

It proved itself before it shipped: on a real archive it took 238 findings down to 42, and the 21 repairs it proposed matched what a person had already decided by hand, one for one.

--json says everything