My source of truth for this website is plain text files stored in iCloud. I read the text files into structured data and shove that derived data into a sqlite3 database. I idempotently upsert the data so I can keep the existing database and it’s active connections in place. The database is long lived, but I think of it as disposable. I am always able to recreate it based on the source of truth in plain text files.
The publishing flow basically distills down to this:
# Create a new piece of content stored in iCloud
echo "# Hello world" > ~/Documents/chicken/index.md
# Copy the content onto my remote server "egg"
rsync ~/Documents/chicken egg:/chicken`
# Parse into structured data in a relational database
ssh egg "~/bin/upsert /chicken /yolk.sqlite"
This works fine. But there is potential for errors on the horizon.
Basically, all the downsides to manually managing plain text files. Making sure the file path follows my conventions when rapidly publishing content is the biggest thing that seems like I’ll mess up.
Rapidly publishing content is not something I’ve been doing, but it’s something I’ve been working towards. I plan to build out a POSSE strategy for broadcasting content into the social media silos. Committing (loosely defined) my data will have one-time side-effects of broadcasting to third party services. And I’ll need a source of truth for tracking the state of that broadcast side-effect.
Another potential for errors: how instantaneous and reliable is iCloud syncing? I cannot rsync from my iPhone to the remote server. I need to use a Shortcut from my iPhone to tell my Mac to rsync to the remote server. Do I need to cook up some kind of safety check that iCloud is all synced up? Even if I did build such a check, are there race conditions that could lead to false positives? How do I know things are ready to be committed? Is the desire to write in iA Writer on my phone worth all this complexity?
All this uncertainty tempts me to build a web based flow where I can commit directly to my server. The point of commitment is clear. The request/response cycle brings certainty that the commit happened. But it breaks my source of truth. Now the meant-to-be-ephemeral database holds data that does not exist in iCloud text files.
Reading the text files into structure data was simple enough. Dumping the structured data back into a plain text file would be just as simple. Reliably loadind and dumping data without loss is a thrill. I could rsync in two directions and make it work. The data set is small.
But I’ve lived through some horror stories of production software with data flowing in two directions. Data should flow in one direction if at all possible.
Do I actually need to store my writing in iA Writer. Is retro-fitting my previous micropub workflow into my new world a more viable strategy? I could imagine a robust flow that stores my truth in text files, even if that’s not the source of truth.
POST /micropub from anywhere in the world commits my new entry into the databasersyncGit and iCloud are not good together. So it seems like there’s a necessary point of commitment where I crossover from iCloud (centralized wall garden that I love but can’t control) to Git (decentralized open source that I love and can control).
Basically, I’ve created my own chicken egg problem. Merry Christmas to me!
Published: 2024-12-29