Untitled

I’ve got some premature optimization on the mind. Right now I read and upsert a row for every single text file in the tree. This is fine while the data set is small. What happens as it grows? The performance time will grow linearly at best.

I could minimize the necessary upserts by only upserting files that have changed recently. I could get this list of files with something along the lines of.

find ~/textfiles -type f -newermt "$(stat -c "%y" /yolk.sqlite3)"

This would grab only the text files that have changed since the last time the sqlite3 database was written. This will work as long as the only thing that writes to the database is this upsert mechanism.

But thinking in fractal scenarios—that could change. If it does, it’d be easy enough to swap out a new source of truth in place of $(stat -c "%y" /yolk.sqlite3).

That’s probably a named concept waiting to emerge. Bash scripting resists named concepts.