Document Component Systems

My goal for this next week is to make this prototype usable. It's almost there honestly, but there a couple elements missing that will make me feel properly comfortable using it on a daily basis.

The first is replacing message passing with function calling, as I mentioned last week. The second is adding a new way to store state information.

Before I get to that though, let's talk about the way I want to use this thing.

At it's core, it's a tool for writing documents that relate to one and other, and programs that interact with them. I want to use it to:

• Keep track of different topics I want to write about, and manage a pipeline from outline -> published work
• Establish an evergreen writing practice
• Keep a meditation log
• Store all my highlights and annotations pulled from my e-reader
• Keep track of my daily routine (in terms of writing different things)

If you think about the kinds of programs neccessary to implement these, they all store and/or read metadata about documents. For example the meditation log would have a document for each meditation session, with the session date, time, and length. The writing pipeline would store different writing projects along with a status, and maybe a "last edited" property.

Other documents, like the daily routine program, would read from those other programs, checking to see whether I meditated, or worked on a draft or whatever. (This is essentially the "self-verifying todo list" I was working on nearly 10 weeks ago, but native to a writing tool).

In the few test programs I've written with the prototype thus far this pattern is already emerging. Right now, programs store their state as JSON objects. And so, when storing metadata for different documents, they store an array of objects, with a property referencing the unique ID of the document.

This works fine from the program's perspective, it can read and write to the JSON object easily, and the logic for how the state is updated (and hence it's shape) is co-located with how it's read. But, this is quite painful for any other program that wants to read this state.

What if instead, we had an explicit API for this common case. A program could register specific "Components", mappings between documents and specific data.

So instead of:

state.push({document: "DOCUMENT ID", date: "2020-11-11", length: 30})

You would get:

Component.set("DOCUMENT ID", "Meditation", {date: '2020-11-11', length: 30})
// OR 
MeditationComponent.set("DOCUMENT ID", {date: '2020-11-11', length: 30})

Functionally, this is the same, but it make a couple things easier, like finding all the components programs have associated with a particular document, or finding all the documents with a particular set of components.

This is quite reminiscent of a pattern in game development called ECS, for Entity Component System. Entities are unique ids, Components are chunks of data associated with those entities, and Systems are programs that run in a loop, iterating over all entities with particular components.

The pattern has performance benefits, but it's interesting to me more for the clean mental model, and the separation of concerns between different components and systems.

It's a real possibility that this kind of architecture is not well suited for the personal documents use-case, and I'll discover that once I'm a thousand documents deep, but for now, it seems like it fits quite nicely to the problem space!