Chugging along, looping with repls

Okay a first take at this new "worklog" style newsletter.

What's happening with Hyperlink.academy

This week we "pushed to master" with the new hyperlink.academy site. You can take a sneak peak if you like but stuff there is still extremely under-construction so please be generous.

Pushing to master is a step before even a public alpha but it still feels worth celebrating. I remember how good it felt the first time I published anything related to fathom online. What's interesting is that it never stopped feeling good. Every little change I'd push online was a thrill, and it is to this day.

Next we're going to run a small course called (of course) The Meta Course. A course for creating courses. The plan is to get some cool folks together and try out an experimental structure to help us develop some interesting ways of learning some interesting things. Along the way we'll get to test out the site, all the integration work with discourse, and just get a feel for what learning looks like in this new system we're exploring.

I'm also excited to start refactoring now that I've laid out the vast majority of the pages and features of the site. There's a lot of redundant code that I'm just itching to delete and pull into functions.

There's also a lot of opportunities to make things easier to change and iterate on. For example, I plan on pulling out all of the copytext from the functions defining layout and store them in one easily modifiable and discoverable place in files.

What I planned to do with fancynote

Last week I set myself a collection of loosely related tasks for my experimental note taking language: Create an error system, implement a filter/query function, and start defining the mapping between fancynote and html in fancynote itself.

What actually I did

Almost none of that. Instead I took a step back and started laying down the groundwork to hopefully get there

Cleaning up

I simplified the parser function, removing a full 8 lines (more impressively stated as 26%)

function generateSubtree(input:Token[]): [Atom | List, number] {
  let AST:AST = []
  if(input[0].value !== "[") return [{type: 'atom', value: input[0].value}, 1] // If it's not a list just return
  // if it is a list, process every element
  for(let i = 1; i < input.length;){
    let token = input[i].value
    if(token === ']') return [{type: 'list', children: AST}, i+1] // if the list is closed return
    let [block, length]= generateSubtree(input.slice(i)) // other wise process the rest of the tokens, either a single token or a list
    AST.push(block) // add that to the list you've got
    i += length // move forward by however many tokens you processed
  }
  throw Error('input does not terminate')
}
export function parser(input:string):AST {
  let tokens = lexer(input)
  let output:AST = []
  let position = 0
  while(position < tokens.length) {
    let [node, lengthParsed] = generateSubtree(tokens.slice(position))
    output.push(node)
    position += lengthParsed
  }
  return output
}

I also simplified the lexer, the program which handles creating the list of tokens this one needs, by removing it's wierd newlines handling. This was to enable printing out the program again (more on that later)

Back to text-editors

I also started experimenting with an emacs mode for writing fancynote. I lasted a whole week before trying to do more text-editor-y stuff lol.

The emacs mode looks like this:

(provide 'fancynote)
(defconst fancynote-mode-syntax-table (let ((table (make-syntax-table)))
                                        (modify-syntax-entry ?\[ "(]" table)
                                        (modify-syntax-entry ?\] ")]" table)
                                        table))
(add-to-list 'auto-mode-alist '("\\.fn\\'" . fancynote-mode))
(define-derived-mode fancynote-mode nil "Fancynote"
  "major mode for editing fancynote files."
  :syntax-table fancynote-mode-syntax-table
  (set (make-local-variable 'indent-line-function) 'indent-relative)
  (display-line-numbers-mode)
  (auto-fill-mode)
  )

Mostly it just sets it up so files I open that end with .fn work the way I expect. Maybe it'll be a good basis for experiments but I'm not feeling to excited about trying to make sense of elisp.

Instead I'm going to do a broader survey of my options in building text-editing interfaces. I started taking some notes here.

Setting up a REPL

I also started the bare minimum of a REPL, which I talked about last time as one of the ways I hope to make fancynote as rich an experience as something like Notion or Roam.

let command = ''
rl.prompt()
rl.on('line', (line)=>{
  command += '' + line
  try {
    let ast = parser(command)
    let text = astToString(ast)
    console.log(text)
    command = ''
    rl.prompt()
  } catch(e) {
    command += '\n'
    rl.prompt(true)
  }
})

It's a small loop. It asks the user for input with rl.prompt() then if that input terminates it parses it, and then turns that parsed AST back into a string and returns it. Astute among you may have noticed that this does absolutely nothing.

However, writing astToString was more complicated than you might think. When you parse a file you lose a lot of contextual information. Things like spaces and newlines aren't important for the structure of your program so the parser just skips over them. Turning the data-structure of your program back into a human readable string, means you have to keep enough information to do so.

Anyways, this REPL is going to be the basis for future experiments with computational work!

What I plan to do:

This week I wanted to set up a proper time for writing. It didn't happen so I'm going to try again.