Crafter Hours Development Log

Stream Recording ⋅ Project Code

• Set up basic HTTP server with Guile Fibers
• Established basic route handlers
• Basic page rendering with SXML
• Created record type for the package model

Stream Recording ⋅ Project Code

• Created a basic sqlite schema for the package model
• Implemented basic CRUD operations for package
• Split out package model into its own module
• Updated the package list and detail pages to load from the database
• Attempted to set up a coop REPL server to no avail

Stream Recording ⋅ Project Code

• Figure out how ELPA repositories expose their package information
• Download the archive-contents file and process the contents, determine structure
• Insert initial package details into our sqlite database
• Update package details page to display version and description

Stream Recording ⋅ Project Code

• Fleshed out data model to add package-author and package-source
• Revamped SQL schemas to add tables for package_author, package_authors (relation table), and package_source
• Updated insert-package! procedure to update package and related tables with information from package metadata
• Attempted to update code to import package archives but failed due to technicalities

Stream Recording ⋅ Project Code

• Integrated trev’s fixes for syncing package archives
• Split out database population into a Scheme script sync-archives
• Split out server startup to a script start-server
• Split server handling into the (site server) module
• Split current view templating into (site views *) modules
• Established basic (site db) module to abstract database handling
• Integrated existing scss->css code to generate a cached stylesheet for all pages

Stream Recording ⋅ Project Code

• Set up a REPL-driven workflow for live UI development
• Add routes and views to show all keywords and packages by keyword
• Add SQL queries for keyword IDs by name and packages by keyword
• Improve the site CSS somewhat to make it more readable

Stream Recording ⋅ Project Code

• Created basic routes, views, and schema for the user type
• Create route and form for user login (no cookies yet)
• Created the schema for package tips

Stream Recording ⋅ Project Code

• Draft out basic package tip authoring experience with Datastar (no users, no SQL)
• Write a basic Datastar SSE library

Stream Recording ⋅ Project Code

• Implemented a full login/logout flow with Datastar
• Used jwt tokens to store session details
• Added an authenticated wrapper for abstracting session checks in handlers

Stream Recording (Part 2) ⋅ Project Code

• Finished tip authoring experience with DB interaction
• Added link to tip author profile
• Added initial Markdown formatting for tip text
• Added basic tip voting (not backed by DB yet)

Stream Recording ⋅ Project Code

• Created a DB schema update framework
• Created a table to store applied migrations
• Wrote unit tests to verify that the framework works
• Moved current table initialization queries to first migration

Stream Recording ⋅ Project Code

• Prepared the site for deployment with Guix
• Packaged the current site code in Guix
• Set up a basic server configuration
• Set up a Shepherd service to run the Guile app
• Set up an nginx reverse proxy for hosting the site
• Set up a place to host the database file (under /var/db)

Stream Recording ⋅ Project Code

• Finished password salting algoritm
• Discussed how to handle session token secret, deferred until certbot is set up
• Figured out stable package ID strategy
• Designed the first-pass package sync/update strategy (with disambiguation)

Stream Recording ⋅ Project Code

• Deployed a staging build of the site to a VPS
• Set up certbot to auto-provision a certificate for HTTPS
• Set up the new server-configurations repository

Stream Recording ⋅ Project Code ⋅ Server Configuration

• Converted emacspackages.com repo into a Guix channel
• Updated server-configurations repository to use the new channel
• Ensured that the database is working after running sync-archives.scm manually
• Created a one-shot Shepherd service that runs sync-archives.scm to populate the DB
• Cleaned the manual GUILE_LOAD_PATH manipulation in script files

Stream Recording ⋅ Project Code ⋅ Server Configuration

• Figured out why the site is down
• Made sure initial data sync is working correctly
• Made sure there are no lingering issues from the channel conversion

Stream Recording ⋅ Project Code

• Implemented the user registration page
• Created a basic framework for forms with validation

• Move over more library code to guile-webframe
  • form validation (or larger UI framework?)
  • database migration
  • logger
  • site backend utils (session, password, etc)
• Delete the existing database one last time
• Provide an interface for a user to pick which packages they use
• Ensure tip creation requests are being authenticated
• Ensure that DB migrations are being applied by sync-archives with existing DB

• Switch to PostgreSQL
• Implement package repo data sync/merge?

• Add migrations for:
  • Tip votes
  • Vote count per tip
  • Add a link to user’s Emacs config repo in profile
  • Adding the “I use this” mapping for packages
• Add author details page to list which packages they wrote/maintained
• Implement proper session expiration
• Load session token secret from an environment variable, etc
• Improve the internal API for generating reponses with page bodies
• Flesh out CSS for all pages
• Finish porting ELPA, NonGNU ELPA, and MELPA package details into our database
• Search for packages by name on the site
• Improve package details page to display all information
• Package listing page should use a table with columns
• Define the strategy for periodically updating package details (upserts?)
• Store the last updated time for a package record
• Make sure to log the source of the package (elpa, nongnu-elpa, melpa, melpa-stable)
• Deduplicate package information if they share the same name?
• Sort package listing alphabetically? (query parameter?)
• Need a way to represent package dependencies across package sources but tie them together at the higher level of packages
• Consider changing ID strategy for packages to not use an auto-incremented, numeric ID