Digital Garden

Design tutorials as opinionated usage guides

Mon, 09 May 2022 20:59:00 +0000

Back in the day, Smartly.io hired a lot of Flowdock (now defunct chat application, like Slack) employees back in the days, for example my manager. According to my understanding, this lead to flourishing usage of the Flowdock application. One of the, alleged, reasons for this was that those who had both designed and implemented the product were able to guide and educate other employees within Smartly on how to use the application to its fullest.

I have also noticed that I would gain a lot from an opinionated guide on how to use a program/application. Right now, as I am working on my Master’s thesis, I’m using Zotero for managing research. Zotero allows you to take both notes separately from a document (which then can be mapped to other notes with a “related-to” relationship) or straight into a PDF (annotations and highlights). Balancing these two features are a bit tricky, and I would love for someone at Zotero to explain to me how to “actually” use the program.

Note: I do realize that having an opinionated guide might result in some usage data being lost, as users are now following your guide more heavily. This could be mitigated by presenting the usage guide as a “design document” where the core workflow is presented. How to actually implement this is not something I’ve given enough thought.

Digital Garden

Sun, 31 Oct 2021 11:59:00 +0000

A digital garden is an intersection between a notebook and a blog.

I use Neuron for my notetaking to keep my ideas/notes in a Zettelkasten. Sometimes I write stuff that could be nice to share, other times not. However, Neuron supports tagging notes, so I got the idea that I could simply tag those notes I would like to share.

I got to work and got something up with the following setup:

Neuron can query your notes for tags with --tag.
- There also exists a Docker image for neuron.
Hugo is a static site generator that I’ve wanted to try out for a while.
- I’ve been eyeing the terminal theme for a while.
Python-Markdown has a plugin for reading YAML metadata of Markdown files.
- Use this to manipulate Neuron metadata into something compatible with Hugo.
- Wrote a Python script to do this.
  - Reads Neuron query data from stdin and manipulates metadata.
Deploy everything with Ansible.
- Automation would be great in the future, but since I sync my notes with Syncthing for realtime reading between my devices I’ll have to find a file watcher.

Source code:

Limitations:

No easy automation as of now.
This setup does not support Neuron zettel links.

Troubleshooting NixOS and ZFS

Fri, 22 Oct 2021 20:50:00 +0000

Incident on 21/10/2021. NixOS ZFS partition on laptop reported 0 available bytes on all pools.

Teardown:

Tried to setup Android Studio with emulator capabilities on my NixOS machine
- Installed a bunch of packages.
- Nix the package manager sucks up a lot of storage if not properly scrubbed.
- Installed and built so much on my machine that suddenly nothing worked anymore.
du reported 100% full on everything.
- Started getting so weird that my command line prompt started printing random stuff since it couldn’t create a tmp file.
- I could not collect garbage with nix-collect-garbage since it couldn’t create a lock file.
Rebooted the machine, wrong call.
- Most services enabled at boot reported failure to start.
- Got to login screen.
  - Non-existent users very denied entry.
  - Existing users (root and my cookie) passed the login, but due to no available space were thrown back into a login.
Troubleshooting with built-in tools.
- Laptop firmware settings unable to help.
- Lenovo has SMART tools available.
  - Did not report hardware error on disk.
  - All SMART attributes seemed in order.
Nikola flashed a USB with an ISO of NixOS.
- Able to access a command line with my system.
- smartctl -a/x confirmed no hardware errors present.
- Also tried to use Finnix, did not assist me due to too old built-in version of ZFS (v23 vs v28 on Nix).
  - Thought NixOS wouldn’t have tools for ZFS, but both zpool and zfs are available.
Tried to get access to my ZFS pool (rpool).
- Imported all ZFS pools available.
- Loaded encryption keys for pool.
- Mounted dataset through mount as datasets were of type legacy.
```
zpool import -a
zfs list -a
zfs load-key -a
mount -t zfs rpool/home /mnt/rec
```
Attempt to backup /home to external drive.
- 70GB of data.
- External drive was connect with USB, so extremely slow.
  - Learnt that cp does not have an internal mechanism for progress report.
  - You can, however, get the cp process ID and then watch the opened files and object reference from /proc/<id>.
```
pgrep -x cp            # -x gets exact and only the ID
cat /proc/<id>/fdinfo  # gets object reference
ls -l /proc/<id>/      # lists open files
```
- Due to enormous write size (and impatience) the external (NTFS) drive’s MFS got corrupted due to lazy umount.
  - Tried to salvage with ntfsfix on NixOS command line, no success.
  - Final option is to use chkdsk on a Windows machine, but that can take around 8 hours per TB.
Attempt to garbage collect rpool/root/nixos manually.
- nix-env allows for defining system profile.
- Idea was to delete old NixOS generations to free up space.
```
nix-env -p /mnt/rec/nix/var/nix/profiles/system
```
- Failed again due to no disk space.
Started reading up on ZFS snapshots.
- ZFS snapshots do not take storage space before something changes on disk.
- My machine has automatic ZFS snapshots by a monthly, weekly, daily, hourly, and “frequent” cadence.
- THEORY:
  - Due to the snapshots running in the background all of the referenced storage from zfs list reported the full size.
    - Referenced storage “represents the total amount of space referenced by the active share, independent of snaphost … the share would consume should all snapshots be destroyed”.
  - Thus deleting files didn’t report more available space.
- Deleted some of the newer snapshots to free 14.1 GB.
- Deleting old generations now worked.
```
nix-env -p /mnt/rec/nix/var/nix/profiles/system --delete-generations 30d
```
Finally able to login into system and GC.
- Followed NixOS doumentation to GC properly.
Next steps are:
- to implement the documentation steps for automated GC,
- setup proper backups to my server for critical files,
- invest in a proper harddrive that snapshots and larger files can be exported to.