p3k



 pзk  (pronounced "peek") is the name of the software that runs aaronparecki.com and indiewebcat.com.

"p3" is short for "personal publishing platform".

Source Code
While the entirety of p3k is not open source, I have open-sourced many of the components. I've chosen to provide components of my site rather than the whole thing because in reality nobody is going to want to use the exact same workflow I have built. Instead, components of my site can be re-used by others in their own way.

Apps

 * Quill - Posting interface for many kinds of posts
 * Teacup - food-tracking client
 * Gramophone - a podcast posting client
 * Switchboard - a PuSH hub
 * Atlas - APIs for looking up information about locations
 * Compass - a GPS tracking server
 * Telegraph - webmention sending API
 * webmention.io - webmention receiver
 * XRay - extract post and author data from a page
 * Aperture - a Microsub server
 * Monocle - a Microsub reader
 * Watchtower - a minimal API to watch web pages for changes, similar to Superfeedr
 * See also: p3k naming convention

Libraries

 * p3k-http - a small HTTP client
 * p3k-utils - several helper functions for dealing with dates, timezones, URLs, and more
 * p3k-websub - a library for subscribing to and publishing WebSub feeds
 * p3k-micropub - utilities for handling Micropub requests
 * php-mf2 - a PHP Microformats 2 parser
 * mention-client-php - A PHP client to send webmentions and pingbacks
 * php-comments - Helper for parsing and presenting comments from external sites
 * link-rel-parser-php - Parse HTTP  headers into a structured format
 * indieauth-client-php - Sample implementation and helper methods for an IndieAuth client
 * date-formatter-php - Render dates and date ranges in a human-readable format, including proper microformats-2 markup
 * emoji-detector-php - Finds all emoji in an input string and return information about each emoji character
 * p3k-timezone - Find the timezone for a given latitude and longitude
 * clone-media-fragment - enables YouTube-like time offset URLs for audio and video tags
 * tantek/cassis - Used for truncating and ellipsizing post text

IndieAuth
p3k has a built-in authorization endpoint, which supports customizing the scope granted, as well as restricting which "channels" apps can post to.



Folder Structure
All content in p3k is stored as files on disk.

There is a main folder "posts" which contains individual files for each post as well as any associated files. Posts are stored in a folder structure by year/month/day/index.

The index increments with each new post created on that day. The index is not necessarily time-ordered, since quite often various import scripts create backdated posts after newer posts were already created.



Ultimately each post ends up with its own folder containing the primary post file, and optionally a   folder containing associated files. The web server serves files from this folder directly.

A post with a URL of:

will correspond with a storage file:

The image at the URL:

will be stored on disk as:

File Format
The contents of the  file is described below.

There is a YAML header block which contains some top-level properties that are core to p3k, such as published, type, content-type, and client-id. These are indexed in the database and control how the post is sorted and how it is rendered.

There is another top-level property called  which contains the Microformats2 vocabulary of all the post contents, with the exception of the   property.

The header block is delimited by  , and everything below the header is considered the   Microformats2 property. The  property notes the content type in order to indicate which renderer to use. (Either text, in which case a basic autolinker is used, Markdown which renders the Markdown as HTML, or HTML which outputs the contents directly without any processing.)

--- published: "2017-04-15T18:23:00-07:00" type: entry content-type: text/plain client_id: https://ownyourgram.com properties: category: indieweb location: type: - h-card properties: name: Migration Brewing Company latitude: 45.526300 longitude: -122.636451 locality: Portland region: Oregon country-name: USA syndication: https://www.instagram.com/p/BS7XoRCANxF/ photo: photo.jpg p3k-channel: - photos - primary ...

Scheming. #indieweb

All values can be either arrays or single values, and there is some code to abstract reading and writing the values regardless of whether they are strings or arrays.

All properties in  can be read and written via Micropub requests.

Indexing and Caching
To render lists of posts, such as channels and tags, a database indexing strategy is used. Channels are top-level URLs such as /notes, /photos, /checkins. Tag pages live under the "tag" subdirectory, e.g. /tag/indieweb. The home page is actually just another channel named "primary".

Certain properties of each post are cached in a MySQL database and used when querying channel and tag pages.


 * the published date, stored as UTC date with separate timezone offset
 * the  property (tags)
 * the  property

To generate a page such as the home page or a tag page, the database is queried and sorted by published date. The database returns the list of filenames, and then each post file is read from disk and rendered in a list.

Feature Announcements

 * 2019-09-30 Unlisted Posts
 * 2019-02-25 Emoji Avatars
 * 2019-01-21 Monthly Summary Pages
 * 2018-01-21 Home Page Pixel Art
 * 2018-01-06 Code Snippets
 * 2018-01-05 Rich Link Previews
 * 2017-05-19 Blue Dot
 * 2017-04-27 Now Page
 * 2017-03-22 Post Audience
 * 2017-03-18 Checkins

Working On
The things I am currently implementing in p3k:


 * https://github.com/aaronpk/p3k/issues?labels=priority%3Ain-progress

Itching
These are a collection of annoyances that have respective features / improvements. As their annoyance level bubbles to the top, they're likely to become concrete "Working On" tasks.

Random Ideas

 * "Freeze" or "lockdown" mode.
 * Stop showing any new content in view until after this mode is de-activated. Kind of like Known's delayed checkins, except for every post. Probably okay for posts to be public if the permalinks are visited though, it's more about discoverability of posts.
 * I basically want a giant "pause" button for my website, where it keeps storing posts I create, but if you look at my home page or any feeds, it looks like I haven't posted.
 * I do want posts to continue sending webmentions, so that i can continue to have conversations with people, but the posts just wouldn't show up in my /replies page or anything until later
 * Link to specific blog posts on year pages, to be able to link to my year-in-review post on 2017.
 * Show the logs of webmentions sent when I'm looking at a post

Blog posts in lists
Currently, any time you are looking at a list of posts on my site (my home page, a tag page, a search, etc), the full contents of articles is shown. This makes it hard to quickly scroll through the posts in the list. I'm considering switching back to showing just the name, summary and featured image. (This means I'll need to do a better job of setting featured images for my blog posts, but that would also be a good solution for generating the twitter/facebook meta tags too).

However, I do like that my home page shows the full contents of articles. So maybe the answer is to special-case the home page to show the full contents, but everywhere else in lists the articles would show the collapsed view.

Archive Pages
I'm not super happy with how to browse older content on my site. Mainly what's missing is a way to quickly browse past blog posts.

What I do like:


 * Day permalinks: https://aaronparecki.com/2017/04/19/
 * Month permalinks: https://aaronparecki.com/2017/04
 * Year permalinks: https://aaronparecki.com/2017

The month permalinks are cute, but they aren't super actionable since there is no indication of *what* to find behind each icon. It might be nice to have some sort of additional summary of the month below or above the calendar. Maybe including things like:


 * List of the name of each blog post that month
 * Most common tags used in posts
 * Summary of transport posts (number of miles biked, walked, number of flights, etc)
 * Showcase events from the month

Article Archives
Here are some examples of blog post archive pages that I like. I might want to change my articles list to something more like these rather than showing the full contents of each post.


 * https://snarfed.org/archive
 * http://v6.robweychert.com/blog/
 * https://jvns.ca/
 * https://rachelandrew.co.uk/archives/

Considering changing my articles page to show just the name, featured image and summary (possibly autogenerated if there is no manual summary) of the post.

As of 2018-01-06, my articles page shows a table of contents at the top.

Automatic Yearly Stats
While creating my 2017 in Numbers post, most of that aggregation could be done automatically, as well as incrementally. I was thinking about hard-coding some aggregations to run, and displaying them on my year pages like https://aaronparecki.com/2017 updating the aggregations on a cron job daily or so.


 * Time and distance in each mode of transport: walking, biking, cars (driving, taxis, car2go), trains, planes

Heart Rate

 * Import heart rate data from FitBit to a Compass database
 * When a new bike/walk/ride is posted, query the Compass database and find heart rate info
 * Show heart rate range on bike/walk/runs

Address Book
Currently I do have a functional address book, but I don't have an easy way to populate it or browse it.

I can use Micropub to create an h-card post on my site, which will properly index peoples' URLs and nicknames, so that if I mention someone's @-nickname in a post it will link to their site instead of to Twitter.


 * Need to figure out a permanent solution for a URL scheme for nicknames
 * /nickname/ (trailing slash is part of the mechanism my site uses for top-level URLs like this, would be hard to remove)
 * /nickname (would require a bit of work to not have the trailing slash due to some internals of my site)
 * both of the above have the potential to conflict with page names and channels, like /gps/, /rides
 * /u/nickname (too similar to reddit?)
 * /people/nickname (too long?)
 * /@nickname

Venues / Checkins

 * ✅ Checkin posts, e.g. https://aaronparecki.com/2017/04/19/9/
 * Venue permalinks
 * Find a good URL structure (I think I'm happy with the current solution)
 * Need to store a mapping between Foursquare venues and my own permalinks, including handling what happens when Foursquare venues are merged/deleted
 * Need a good venue search, or just defer to Foursquare checkins for now
 * Replicate Foursquare lists on my site, along with a comment about why a venue is in that list
 * Show venues clustered on a map
 * Figure out what a venue permalink shows about the venue
 * Comments from me about the venue? (e.g. foursquare tips)
 * Collections the venue appears on
 * A list of all my checkins at the venue? Last couple checkins? Paged feed of checkins?
 * How to handle private venues? (Both from PESOSing foursquare checkins of private venues, and for checking in only on my site)
 * Having real venue support would also make it easier to create events at locations, rather than just named places with no location info

Github Issues
https://github.com/aaronpk/p3k/issues?labels=priority%3Aitching

Location

 * Publish city change post when I end up in a new location

Since I carry an active GPS tracker at all times, I collect a large amount of location data. It currently provides data to location-tag all my posts, and sets the timezone of the posts to my local timezone, and includes weather info on posts. I do not have a feed of the raw location data available.

There are other non-venue locations I may want to publish, such as when I get into a neighborhood I haven't been to in a while. Ideally my phone would detect automatically when I've entered a new area or an area I haven't been to in a while. Probably I would want it to prompt me before publishing anything publicly. I could receive a push notification saying something like "It looks like you're in NW Portland for the first time this year! Publish this?" These posts are usually only interesting when there is some additional information that provides context, such as "for the first time this year".