2021/Pop-ups/Webmentions Beyond Webmention.io

From IndieWeb

Webmentions Beyond Webmention.io was a pop-up IndieWebCamp session in 2021.

Details


Webmentions haven't really been revisited in some time (and with the advent of people leaning to Webmention.io). This is a chance to see what's been really wanted, what hasn't worked and where we can go with it.

Let's webmunch on the riddle of webmentions. How can we get more people hosting their own webmention sending, receiving, and validating?

How can we prevent Webmention.io from being the beginning and end of IndieWeb participants' use of webmentions?


Possible Topics of Discussion

  • Webmention-brainstorming
  • Status
    • Status Page - The purpose of a status page is to notify the sender as to whether the webmention succeeded or failed and why. Standardizing the format and encouraging more implementations to offer this feature would have several positive effects.
    • Status Query Results - Status could also be queried by a header
    • Optional callback in the webmention to indicate the outcome of processing
      • in theory, a 200 OK response is accepted, saved, can be seen, and a 202 is the accepted, queued, come back later
      • I wonder if webmention processors respect this
  • Retry recommendations.
  • Rate limiting on endpoints
  • Webmention permalinks
  • What to Store about Received Webmentions
  • What to Store about Sent Webmentions...or the failure thereof
  • Should All Types of Posts Send Webmentions?
  • Webmention and POSSE deduplication.

Interest

Add your +1 for interest in this topic:

Participants

Jokes

  • "I want to tell a webmention joke. I know I'm the source, but I don't know who the target is yet..."
  • "I'll vouch for that joke."

Quotes

  • Jacky Alciné "webmentions are like the plywood you build a ship out of to carry your content on"
  • Kevin Marks "Stop naming projects webmention"
  • Peter Molnar "Sounds like you're looking for an ontology..."

Notes

fluffy: old blog post with some wishlist items

Why do people use webmention.io?

  • Easy to set up, add one line of HTML
  • Parses the webmention URL and normalizes the content into a JSON format
  • Provides output options for retrieving the data
    • Provides an API to fetch received webmentions
    • Provides a web hook to push comments out too

What are the difficult parts of receiving?

  • Getting authorship and comments formatted nicely
  • Handling different sorts of replies
    • e.g. not showing rsvps to non-RSVPable content
    • collapsing low-content replies like likes, emoji responses etc. into facepiles
  • Handling security issues: safely sanitizing and formatting external HTML, copying external profile photos onto your own domain to avoid cross-domain requests.
  • Dealing with spam
  • Handling updates/deletion of replies


How can we remove some of the load from webmention.io and create alternate implementations?

  • Why do so many folks use WM.io?
    • static sites
    • lot of blog entries on "how to add webmentions to your site simply" pointing to webmention.io
    • self-hosted out of the box receivers are rather rough to set up apart from WordPress (eg. I failed to set up webmentiond (the go one) myself)
    • why not webmention.herokuapp.com ? -> are there any posts about it?
    • why not mention.tech ? (because it is ugly as sin perhaps)
    •   Jamie Tanna static site, just haven't gotten around to it
  • aaronpk: just having the receiver isn't enough; you need to present the parsed results usefully too
  •   Jamie Tanna has seen some folks who store webmentions into a Git repo / raise issues i.e. https://github.com/qubyte/qubyte-codes/issues/167
  • Wouter Groeneveld: I have a static Hugo website and stuff gets auto-committed using a few self-written tools. I have written my own webmention endpoint which is a single binary in Go: https://github.com/wgroeneveld/go-jamming to see how it works see https://github.com/wgroeneveld/brainbaking/ (still have to elaborate the README) feel free to ping me if anyone wants to use it.
  • petermolnar: committing the webmention to github automatically could be good
  • Marty McGuire: I use wm.io's webhook feature and save them as Hugo data to sense of in my templates.
  • jacky.wtf: being able to pull the parsed result is important
    • Barnaby Walters parsing, sorting and sanitizing webmentions for display is non-trivial, easier to rely on a pre-existing, trusted implementation. mf parsers exist, but how many languages do we have “full-stack” webmention processing libraries for, which handle everything necessary to safely parse and display replies? -> Wouter Groeneveld nobody mentioned webmention.rocks test suites yet that can be used to verify the parsing. This saved my ass in my own feeble attempt to implement stuff ;-) +1, webmention.rocks is amazing
  • petermolnar: what are people willing to selfhost? answers from the room: chances are most want to self-host nothing!
  • Ryan Barrett what are stats on webmention recipients? how many use webmention.io? 2%? 10%? 50%?
    • What percentatge of these users are static sites? +1 +1
    • What percentage of them are platforms without custom implementations?

Stats from 2017 (https://indiemap.org/docs.html#rel_me_histogram ) showed that the vast majority of indieweb sites were either WordPress or Known, which handled webmentions themselves. Micro.blog is the one new big one, it also builds them in. What's the scope of the wm.io problem specifically?

Wouter Groeneveld don't forget that wm.io also supports PINGBACK! and I have no idea why it's ignored here. Most sites - blogs on Wordpress - have it built-in. I've received more pingbacks through my pingback endpoint than webmentions...

How might we break down webmention.io down into smaller/easier constituent parts as a means of solution for this?

q for Aaron: what is rough cost of running wm.io per month?

  • Wouter Groeneveld: how do we handle SPAM? Any strategies for that? I've had a few so far. => aaron says pingback endpoints are mostly hit by spam. Is it a matter of time before webmentions get as popular and also become victim of that?
    • Barnaby Walters I’ve already had some webmention spam via brid.gy, mostly from twitter accounts -> Wouter Groeneveld did you process/delete this manually? -> yup, it’s happened little enough that I haven’t needed to build a UI for it yet, but it’ll definitely be necessary in the future.

Been made a bit more "cool" by Max Böck, Shawn Wang, Jason Lengstorf, Sia Karamalegos

  • aaronpk: people get excited about webmentions and webmention.io is easiest - there are vastly more on WordPress or micro.blog, but publicly people get excited about adding it to their sites. It's worth focusing on that narrow use case as that is where the excitement is in public.
  • David Somers: how about a webmention receiver that sends out a micropub comment
    • Aaron Parecki: doable, but not seen anyone doing it, and could involve some extra work / spec investigation
    • Kevin Marks: appears to be done already by Bridgy - bridgy will post webmentions as comments on blogger for you , for example
    • snarfed: webmention is often much sooner implemented than micropub
  • tomlarkworthy: offload state storage into s3 buckets, that may be simpler than an API
    • https://geekflare.com/self-hosted-s3/ s3 API is almsot industry standard at this point
    • Marty McGuire: a full webmention receiver could be built as a GitHub Pages repo with GitHub actions, "publish" them as JSON data. Many SSGs can pull down data like that for compilation.
  • Jacky Alciné: webmentions are like the plywood you build a ship out of to carry your content on - is there out of the box
  • What if webmentions.io did the collection and parsing of things, but quit doing storage and pushed that piece onto one or more other locations/services?
    • This might nudge Netlify or other services to create space to store that data for people's sites and also help encourage other innovation on things like analytics, moderation, etc?
  • petermolnar: do peopel who use webmention.io understand what they;re using? Do they see it as a comment system
  • aaronpk: a lot of people see it as getting twitter comments on their website - they don't know about wm.io and bridgy
  • Chris Aldrich: when i see this I send webmentions to people so they can see non-twitter use cases
  • Wouter Groeneveld about auto-Tweets that appear on your website through brid.gy: https://sebastiangreger.net/2018/05/indieweb-privacy-challenge-webmentions-backfeeds-gdpr/ - this involves privacy concerns... That is why I implemented an "anonymous" system for webmentions that come from "webmention silos" such as brid.gy in https://github.com/wgroeneveld/go-jamming
    • Same with fb content — I’ve had one person request that all their responses to my content backfed from FB to my site be removed. This is another thing which it’d be worth having a UI and automation for, in order to be able to easily comply with GDPR. I think it’s less likely with content from twitter, but people who mostly use facebook tend to see it as being cut off from the rest of the web, and are unpleasently surprised when stuff they wrote there mysteriously shows up elsewhere, with their name and profile photo (!) next to it. Also, AFAIK there’s no way for content backfed via bridgy to be deleted automatically if the original comment is deleted +1!
  • web actions are a way to embed arbitrary comment UI - https://indieweb.org/webactions
  • Who's interested in building a webmention.io like product, but hasn't yet implemented?
    • woutergroeneveld: was a pain to get everything implemented (but webmention.rocks helped)
      • lots of cases for how it's implemented - Header/Link/rel=
      • there were microformats parsers already there, but deciding how to show it on your site was a lot of choices
    • jamietanna is up for building it - just has higher priority things to do
      • +1 on needing an MF2 parser, I'd be writing this for Java which doesn't have one (yet) -> http://any23.apache.org/ ? haven't tried it.
    • kevinmarks: updates to Mention.tech harder - AppEngine on Python 2.7 means would have to rebuild in Python 3 also google has deprecated all the libraries I used for it
    •  Barry Frost is interested in building a receiver that will send the parsed webmention to your server via a webhook. He aims to make the JF2 object compatible with webmention.io's format.
  • jacky: should we define a way for a receiver to return mentions in a common form?
    • aaronpk: the token is optional, you can make them public

Steps from Webmention to comment display

  • aaronpk: we should list all the steps involved: +1, something similar to the mf2 parsing prose algorithm for thorough webmention handling would probably ease implementation
    • 1. webmention verification
    • 2. what do you do with the page you found? mf2 parsing is normal
      • 2.a -> chance to do MF2, JF2, JSON-LD, etc or fb or twitter markup
      • Potentially also archive the HTML and HTTP headers for future reference or re-parsing
    • 3. take the mf2 and turn it. into a comment - wm.io does this in jf2 with massaging
      • angelo mf2util in python will do the work here too - is xray the equivalent?
      • jacky: granary.io is another tool for doing this
    • 4. storage - where do you put it?
      • Check against existing stored webmentions to determine if we’re updating or deleting a reply
    • 5. retrieval - client side, build process etc
    • 6. rendering - turning the retrived data into page content
    • 7. client storage - adding to receiver's site
  • webmention.io does 1-5
  • Would be great to have a thing that's straightforward to self-host, cf selfauth, where it's a PHP bundle you drop onto a server and it uses local database (incl. SQLite)

petermolnar:Could we look at having a comment using an oembed-style representation for a comment? Monocle/Quill does preview/reply context, built by the receiver, but OEmbed would produce the data on the sender's side, could then be embedded directly in the received

Granary.io mentioned as a service for converting one format to another

10 minute break here

There is this diagram:

Put issues and thoughts in here:

Join us via IndieWeb chat: https://indieweb.org/discuss

Discussion during the break

Posts

See Also