Vouch

From IndieWeb
(Redirected from webmention with vouch)


Vouch icon
Vouch icon

This article is a stub. You can help the IndieWeb wiki by expanding it.


The Vouch protocol is an anti-spam extension to Webmention that can also be used to customize how your site accepts responses from different audiences.

Webmention with Vouch depends on understanding Webmention. Please read that first.

Status
This is a Living Specification yet mature enough to encourage implementations and feedback.
Latest Published Version
https://indieweb.org/Vouch
Participate
issues
discussion on #indieweb-dev (on Libera IRC)
Editors
Aaron Parecki
Tantek Çelik
Authors
Other contributors: revision history
License
Per CC0, to the extent possible under law, the editor(s) and contributors have waived all copyright and related or neighboring rights to this work. In addition, as of 2024-12-24, the editor(s) and contributors (2015-04-07 onward) have made this specification available under the Open Web Foundation Agreement Version 1.0.


Summary

  • If Bob is sending a webmention to Alice
    • (who don't know each other, i.e. Alice has never linked to Bob),
  • Bob sends along a third parameter, "vouch",
    • where the vouch is a URL on Carol's site to a page that has a link to Bob's domain (thus "vouching" for Bob),
    • and Carol's site is a domain that Alice approves of (e.g. has linked to in the past).

By doing so, Bob is telling Alice:

  • please accept my webmention,
  • because Carol,
    • who you approve of,
    • vouches for me.

Protocol

Webmentions are sent "from" a source URL "to" a target URL to notify the target that it has been mentioned at the source URL, with an optional "vouch" URL to help the target determine whether they should accept the webmention.

  1. Alice posts some interesting content on her site (which is set up to receive Webmentions).
  2. Bob sees this content and comments about it on his site, linking back to Alice's original post.
  3. Using Webmention, Bob's publishing software automatically notifies Alice's server that her post has been linked to by the URL of Bob's post by sending both his URL(source) and hers(target) to her.
  4. Alice does not know Bob...Alice has never linked to Bob. Bob sends along a third URL parameter called 'vouch'. Vouch is a URL belonging to Carol.
  5. Alice verifies that the source and target are valid URLs
  6. Alice checks if she approves Bob's domain. If she does, then Alice's publishing software verifies that Bob's post actually contains a mention of her post and then includes this information on her site, otherwise she rejects the webmention.
  7. If she does not approve of Bob's domain, Alice checks if she approves of Carol's domain.
  8. If she does, then Alice checks if the page provided has a link to Bob's domain.
  9. Finally, Alice's publishing software verifies that Bob's post actually contains a mention of her post and then includes this information on her site, otherwise she rejects the webmention.

If any checks fail, Alice's software returns 400.

Why

You should implement receiving Webmention with Vouch in order to automatically block all automated spam and to aid in moderation of mentions.

You should send webmentions with vouches to increase the chance that your webmention comments will be accepted, or approved from moderation in a faster manner. As more and more sites implement Webmention with Vouch, plain webmentions may be blocked more frequently or subject to more scrutiny.

How To

Vouch Selection

  1. The receiver posts a blog post on their blog
  2. The sender writes a post on their blog that links to the receiver's post.
  3. After publishing the post (i.e. it has a URL), the sender server notices this link as part of the publishing process and the server does Webmention discovery on the receiver's post to find its Webmention endpoint (if not found, process stops).
  4. The sender checks their own list of approved domains to see if the receiver is listed.
    • If so, the sender proceeds to Sending with vouch set to an appropriate link. This could be a public list of approved domains, if the sender maintains one, or a previous post where they linked to the receiver.
  5. The sender checks the receiver's site for a list of approved domains, or uses some other means to get a list of domains that the receiver has previously linked to.
  6. The sender checks if the receiver's list already contains their site.
    • If so, the sender stores this link in their private list and proceed to Sending with vouch set to permalink to the page on receiver's site that contained this entry.
  7. The sender compares the receiver's list to their own list of followers.
  8. If the sender finds a match between the lists, he proceeds to Sending using the URL stored with the match in their list as the vouch value.
  9. If the sender has not found any relation, he proceeds to Sending without a vouch value.

Sending

  1. The sender's server sends a webmention to receiver's webmention endpoint with
    • source set to sender's post's permalink
    • target set to receiver's post's permalink.
    • vouch (if any) set to permalink of a page that links to the sender's site, that the receiver will see as an acceptable source.

Receiving

  1. The receiver receives the webmention
  2. They verify that target (after following redirects) in the webmention is a valid permalink on that site (if not, processing stops)
  3. The receiver's server verifies that the source (when retrieved, after following redirects) in the webmention contains a hyperlink to the target (if not, processing stops)
  4. The receiver's server checks for the existence of a vouch value. (if not, webmentions can be marked for moderation and processing stops or proceed to further spam filtering methods, etc)
  5. The receiver's server checks that the base site of vouch is either his own site, or a site in his approved list. (if not, webmentions is marked for moderation and processing stops or proceeds to further spam filtering methods)
  6. The receiver's server verifies that the vouch (when retrieved, after following redirects) contains a hyperlink to the domain used in source (if not, processing stops)

Flow Chart

Summary description of Webmention with vouch flowchart from 2014-285 IndieWebCamp Cambridge.

Webmention support only:

  • start: receive webmention
  • are source and target valid URLs?
    • no: return 400
  • approve source?
    • no: return 400
  • does my site accept webmentions for target AND does source link to target?
    • no: return 400
  • 200 accept webmention


Webmention with Vouch support:

  • start: receive webmention
  • are source and target (and vouch if present) valid URLs?
    • no: return 400
  • approve source?
    • no: has vouch param?
      • no: return 449
    • approve vouch?
      • no: return 400
    • does vouch link to source's domain?
      • no: 400
  • does my site accept webmentions for target AND does source link to target?
    • no: return 400
  • 200 accept webmention


Webmention with Vouch support with async network traffic:

  • start: receive webmention
  • are source and target (and vouch if present) valid URLs?
    • no: return 400
  • approve source?
    • no: has vouch param?
      • no: return 449
    • approve vouch?
      • no: return 400
    • start async process to check webmention with vouch links
    • return 202
  • start async process to check only webmention source and target links
  • return 202


Async process to check webmention links:

  • does my site accept webmentions for target AND does source link to target?
    • no: status 400
  • is there a vouch to check?
    • no: status 200 accept webmention
  • does vouch link to source's domain?
    • no: status 400
  • status 200 accept webmention


Webmention with Vouch testing support (not in flow diagram) with async network traffic:

  • start: receive webmention
  • are source and target (and vouch if present) valid URLs?
    • no: return 400
  • has vouch param?
    • approve vouch?
      • no: return 400
  • approve source?
    • no: has vouch param?
      • no: return 449 or
        • warning: might not accept this unvouched in the near future.
  • start async process to check webmention with vouch links
  • return 202

IndieWeb Examples

IndieWeb Examples of receiving and sending Webmention with Vouch, in order of implementation.

Aaron Parecki

Aaron Parecki's previous version of p3k supported receiving Webmention with Vouch testing (unvouched warning option) on aaronparecki.com since 2014-10-17. E.g.:

Ben Roberts

Ben Roberts uses Postly on ben.thatmustbe.me to support sending and receiving Webmention with Vouch as of 2014-10-17. E.g.:

gRegor Morrill

gRegor Morrill uses ProcessWire on gregorlove.com to support sending and receiving Webmention with Vouch as of 2015-11-29.

Martijn van der Ven

Martijn van der Ven supports receiving Webmentions with Vouch on his endpoint, and does rudimentary checking of the validity.

  • It flagged a Webmention received from gRegor Morrill as invalid because GitHub was set as a vouch. Anyone could have created a page on GitHub to create a “fake” vouch for themselves, so I block such silos.

Design Considerations

Easy For Receiver Implementation

In general a key goal with "vouch" is something that is easy for the receiver to process, and instead put more burden on the sender.

Shift Burden To Sender

Typical protocols susceptible to (abused by, made nearly useless by) spam (e.g. SMTP, Pingback, Trackback) make it very easy for the sender to send, and thus put all the burden of dealing with filtering undesired senders on the receiver.

Vouch pushes the (potentially cognitively challenging) work of 2nd degree discovery onto the sender of the webmention, and instead makes it easy for the receiver to implement.

Zero moderation

Another key goal is to build a system with zero moderation tax, at least to deal with automated spam systems.

Which pages to trust?

A few ideas of how to build a list of pages that you accept as Vouch links. These could have different priorities or levels of trust, with different consequences.

  • pages you follow
  • pages of people you explicitly trust (friends)
  • event-pages for events you have attended (maybe doesn't scale for large events, but e.g. visiting an IndieWebCamp likely means I'd want to see responses from other attendees)
  • members of a web ring
  • your own site


FAQ

(in progress)

Is vouch saying you know me from

Q: Is a vouch saying "you may know me from…" ?

A: Not quite. No assertion of "may" nor "know". Just a "here's someone's {C} (perma)link, you {B} have linked to {some URL at C's personal domain}, where {C} links to me {some URL at A's personal domain}"

Do I vouch for my friends

Q: Do I vouch for my friends?

A: No, no assertion of friendship is implied or used by vouch.

What is HTTP 449

Q: What is the HTTP 449 status code?

A: HTTP 449 is a status code used by Microsoft to mean "Retry With"[1]

Why not HTTP 401

Q: Why not use the HTTP 401 status code?

A: HTTP 401 requires that the response include a "WWW-Authenticate" header, and says that the client can retry the request with authentication. Since Vouch does not involve authentication of any kind, this is not an appropriate response code to use.

Why not HTTP 403

Q: Why not use the HTTP 403 status code?

A: While there may be future uses of 403 in Vouch, currently the meanings of 403 do not map to anything in vouch.

From https://en.wikipedia.org/wiki/HTTP_403

A 403 response generally indicates one of two conditions:

  • Authentication was provided, but the authenticated user is not permitted to perform the requested operation.
  • The operation is forbidden to all users. For example, requests for a directory listing return code 403 when directory listing has been disabled.

Vouch is not about authentication, thus the first makes no sense. Nor would vouch be indicating that an operation is forbidden to all users, thus the second does not apply either.


From http://tools.ietf.org/html/rfc7231#section-6.5.3 

The 403 (Forbidden) status code indicates that the server understood
the request but refuses to authorize it.  A server that wishes to
make public why the request has been forbidden can describe that
reason in the response payload (if any).

If authentication credentials were provided in the request, the
server considers them insufficient to grant access.  The client
SHOULD NOT automatically repeat the request with the same
credentials.  The client MAY repeat the request with new or different
credentials.  However, a request might be forbidden for reasons
unrelated to the credentials.


While this could work for this purpose, 403 implies that the server will not take any action. There is no actual requirement that webmentions that are not vouched be dismissed. It is perfectly acceptable to put unvouched webmentions in to a moderation queue for example. The response only indicates that it would be better to reply with vouch.

How can a sender find a vouch link

Q: How can the sender find such a vouch link?

A: How do people in general explain how they know each other? We can push this question off to arbitrary social sidebands of choice, the way people share links informally in all sorts of ways.

In addition, since such social questions are of interest beyond commenting (as evidence by Facebook and other silos showing how many friends in common you have and who they are), there's likely to be one or more aggregators that surface this information in some queryable way.

If Alice is already a reader of Bob's content (which Alice should be, since she is sending them a comment on one of Bob's permalinks), then Alice should already be somewhat familiar with who Bob links to (or can quickly check Bob's home page or recent posts for people). Alice should also be relatively familiar with who links to her own site (hint: Alice's recent incoming webmentions, or recent HTTP referers [sic]).

Thus with a quick look at Bob's home page (or recent posts) Alice should be able to trivially recognize "there's someone (Carol) that sent Alice a webmention recently," and go pull that out of Alice's queue/history of recent webmentions, and use that as a vouch URL.

Someone might also setup a service like say socialsearchme.com where you can put in your personal site, and see who has linked to you (most recently, most frequently), who you've linked to, the overlap, as well as how many and who you have in common.

In which case, as a user (Alice), after commenting (and presumably having their "normal" webmention rejected), they could check a hypothetical service like socialsearchme.com for themselves, see that they'd linked to Bob, and then see how many (if any) people they have in common, especially, if there are any people that Bob has linked to that link to Alice, and if so, presto, that's a "vouch" URL.

This manual human step of checking, i.e. "Hey how do we know each other" adds to the strength of the social tie between the commenter and the post author.

How do I verify a vouch

Q: How do I know if the vouch I received is legit. i.e. do I have to verify that there is a link between sender and voucher?

A: As the receiver, you have to do two (relatively) simple checks.

First, regarding: "don't I have to verify that there is a link between sender and voucher" - in one direction only, that vouch URL links to sender's personal domain. That's the easy check.

The second, potentially more challenging check, is you have to verify that you approve the vouch's site/domain (e.g. perhaps a link from your blogroll, or a whitelist, or your Twitter followings, nicknames cache, outbound link cache (i.e. a cache of every domain you've linked to), etc. take your pick).

Must I trust voucher knows source

Q: But can anyone can send me a webmention with voucher = someone that I've linked to, should I just trust that they know the source?

A: No notion of "trust" needed. Just links. You MUST verify that the vouch (presumably a permalink at the vouch's site) links to the source's site.

Continued: "or do I go fishing in your friendslist to verify it?" No. There is no need to crawl anywhere from the vouch URL.

Why does Twitter not worry

Q: Why don't people worry about this for Twitter?"

A: They do. :) See Twitter silo mentions of Tantek Çelik's Twitter profile: https://twitter.com/search?f=realtime&q=%40t&src=typd with plenty of spam.

Must I always send a vouch

Q: If I want to send a webmention to a colleague for the first time, I have to somehow...

A: Not necessarily. The Vouch protocol is a backwards compatible extension to Webmention. E.g.:

  1. You send a webmention as if you would today.
  2. Perhaps the receiver is already a fan of yours (secretly, in their private whitelist, nicknames cache, etc.) and thus your webmention is accepted without any additional work.

Can a vouch apply to shared membership

Q: Can a vouch can apply to a shared membership too? E.g. chat-names or next-hwc, or XOXO?

A: A vouch URL could be for example User:Kevinmarks.com, which obviously links to the domain(s) (known.)kevinmarks.com, and thus vouches for any permalinks at those domains. Then the receiver of the webmention has to decide, do I approve of indieweb.org (e.g. have linked to it before, likely yes), ok I'll accept that site as a vouch, and the specific vouch URL, and thus the webmention.

Is the vouch a sponsor

Q: Is the vouch is a sponsor?

A: No, the vouch is not a "sponsor" or any other new term. The vouch (C) is just a (perma)link, which directly links to A's domain, and which B is likely to approve (e.g. has linked to C's domain previously).

Does vouch make all webmentions manual

Q: Does vouch make the whole webmention sending manual? No auto-sending webmentions?

A: It does not. For example it is likely your code can programmatically itself test http://indieweb.org/irc-people and use it as a default vouch.

From there, you can decide how much intermediate UI you want to show, "The comment you sent is awaiting some form of vouching that they link to that links to you" with a URL field to enter. Plenty of opportunity to crawl, cache, innovate there.

Does vouch make it too hard to send webmentions

Q: Does this make it too much work to send webmentions?

A: Maybe. We won't know until we attempt to implement Webmention with Vouch and see what roadblocks we run into.

When do I know to send a Vouch with a Webmention

Q: When's a 'good' time to send a Vouch with a Webmention?

A (from Jacky Alciné on 2021-03-31 10:19 PST): It'd be ideal to do so if the Webmention endpoint explicitly asks to. If the Webmention is capable of determining prior success rates, it can omit them in that case. But if it's one to be very 'friendly' (like to ask first), then sending a Vouch to a very public page (like chat-names) could help coax a endpoint to trust them (keeping bad actors on the Wiki would be ideally hard to do).

How can I use Vouch with backfeed from silos

We recommend that you don't. Vouch is promising for native IndieWeb moderation, but it expects each user has their own domain, which generally isn't true on silos. Also, quality and behavior of users, posts, comments, moderation, etc on silos varies significantly from IndieWeb sites, and across different silos themselves. If you want to programmatically filter backfeed webmentions, we recommend special casing backfeed senders like Bridgy based on source URL domain, or individual silos or even individual silo users based on u-url domain and path.

Also see backfeed#Moderation_and_spam

In Progress

Vouch was braindumped into IRC: http://indieweb.org/irc/2014-09-28 (and following days).

This page is in progress documentation of that IRC braindump, with follow-ups.


Issues

Issues are hosted at https://github.com/indieweb/vouch/issues

to do

To be written up here in this wiki page from that braindump

  • clean up vouch protocol flow summary
  • vouch protocol flow details
  • vouch FAQ

To be drawn and posted:

  • vouch user-flow diagram
  • vouch protocol-flow diagram

Sessions

See Also

Retrieved from "https://indieweb.org/wiki/index.php?title=Vouch&oldid=90724"