WebSub
WebSub is an open standard (W3C Recommendation) notification-based protocol for web publishing and subscribing to streams and legacy feed files in real time, previously known as PubSubHubbub or PuSH, and briefly PubSub.
- Latest published version: https://www.w3.org/TR/websub/
- Latest draft: https://websub.net/draft
WebSub started as PubSubHubbub, was refined in the W3C Social Web Working Group, and published as a W3C Recommendation.
Why
You should implement WebSub for your home page updates so that IndieWeb readers (like Woodwind and Monocle) can:
- promptly fetch your posts when you publish them
- and avoid polling your server with unnecessary traffic
How
See: How to publish and consume WebSub
IndieWeb Examples
There are several indieweb sites producing WebSub notifications, and a few indieweb-centric applications that consume them (see in particular indie readers, Shrewdness and Woodwind). Currently there are no known indieweb sites that subscribe to anything via WebSub, but there are a few separate-UI indie-readers that use WebSub to subscribe to h-feed streams.
Tantek
Tantek รelik uses Falcon to send WebSub notifications from his site tantek.com
- 2010-02-01 onward PuSH 0.3 notification sent for each entry in his Atom feed file
- 2015-03-18 onward PuSH 0.4 notification sent for each h-entry post on tantek.com HTML + h-feed home page
Aaron Parecki
Aaron Parecki uses p3k to send PuSH notifications from his site aaronparecki.com
- 2012-08-18 through 2015-02-27 - PuSH 0.3 notification for every entry in his Atom feeds (notes, articles and replies). Using Google's appspot.com hub.
- 2015-02-27 onward - PuSH 0.4 notification for every feed that is updated after a post is created, including h-entrys on home page, notes/articles/replies/etc pages, and tag pages. Using Superfeedr's hub.
- 2015-03-26 - changed hub from Superfeedr to Switchboard
Bret Comnes
Bret Comnes sends PuSH notifications for his bret.io Github Pages/Jekyll Atom feed file since 2014-03-16.
- PuSH 0.4 support with his home page h-entrys since 2015-03-20
Kyle Mahan
Kyle Mahan uses Red Wind to send PuSH notifications for new/edited posts and new mentions in his kylewm.com Atom feed file since 2014-03-23
- Now also sending PuSH 0.4 notifications for updates to the main h-feed at kylewm.com, using a hub at superfeedr.com. Confirmed working 2015-02-21.
Barnaby Walters
Barnaby Walters uses Taproot to send PuSH notifications for each new post on his WaterPigs.co.uk homepage HTML feed using PuSH 0.4 and Googleโs hub since 2014-03-25.
David Shanske
David Shanske uses WordPress and the PushPress plugin to send PuSH 0.3 notifications of his RSS feed updates since 2014-02-16.
Matthias Pfefferle
Matthias Pfefferle uses WordPress and the WebSub/PubSubHubbub Plugin (selfdogfooding) to send PuSH notifications since 2011-01-29.
- PuSH 0.4 with h-entrys support since 2012
Christian Weiske
Christian Weiske sends PuSH 0.4 notifications since 2015-04-01 using his own hub, phubb.
Ben Werdmuller
Ben Werdmรผller uses Known to send PuSH 0.4 notifications for each new post on his werd.io homepage HTML feed since at least 2015-05-04 (Known 0.7.8 release date).
Pelle Wessman
Pelle Wessman uses GitHub Pages + Superfeedr to send PuSH notifications from his site voxpelli.com
- 2011-08-09 onward - manual PuSH notification sent for each Atom file
- 2015-04-05 - automated notifications using GitHub page_build webhook
- 2015-05-16 onward - PuSH 0.4 notification sent for the home page as well
Andy Leap
Andy Leap sends PuSH 0.4 notifications for Vendaria.net since 2015-05-16
1000s of Known Sites
1000s of *.withknown.com sites send PuSH 0.4 notifications for each new post on their homepages's HTML feed since at least 2015-05-04 when Known 0.7.8 shipped with reliable PuSH 0.4 support.
There are also numerous (hundreds?) of Known installs that are also all likely running Known 0.7.8 or later and thus send PuSH 0.4 notifications.
Malcolm Blaney
Malcolm Blaney uses dobrado to send PuSH notifications for new and updated posts. Support was added to dobrado on 2016-05-13
fluffy
fluffy uses Pushl to send WebSub notifications at site publish time, using Superfeedr as the hub, as of October 2018
Silo Implementations
Flickr
Flickr supports PuSH to subscribe to photos posted within specific locations or with given tags. https://www.flickr.com/services/api/flickr.push.subscribe.html
Historical/Deprecated
Instagram previously supported PuSH so that apps can subscribe to notifications whenever a user has posted a new photo. Their documentation page (https://instagram.com/developer/subscriptions) now returns 404, which is probably related to deprecation of their API in favor of Facebook's Graph API which only works with business accounts.
Consuming Implementations
The following implementations consume and subscribe to PuSH feeds:
WebSub:
PuSH 0.4:
PuSH 0.3:
- StatusNet
tt-rssWebSub support removed 2017-05-16 (discussion)
How To
Publish and Consume a WebSub-enabled feed
PuSH 0.4 goes beyond previous versions to allow publishers to send push notifications for any HTTP resource (e.g. h-feeds). You should use the newer spec, 0.4. PuSH 0.3 supported push notifications only for legacy XML feed files.
See the main article: How to publish and consume WebSub.
WordPress Plugins for PuSH
The PushPress plugin can be installed on a self-hosted WordPress set up. If you're using WordPress.com, they already support PuSH by default (using a built-in version of this plugin).
An alternative plugin written by Matthias Pfefferle, that supports the more recent PubSubHubbub v0.4 spec, is the WebSub/PubSubHubbub Plugin.
Subscribing to Fragments
Superfeedr also offers the unique ability to subscribe to fragments on a page, using the # symbol. For example, if you subscribe to http://tantek.com/#.hentry, you will receive POST to your webhook/callback endpoint with the content of the first element of class "hentry" on http://tantek.com/
This should be seen as an optimization. A minimal consumer can simply re-fetch the resource itself when it receives a ping.
Testing your PuSH-enabled feed
There are several ways you can test whether or not your PuSH feed and pings are working properly:
Testing PuSH 0.4
- websub.rocks - a validator to help you test your WebSub implementation (publisher and subscriber)
- Subscribe to your home page in one of these indie readers:
- Publish a new post and send a PuSH 0.4 notification
- Watch the reader to see if your post shows up - it should show up in seconds or less.
Testing PuSH 0.3
RSS Reader
Most popular RSS Readers do implement PubSubHubbub, you can just subscribe to your feed on one of them, and see if the update as been propagated after you added content.
Status.net
- subscribe to your home page from a Status.net account
- publish stuff on your home page
- see updates appear in real time on your Status.net account
Bots
There are several XMPP/IRC bots which allow you to subscribe to feeds and be notified of updates via any XMPP or IRC client. Youโll need an XMPP account, or an IRC client.
- https://github.com/julien51/notifix (see below)
- http://blog.superfeedr.com/notifixlight/
- http://push-bot.appspot.com/
Notifix
Notifix is a bot (see above for source code). It's constantly connected to irc.freenode.net. Send him a private message like +help to see available commands. Subscribe with +subscribe <feed>, publish your content and see if you get the ping straight via IRC.
- I have had better experiences with notifixlite than PuSH Bot --Waterpigs.co.uk 03:16, 5 June 2013 (PDT)
Testing your PuSH Subscriber
- websub.rocks - a validator to help you test your WebSub implementation (publisher, subscriber, and hub)
- http://push-tester.cweiske.de/ is a useful application for testing your subscribing code. This is a known-working WebSub publisher, so you can subscribe to it, post an update, and confirm that you received a ping from its hub.
- push-tester is a tool that mimicks a blog with h-feed and h-entry and allows posting new articles with a single click. A configurable PuSH hub is notified about the new post. Public instance: http://push-tester.cweiske.de/
Hubs
- http://pubsubhubbub.superfeedr.com/
- You can also register with Superfeedr to get your own subdomain (e.g. https://kylewm.superfeedr.com). This gives you access to useful stats like number of subscribers and notifications.
- https://pubsubhubbub.appspot.com/
- As far as I have been able to tell, development on this hub completely halted in 2011, and it was left in a weird state. Appears to partially support non-XML feeds but for example requires hub.verify (a parameter that was removed in 0.4). For new development and for all h-feeds I strongly recommend not using this hub. Kylewm.com 13:55, 28 February 2015 (PST)
- User:Onebigfluke.com is aware of the issue and planning to fix it. More details on this Twitter thread Kylewm.com
- phubb - open source PHP PubSubHubbub server
- Switchboard by Aaron Parecki
- Wordpress plugin
- https://websubhub.com/
See https://github.com/pubsubhubbub/PubSubHubbub/wiki/Hubs for more of them.
Testing your hub
websub.rocks has a tool that will generate a page with h-entrys that you can subscribe to.
push-tester is a tool that mimicks a blog with h-feed and h-entry and allows posting new articles with a single click. A configurable PuSH hub is notified about the new post.
Public instance: http://push-tester.cweiske.de/
Discussion
Discussion about WebSub primarily occurs on the GitHub repo, but there's also a W3C community group:
- https://www.w3.org/TR/websub/ - official spec
- https://www.w3.org/community/swicg/ - W3C's Social community group, which works on extensions to the SocialWG's specs, including WebSub. If you're interested in WebSub, and already have a W3C login, you should join as well to help support the overall WebSub effort.
Brainstorming
Non-Web Facing Consumers
How can we support PuSH consumers that do not have a publicly routable URL, such as devices behind a firewall or NAT? Maybe a hub or an external service could provide an alternative subscription mechanism such as websockets or eventsource, which could then make the PuSH subscription on behalf of the consumer. Aaron Parecki 12:32, 26 May 2015 (PDT)
Issues
Too Complex
In the past (2013 era) there was controversy about PubSubHubbub being too complex for the indieweb. Since then numerous indieweb sites support PuSH notifications of their published content, and we have a few new PuSH hubs built and maintained by indieweb folks, as well as readers subscribing to PuSH updates. The below is left as historical record of a past issue.
- "PubSubHubbub (by Google for Google) or any push based solution for the web is unnecessarily complex for #indieweb. Polling works just fine.": http://indiewebcamp.com/irc/2013-05-29#t1369859193
- Polling is fine for async but we're past that now. Kylewm.com 22:24, 25 February 2015 (PST)
FAQ
Q: How do I update the hub I am using and ensure subscribers update accordingly?
A: Subscribers should poll occasionally to see if the hub has been updated. You can list and ping both hubs for a while to speed up the process. [1]. Also, hubs should actually notify the hub URL (as part of the discovery links) which means that subscribers will know about the designated hubs with every notification, making it completely optional to have a "routine" polling. It's considered good practice when a feed/publisher changes its hub to have a period during which it *also* pings the old hub (even after the discovery link was removed).
Q: Is it allowed to have the hub link in the header, but the self link in HTML or vice versa?
A: The spec says: "the publisher SHOULD include at least one Link Header [RFC5988] with rel=hub (a hub link header) as well as exactly one Link Header [RFC5988] with rel=self (the self link header)". That implies that they must be specified together.