Micropub

From IndieWeb
Jump to: navigation, search

Micropub placeholder logo

Micropub is an open API standard (W3C Recommendation) for creating, editing, and deleting posts on websites, like on your own domain, supported by numerous third-party clients, CMSs, and social readers.

Why

As a user the best reason to use Micropub apps is user choice. You choose which Micropub applications you want to use to publish, and separately, which Micropub supporting service or open source software you want to use.

As a developer there are numerous reasons including:

  • Test suites. Complete client and server test suites: https://micropub.rocks/
  • Diverse network effects. There’s already a diversity of clients and servers for your implementation to interoperate with.
  • User privacy and security. Replace any use of MetaWeblog or AtomPub (which depend on password sharing, often in the clear) with Micropub, which is OAuth-based.

Clients

Main article: Micropub/Clients

There are a wide variety of apps (web, iPhone, Android) that support Micropub for posting and editing articles, short notes, comments, likes, photos, events, or other kinds of posts on your own site.

View the Micropub Clients main article for a list of sites and client applications that publish via Micropub as well as a table indicating implementation statuses.

Servers

Main article: Micropub/Servers

Examples of Micropub endpoints are listed at Micropub/Servers.


The remainder of this page describes highly technical details of Micropub, and is currently written for developers.


Background

The Micropub vocabulary is derived directly from the Microformats vocabulary. Micropub is meant to be a serialization of Microformats that can be submitted as an HTTP POST. The method for developing new Micropub vocabularies is to look at the Microformats representation and work backwards.

Syntax

Similar to how microformats has a relatively small ruleset for parsing HTML documents into a data structure, Micropub defines a small set of rules to interpret HTTP POST and GET requests as Micropub commands. Where Microformats does not require changing the parsing rules to introduce new properties of an object such as an h-entry, Micropub similarly does not require changing parsing rules to interpret requests that may correspond to different post types, such as posting videos vs "likes".

The Micropub syntax describes how to interpret HTTP POST and GET requests into a useful action the server can take.

The full specification can be read at https://www.w3.org/TR/micropub/

Overview

All Micropub requests to create posts are sent as form-encoded[1], multipart form-data [2], or JSON encoded HTTP requests. Responses typically do not include a response body, indicating the needed information (such as the URL of the created post) in HTTP headers. When a response body is required, it is returned as either form-encoded or JSON[3], depending on the HTTP Accept[4] header.


How to implement

How to implement the Micropub API, both in a client that can discover an endpoint and publish to it, and on a server to support an endpoint to create/update/delete posts in response.

Work your way through the test suite and submit an implementation report for Micropub!

Endpoint Discovery

It should be possible to configure an API client by authenticating as your domain name using IndieAuth. After signing in, your domain needs a way to specify the API endpoint the client will use to create new posts.

Add a <link> tag in the HTML head of your home page, or send an HTTP Link header.

HTTP Header

Link: <https://example.com/micropub>; rel="micropub"

HTML Head

<link rel="micropub" href="https://example.com/micropub">

Authentication

Micropub requests are authenticated by including a Bearer Token in either the HTTP header or a form-encoded body parameter as described in the OAuth Bearer Token RFC.

Authorization should be handled via the IndieAuth protocol, built on top of OAuth 2.0. See obtaining-an-access-token for more details. An app that wants to post to a user's Micropub endpoint will need to obtain authorization from the user in order to get an access token.

HTTP Header

Authorization: Bearer XXXXXXXX

Form-Encoded Body Parameter

access_token=XXXXXXXXX

Scope

The client may request one or more scopes during the authorization request. It does this according to standard OAuth 2.0 techniques, by passing a space-separated list of scope names in the authorization request.

The authorization server must indicate to the user any scopes that are part of the request, whether or not the authorization server recognizes the scopes. The authorization server may also allow the user to add or remove scopes that the client requests.

For example, most Micropub servers require clients to obtain the "create" scope in order to create posts. (The legacy "post" scope previously encompassed both creating and updating posts.) However, some servers may require more granular scope requests, such as "delete" or "create:video". See scope for more details and a list of all currently used values for scope.

Verification

A micropub client can verify that an access token is still valid (and other details) by querying the user's token endpoint.

Issues:

  • this does not cover the case where the server has decided not to accept it anymore, e.g. because the user's access to the site has been removed
    • if a micropub endpoint plans on rejecting a token in the future, it should ensure the token endpoint also rejects the token. For systems where the token endpoint and micropub endpoint are part of the same software, this is an implementation detail. If the token endpoint and micropub endpoint are on different systems, there needs to be a mechanism for the micropub endpoint to expire the token at the token endpoint. This is what's known as token revocation, and has been written up for OAuth 2: https://tools.ietf.org/html/rfc7009

Form-encoded Microformats Representation

For the simplicity of writing clients and servers, all Micropub endpoints must handle the standard form-encoded format. At a most basic level, you should be able to write an HTML form and set the form action to your own endpoint and use it to post to your site.


Examples of Creating Objects

How to create some of the common post types

Indicating the object being created

To indicate the object being created, use a property called "h", (which would never be the name of a property of a microformats object), followed by the name of the microformats object. Examples:

  • h=entry
  • h=card
  • h=event
  • h=cite

h-entry

The following properties may be included in a request to create a new h-entry:

  • name
  • summary
  • content
  • published
  • updated
  • category = tag1, tag2, tag3 (sent as array syntax: &category[]=tag1&category[]=tag2)
  • location
  • in-reply-to
  • repost-of
  • syndication
    • Pass one or more URLs pointing to places where this entry already exists. Can be used for PESOS implementations.
  • mp-syndicate-to = https://myfavoritesocialnetwork.example/aaronpk, https://archive.org/, etc.
    • This property is slightly different from the others since it is giving a command to the server rather than describing an object, which is why it is prefixed with "mp-".
New Note

Posting a new note with tags, syndicating to myfavoritesocialnetwork:

  • content
  • category
  • mp-syndicate-to
  • published (optional, defaults to "now" if not present. Useful for writing offline and syncing later)
POST /micropub HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=entry
&content=The+%40Jawbone+UP%2C+my+favorite+of+the+%23quantifiedself+trackers%2C+finally+released+their+official+API%21+http%3A%2F%2Fjawbone.com%2Fup%2Fdeveloper%2F
&category[]=jawbone&category[]=quantifiedself&category[]=api
&mp-syndicate-to=https://myfavoritesocialnetwork.example/aaronpk
Minimal Example
POST /micropub HTTP/1.1
Host: example.com
Content-type: application/x-www-form-urlencoded
Authorization: Bearer XXXXXXX

h=entry
&content=Hello+World

curl https://example.com/micropub -d h=entry -d "content=Hello World" -H "Authorization: Bearer XXXXXXX"
New Reply

Posting a new reply, syndicating to myfavoritesocialnetwork

  • content
  • in-reply-to
  • mp-syndicate-to
  • published
POST /post/new HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=entry
&content=%40BarnabyWalters+My+favorite+for+that+use+case+is+Redis.+It%27s+easy+to+set+up+and+use%2C+I+often+use+it+to+move+data+between+apps+written+in+different+languages+too.
&in-reply-to=http://waterpigs.co.uk/notes/4S0LMw/
&mp-syndicate-to=https://myfavoritesocialnetwork.example/aaronpk
New Repost

Posting a new repost, and adding additional tags.

  • repost-of
  • category
POST /micropub HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=entry
&repost-of=http://waterpigs.co.uk/notes/4S0LMw/
&category=realtime
New Article

Posting a new article

  • content
  • name
  • category
  • published
POST /micropub HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=entry
&content=Now+that+I%27ve+been+%5Bhttp%3A%2F%2Faaronparecki.com%2Fevents+creating+a+list+of+events%5D+on+my+site+using+%5Bhttp%3A%2F%2Findiewebcamp.com%2Fp3k+p3k%5D%2C+it+would+be+great+if+I+could+get+a+more+calendar-like+view+of+that+list.+%0A%0ASince+I+live+in+Google+Calendar+every+day+anyway%2C+it+would+be+great+to+use+that+interface+to+browse+my+%23indieweb+events+as+well%21+Since+my+events+page+is+marked+up+with+%5Bhttp%3A%2F%2Fmicroformats.org%2Fwiki%2Fh-event+h-event+microformats%5D%2C+all+it+would+take+is+to+write+an+h-event+to+iCal+converter+script+to+generate+an+iCal+feed+from+my+list+of+events.+Then+I+could+just+subscribe+to+the+iCal+feed+from+within+Google+Calendar.%0A%0A%23%23%23+Bonus%3A+read%2Fwrite+access+to+indieweb+events+via+Google+Calendar%0A%0AEven+better+would+be+to+use+Google+Calendar+to+also+create+events+on+my+site.+Unfortunately+Google+Calendar+doesn%27t+support+CalDAV%2C+so+we+can%27t+do+it+that+way.+%28Of+course+I+could+use+Apple%27s+iCal+to+publish+directly%2C+but+that+also+means+I%27d+have+to+write+some+code+tot+speak+CalDAV%29.+%0A%0AInstead%2C+I+can+create+a+%22write-only%22+calendar+in+Google+Calendar%2C+and+have+p3k+subscribe+to+it.+Any+new+events+in+that+feed+would+be+moved+over+to+the+internal+events+page+and+deleted+from+the+Google+Calendar.
&name=Itching%3A+h-event+to+iCal+converter
&category[]=indieweb&category[]=hevent&category[]=events&category[]=calendar&category[]=p3k
New Bookmark

Posting a new bookmark with name, quote, and tags.

  • bookmark-of
  • name
  • content
  • category
POST /micropub HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=entry
&bookmark-of=https%3A%2F%2Fplus.google.com%2F%2BKartikPrabhu%2Fposts%2FUzKErSbfmHq
&name=To+everyone+who+is+complaining+about+Popular+Science+shutting+down+comments...
&content=%22Why+is+there+this+expectation+that+every+website+should+be+a+forum%3F+No+website+has+any+obligation+to+provide+a+space+for+your+rants.+Use+your+own+space+on+the+web+to+do+that.%22
&category[]=indieweb&category[]=comments
Adding Files

When a Micropub request includes a file, the entire request is sent in the multipart/form-data encoding, and the file is named by content type, either "audio", "video" or "photo". A request may include one or more of these files.

When OwnYourGram makes a Micropub request to post a video, it also sends a photo which is a thumbnail preview of the video.

In PHP, these files are accessible using the $_FILES array: 

$_FILES['video']
$_FILES['photo']
$_FILES['audio']

see also: Micropub-brainstorming#Uploading_files_in_separate_requests

Note that there is no practical way to upload a file when the request body is JSON encoded. The media endpoint needs to be used in that case.

curl example: 

$ curl -H 'Authorization: Bearer abc'\
       -F 'h=entry'\
       -F 'content=an image!'\
       -F 'photo=@/home/cweiske/Bilder/openid-id.bogo.png'\
       'http://known.bogo/micropub/endpoint'

h-event

The following properties may be included in a request to create a new h-event:

Posting a new event

  • name
  • summary
  • description
  • start
  • end
  • duration
  • category
  • location
POST /micropub HTTP/1.1
Host: aaronparecki.com
Content-type: application/x-www-form-urlencoded

h=event
&name=IndieWeb Dinner at 21st Amendment
&description=In SF Monday evening? Join @caseorganic and I for an #indieweb dinner at 6pm! (Sorry for the short notice!)
&start=2013-09-30T18:00:00-07:00
&category=indieweb
&location=http://21st-amendment.com/


h-cite

The following properties may be included in a request to create a new h-cite:

(The following list is from microformats.org/wiki/h-cite)

  • name
  • published - date of publication of the work (not the date the h-cite was created)
  • author - URL of an h-card
  • url - a URL to access the cited work
  • content - the content or partial content of the work itself, such as when including a blockquote snippet of a work

Nested Microformat Objects

Whenever possible, nested Microformats objects should be avoided. A better alternative is to reference objects by their URLs. The most common example is including an h-card for a venue, such as checking in to a location or tagging a photo with a person or location. In these cases, it is better to reference the object by URL, creating it first if necessary.

For simple values such as an h-geo property with latitude and longitude, just use a Geo URI such as geo:37.786971,-122.399677

This technique has the advantage of ensuring that each object that is created has its own URL (each piece of data has its own link). This also gives the server an opportunity to handle each entity separately. E.g., rather than creating a duplicate of an existing venue, it may give back a link to one that was already created, possibly even merging in newly received data first.

For more complicated objects, it is better to first create an object on the target site so that it has its own URL, then reference that object's URL in the main request.

For example, creating a checkin post would involve two POST requests:

First create the venue by posting an h-card: 

POST /micropub

h=card
&name=Ford+Food+and+Drink
&url=http://www.fordfoodanddrink.com/
&street-address=2505 SE 11th Ave
&locality=Portland
&region=OR
&postal-code=97214
&geo=geo:45.5048473,-122.6549551
&tel=(503) 236-3023

Response: 

HTTP/1.1 201 Created
Location: http://example.com/venue/10

Then create the checkin post: 

POST /micropub

h=entry
&location=http://example.com/venue/10
&name=Working on Micropub
&category=indieweb

Response: 

HTTP/1.1 201 Created
Location: http://example.com/entry/1001
  • This technique has the advantage of ensuring that each object that is created has its own URL (each piece of data has its own link)
  • Also gives the server an opportunity to handle each entity separately. E.g., rather than creating a duplicate of an existing venue, it may give back a link to one that was already created, possibly even merging in newly received data first.

Handling a micropub request

The following error response shorthands are used in this prose implementation guide:

  • "Return unauthorized" means return an HTTP 401 response, optionally with a JSON-encoded unauthorized error payload as defined here
  • "Return forbidden" means return an HTTP 403 response, optionally with a JSON-encoded forbidden error payload as defined here
  • "Return invalid_request" means return an HTTP 400 response optionally with JSON-encoded invalid_request error payload as defined here
  • "Return insufficient_scope" means return an HTTP 403 response, optionally with JSON-encoded insufficient_scope error payload as defined here

To handle an HTTP request to a micropub-endpoint:

  • Look for the auth token
    • If the request has an Authorization: Bearer header, set auth_token to the value of the string after Bearer , stripping whitespace.
    • Otherwise, if the method is POST and the parsed content of the form-encoded request body contains an auth_token key, set auth_token to the value associated with that key.
  • If no auth_token was found, return unauthorized
  • If an auth_token was found, attempt to verify it. Details are out of the scope of micropub, refer to IndieAuth for more information.
    • If the auth_token was invalid, return forbidden
    • Otherwise, store the user information associated with the auth_token for reference later.
  • If any micropub-extensions are supported, detect whether the request is an extension request and handle it appropriately.
  • If the request is a GET request, parse the query string into query_params
    • If query_params contains a q key, store it under q
      • If q == "config", return a valid JSON-encoded configuration response as defined here
      • Otherwise, if q == "source", check query_params for a url key.
        • If no url key exists, or no content identified by query_params.url exists, return invalid_request
        • If content identified by query_params.url exists, but the current auth_token doesn’t grant the scope to view it, return insufficient_scope
        • Create an empty list called source_properties
          • If query_params contains the key properties, add the value associated under query_params.properties to source_properties.
          • Otherwise, if query_params contains one or more values associated with the key properties[], add each value to source_properties
        • Create and return a valid Source Content response based on the values of url and source_properties.
      • Otherwise, if q == "syndicate-to", return a valid JSON-encoded Syndication Target response.
      • If you ended up here, the request is an unknown GET request. Return invalid_request.
  • Otherwise, if the request is a POST request:
    • If the request has a Content-type: application/json header, parse the request body as JSON and assign the result to request_body. On a parsing error, return invalid_request
    • Otherwise, if the request has Content-type: x-www-form-urlencoded, parse the request body and assign the result to request_body. On a parsing error, return invalid_request
    • Otherwise, if the request has Content-type: multipart/form-data, parse the request body. Assign form parts to request_body and file parts to request_files
    • Otherwise, the request is in an unsupported format. Return invalid_request.
    • If request_body contains an action key:
      • If request_body.action == "delete":
        • If your endpoint doesn’t support deleting, return invalid_response.
        • Check request_body for a url key. If it doesn’t have one, or request_body.url doesn’t identify a piece of content which could be deleted, return invalid_request.
        • Check to see if access_token has the required scope to delete the content identified by request_body.url. If it doesn’t, return insufficient_scope.
        • Delete the content identified by request_body.url. Return an empty HTTP 204 response.
      • Otherwise, if request_body.action == "undelete":
        • If your endpoint doesn’t support undeleting, return invalid_response.
        • Check request_body for a url key. If it doesn’t have one, or request_body.url doesn’t identify a piece of content which could be undeleted, return invalid_request.
        • Check to see if access_token has the required scope to undelete the content identified by request_body.url. If it doesn’t, return insufficient_scope.
        • Undelete the content identified by request_body.url.
          • If the delete was successful and didn’t cause the content’s URL to change, return an empty HTTP 204 response.
          • Otherwise, if the delete was successful and did cause the content’s URL to change, return a HTTP 201 response with a Location: header pointing to the content’s new URL.
      • Otherwise, if request_body.action == "update":
        • If your endpoint doesn’t support updating, return invalid_response.
        • Check request_body for a url key. If it doesn’t have one, or request_body.url doesn’t identify a piece of content which could be updated, return invalid_request.
        • Check to see if access_token has the required scope to update the content identified by request_body.url. If it doesn’t, return insufficient_scope.
        • Handle the Update request according to the spec.
          • If the update was successful and didn’t cause the content’s URL to change, return an empty HTTP 204 response.
          • Otherwise, if the update was successful and did cause the content’s URL to change, return a HTTP 201 response with a Location: header pointing to the content’s new URL.
      • If you got here, an unknown action was provided. Return invalid_response
    • Otherwise (POST request without action key in parsed body), assume that the request is a create request.
    • Check whether access_token has the required scope to create new content. If not, return insufficient_scope.
    • If the request had a content-type other than application/json, normalize the form-encoded parameters into a canonical mf2 structure as follows:
      • Let normalized_body be {'type': ['h-entry'], 'properties': {}}
      • For each key, value pair in request_body:
        • If key == "h", set normalized_body.type to [value]
        • Otherwise, if key ends with "[]", assign normalized_body.properties[key] to a list of all values associated with key
        • Otherwise, assign value to normalized_body.properties[key]
      • Let request_body = normalized_body
    • Using request_body and request_files, handle a Create request according to the spec.
      • If the request was successful, return a HTTP 201 response with a Location: header containing the URL of the created content, and optionally Link rel=syndication and/or rel=shortlink headers.
  • If you ended up here, the request could not be handled. Return invalid_request

Additional notes for developers

  • The reply context is expected to be provided by the Micropub server. [5][6] and will include the entire post.
    • Brainstorm part-context replies etc. A lot of people on IndieWeb quote parts of the original post to clarify or draw extra attention to which part they are responding to, liked, or reposted.

Libraries

Open source libraries & implementations used to support micropub on the client app side and on the API endpoint side on the server:

Issues

Please use the Github repo for issues and discussion related to the current functionality found within the spec.

For issues that have been discussed and are ready to consider for inclusion in the spec, use the Micropub-brainstorming page to collect examples and iterate on the ideas.

Extensions

For adding new features and functionality based on the core Micropub spec, see the Micropub-extensions page to propose new Micropub extensions.

Sessions

See Also

Retrieved from "https://indieweb.org/wiki/index.php?title=Micropub&oldid=75990"