IndieWebCamp September 25 through October 2, 2015

post-type-discovery

Created by Tantek.com on September 26

• Sat, September 26 tantek.com draft stub, brainstorming link, next steps
• Sat, September 26 tantek.com more next steps
• Tue, September 29 tantek.com /* Brainstorming */ list which types I plan on writing up an algorithm for first
• Tue, September 29 tantek.com /* Brainstorming */ note other types being considered, and refine criteria
• Tue, September 29 tantek.com draft with license, methodology, algorithm, next steps, references, see also
• Tue, September 29 tantek.com /* Algorithm */
• Tue, September 29 tantek.com feedback
• Tue, September 29 tantek.com add explicit Participate section to header with links to Feedback, Issues, IRC
• Tue, September 29 tantek.com /* Next steps */ explicitly note plan to to use W3C wiki, github, irc if spec is accepted as an editor's draft in the Social Web WG
• Tue, September 29 tantek.com /* Other Types Under Consideration */ jam per request from KevinMarks
• Tue, September 29 tantek.com FAQ / What about a photo reply
• Tue, September 29 tantek.com /* What about a photo reply */ .
• Tue, September 29 tantek.com put algorithm first, methodology second as explanation for aspects of it
• Tue, September 29 tantek.com Use cases / Synthesizing explicit type formats
• Tue, September 29 tantek.com Implementations section / Granary
• Wed, September 30 tantek.com Normative References h-entry for property names in the algorithm, Activity Streams for JSON, microformats2-parsing for canonical microformats2 JSON, cite normative references inline where needed
• Wed, September 30 aaronparecki.com /* Issues */
• Wed, September 30 aaronparecki.com /* Issues */
• Wed, September 30 aaronparecki.com /* Issues */ videos
• Wed, September 30 tantek.com /* Algorithm */ use summary property if no content property, per issue from aaronpk re: summary
• Wed, September 30 tantek.com /* Issues */ resolved issue lack of content should use summary as fallback
• Wed, September 30 tantek.com /* Other Types Under Consideration */ add more types under consideration as pointed out by aaronpk. invitation, tag-of
• Wed, September 30 tantek.com /* Algorithm */ add line-breaks to make ORs and THENs clearer
• Wed, September 30 kylewm.com /* Issues */ note name.contains(content)
• Wed, September 30 tantek.com resolved issue, "unknown" is worse than "note" as fallback, display/presentation is outside the scope of this specification
• Wed, September 30 tantek.com resolve issue, reject video for now due to only one "current" u-video example. all other video examples are old or don't use u-video
• Wed, September 30 tantek.com /* Methodology */ recent (< 1 year old) permalinks
• Wed, September 30 tantek.com /* Issues */ respond to remaining issues with questions
• Wed, September 30 rhiaro.co.uk /* Next steps */
• Wed, September 30 kylewm.com /* Issues */ add example of name.contains(content), ask if there are examples of content.startswith(name)
• Wed, September 30 aaronparecki.com /* Issues */
• Wed, September 30 rhiaro.co.uk /* See Also */
• Wed, September 30 tantek.com status, editor's draft, mature enough though for implementation and feedback!
• Wed, September 30 tantek.com link implementations

Post Type Discovery specifies an algorithm for determining the type of a post by what properties it has and potentially what value(s) they have, which helps avoid the need for explicit post types that are being abandoned by modern post creation UIs.

Status

This is an Editor's Draft yet mature enough to encourage implementations and feedback therefrom!

Participate

Wiki (Feedback, Open issues)

IRC: #indiewebcamp on Freenode

Editor

Tantek Çelik

License

Per CC0, to the extent possible under law, the editor(s) and contributors have waived all copyright and related or neighboring rights to this work. In addition, as of 2015-09-30, the editor(s) and contributors (2015-04-07 onward) have made this specification available under the Open Web Foundation Agreement Version 1.0.

Use cases

Both creation user interfaces, and post presentation designs are evolving to directly use the presence or absence of specific properties (and their values) directly, rather than depending on any kind of explicit "post type", thus why bother discovering a post type in the first place? This section documents the (few) use-case(s) that is/are known to date.

Synthesizing explicit type formats

There are existing formats that require explicit post types (e.g. ActivityStreams), and code that consumes them expects explicit post types. Post type discovery enabling automatic synthesizing of such formats from posts that merely have a set of content related properties.

Algorithm

The Post Type Discovery algorithm ("the algorithm") discovers the type of a post given a data structure representing a post with a flat set of properties (e.g. Activity Streams (1.0 or 2.0) JSON, or JSON output from parsing [microformats2]), each with one or more values, by following these steps until reaching the first "it is a(n) ... post" statement at which point the "..." is the discovered post type.

1. If the post has an "rsvp" property with a valid value,
   
   Then it is an RSVP post.
2. If the post has an "in-reply-to" property with a valid URL,
   
   Then it is a reply post.
3. If the post has a "repost-of" property with a valid URL,
   
   Then it is a repost (AKA "share") post.
4. If the post has a "like-of" property with a valid URL,
   
   Then it is a like (AKA "favorite") post.
5. If the post has a "photo" property with a valid URL,
   
   Then it is a photo post.
6. If the post has a "content" property with a non-empty value,
   
   Then use its first non-empty value as the content
7. Else if the post has a "summary" property with a non-empty value,
   
   Then use its first non-empty value as the content
8. Else it is a note post.
9. If the post has no "name" property
   
     or has a "name" property with an empty string value (or no value)
   
   Then it is a note post.
10. Take the first non-empty value of the "name" property
11. Trim all leading/trailing whitespace
12. Collapse all sequences of internal whitespace to a single space (0x20) character each
13. Do the same with the content
14. If this processed "name" property value is NOT a prefix of the processed content,
    
    Then it is an article post.
15. It is a note post.

Quoted property names in the algorithm are defined in [h-entry].

Methodology

There are two important aspects to the methodology of the Post Type Discovery algorithm: scope (why is something explicitly in the algorithm), and order (why is something where it is in the algorithm).

Scope

The algorithm could attempt to cover innumerable potential hypothetical post types, or take an evidence based approach, focusing on real world publishing practices. This specification does the latter, specifically by placing a minimum bar of documented real world publishing practices of different visually apparent post types on the open web at recent (< 1 year old) permalinks, each with at least three independent implementations that have converged on what properties (and potentially values thereof) they have to imply their visually apparent post types. As a result of being evidence based, it is likely this specification will expand over time as more apparent post types are published by more convergent implementations.

Order

The algorithm must also specify an order (e.g. of precedence) that various properties (and their values) imply various post types. The algorithm is ordered by post types that are in general "richer" in terms of content as well as show greater cognitive effort by the author.

Other Types Under Consideration

Other types are being considered and will be included in the future iterations of the algorithm based on convergence of publishing patterns and critical mass of adoption thereof.

• event
• checkin
• bookmark
• quotation
• video
• audio
• jam
• invitation
• tag-of

Next steps

This section is for the editor, and may provide hints at updates to come for this specification.

• add explicit JSON examples based on both parsed microformats2 output, and ActivityStreams2
• add example how to infer from a W3C Activity Streams post similar to h-entry properties
• add example how to generate a W3C Activity Streams post from h-entry properties
  • Some notes at Generating an ActivityStream
• update: ISSUE-4: Do we rely on explicit typing or support implicit typing based on explicit property names?
• if accepted as an Editor's draft in the W3C Social Web Working Group:
  • keep a copy on the W3C wiki, e.g. https://www.w3.org/wiki/Post-type-discovery
  • use W3C Social Web github for issues
  • use #social on irc.w3.org for discussion - both of which, officially *instead of* email. Unofficial discussion are of course encouraged anywhere people want!
• end if

Feedback

Feel free to leave feedback about the draft here, please sign your name with ~~~~!

Issues

Open issues on this specification are documented here.

• Some of the entries on http://aaronparecki.com/articles have just a name, or a name and "featured" image, but no content or summary. Maybe the only way to tell this is an article is to go look at the canonical URL.
  • Example permalinks of this? Tantek 22:15, 29 September 2015 (PDT)
    • That is the example. The point is the h-feed contains partial information from the article, but the article permalinks have the full info. http://pin13.net/mf2/?url=http%3A%2F%2Faaronparecki.com%2Farticles Aaron Parecki 08:26, 30 September 2015 (PDT)
• The article vs. note check currently says to check if name is a prefix of content, but this fails the common auto-generated "name" case, where the name contains everything in the post including the content. I have had better luck checking name.contains(content) instead of content.startswith(name) Kylewm.com 21:24, 29 September 2015 (PDT)
  • Note with an implied "name" that contains the "content": https://shanehudson.net/2015/09/24/2042
  • What if we made it an OR? That is "name.contains(content) OR content.startswith(name)" means this is a note? Tantek 22:15, 29 September 2015 (PDT)
    • I can't think of anyone who gives their notes an explicit title that is a prefix of the content. The markup would be odd <div class="e-content"><span class="p-name">This is note</span> with a title that is a prefix</div>. I don't feel too strongly about it, but if we can't come up with any real-world examples, it's probably not worth complicating the spec with content.startswith(name)

Resolved

• In an h-feed, an entry may have a name and summary but no content. In this case it is an article, but step 6 would have already classified it as a note since there is no content. Perhaps update this step to read "If the post has no "content" property and no "summary" property"?
  • Steps 6 and following have been updated to use "summary" as a fallback for "content". Tantek 21:02, 29 September 2015 (PDT)

• It appears that a post with no content and no name would be classified as a note. Would it be better for a possible result of this algorithm to be "unknown"?
  • No. By having the known fallback of a note post, implementation for both publishing and consuming code is more predictable, and reliable. Tantek 21:40, 29 September 2015 (PDT)

• For example, when determining the post type of a comment, I may not want to display anything if the content and name are empty since it would just look like an empty post.
  • What you display is up to you (outside the scope of this specification), and "post type" should only be one aspect of what you use to decide what to display or present. Tantek 21:40, 29 September 2015 (PDT)

• There are several examples of people publishing video posts. Can we add "If the post has a "video" property with a valid URL, then it is a video post." before checking for the "photo" property? (Video posts often have a photo property as a fallback for consumers that don't understand video posts and to show a thumbnail of the video in a feed)
  • There is only one recent example of people publishing video posts with u-video (Ben Roberts), one person no longer publishing (last was 2+ years ago), and no other examples with u-video.[1] One < three - thus video stays as "under consideration" for now. Tantek 22:08, 29 September 2015 (PDT)

FAQ

What about a photo reply

Q: What about a reply that includes a photo?

A: It's a reply.

Q2: Should that show up as a "photo" post?

A2: It should show up as a "reply" and not be in a user's published feed of their photos. The user-centric design here is to treat replies separately, because in practice, when users post replies to others' posts, and include a photo, the photos typically assume the context of that other post, and would look odd outside of it (e.g. in a generic "photos" feed). In addition, by not including reply photos in a user's feed of their photos, it gives the user the freedom to reply to other posts with whatever they wish, including photos, and not have those reply-specific photos pollute their streams of "their stuff" that their followers subscribe to.

A2a: From a presentation perspective, a reply should primarily be displayed as a reply first, and then adapt accordingly to whatever other properties it may have.

Implementations

Implementations, in progress, partial, or complete, of Post Type Discovery.

Granary

Granary synthesizes ActivityStreams, microformats2, and Atom from various input feeds and sources, and as such has some code that can be considered in progress or even a partial implementation of Post Type Discovery:

• Live public demo site: https://granary-demo.appspot.com/
• Issue(s) related to implementing Post Type Discovery: #41

Normative References

• [Activity Streams 1.0]
  • for canonical Activity Streams 1.0 JSON
• [Activity Streams 2.0]
  • for canonical Activity Streams 2.0 JSON
• [h-entry] - h-entry microformats 2 vocabulary
  • for definitions of property names used in the algorithm
• [microformats2] - microformats2 parsing
  • for canonical microformats2 JSON output

Informative References

• h-entry
• RSVP
• reply
• repost
• like
• photo
• article
• note

See Also

• posts - specifically posts#Inferring_post_kinds_from_properties for prior work on implied post types.
• Post type discovery (my implementation pre-this-page) Rhiaro.co.uk 10:05, 30 September 2015 (PDT)