unlisted

From IndieWeb
(Redirected from unlisted URL)

This article is a stub. You can help the IndieWeb wiki by expanding it.


unlisted are publicly visible posts that are not included in a homepage or feed(s), and are typically excluded from site search features and broader web search services.

Similarity to private posts

Unlisted posts can be a simpler, easier to use approximation of private posts, especially when they're:

  • excluded from all feeds
  • excluded from site search
  • not linked from any public posts
  • blocked in search engines via meta noindex

The key differences are:

  • Unlisted posts don't require login or other authentication. Anyone can open an unlisted post's permalink and see it, immediately. This is meaningful for accessibility, especially for wider generations.
  • Unlisted posts are a bit less secure, since they depend on their URLs staying secret. Anyone who has an unlisted post's URL can open and view it.

A few additional layers of security you can consider:

  • Use Capability URLs for permalinks so that they're less guessable.
  • Put all unlisted posts under a common URL path prefix that's blocked from search engines in robots.txt.

IndieWeb Examples

Tantek Çelik

Tantek Çelik's original blog was unlisted (no posts shown on home page, no feed) for a couple years (2002-2003) with explicit blocking of search engines in the site's robots.txt. This helped make it feel "easier" to write more quickly, and about more topics than if everything was being indexed for searching. His thinking was that you would read these posts only in the larger context of that month's stream and thus have better understanding than just finding them directly via a search engine.

  • Subsequently (2004+), he occasionally wrote an unlisted post which had a public permalink summary in the archive of posts for that month, while still linking to the full post on a separate page.
  • -- some of the below may be worth moving to homepage#Tantek and summarizing here --
  • Mid 2006 he started showing his most recent post and other posts made in that same month on his home page tantek.com
  • In March of 2009 he stopped showing posts on his home page and replaced it with a temporary splash screen with just the text "As simple as possible." which redirected after 4 seconds to a hexagon design site map.
  • By July of 2009 he had incorporated the hexagon site map into his home page, no redirection, and soon after added a recommended events feed marked up in hCalendar with a button to subscribe to it.
  • 2010 he relaunched his blog, in January without a stream, in February with a home page stream of posts, which has continued to the present day (2019) without any unlisted posts since

Aaron Parecki

Aaron Parecki has some unlisted posts that do not appear in any normal feeds

  • Unlisted posts do not appear on the home page or feeds, but do appear on the page for the date the post was created, and also appear when searching
  • The URLs are still date-based and sequential, so are easily found by iterating, so this not considered a privacy measure

Martijn van der Ven

Martijn van der Ven has several unlisted posts that can only ever be accessed by people that received their URL

Eddie Hinkle

Eddie Hinkle has some unlisted posts.

  • Unlisted posts do not appear in the following areas:
    • homepage, feeds, local search, or archives.
      • They should logically not appear in search engines either, but he has not added any technical aspects to make that happen.
  • Unlisted posts on his site are not linked to anywhere else and require the user to know the URL in order to access it.

Ryan Barrett

Ryan Barrett uses unlisted posts for some photo albums. He prefers unlisted to private posts for accessibility: none of the people he shares pictures with have IndieAuth logins, nor would they set them up just to see pictures. He also has a couple experimental unlisted posts for the IndieWeb community to poke at:

Fluffy

fluffy uses Publ's 'HIDDEN' entry types for soft privacy on some posts, notably things that need to be linked to easily but shouldn't end up in an index based on a site crawl.

  • this post is a placeholder that missing legacy blog entries get redirected to
  • Has used unlisted posts for personal/private blog posts as well, shared via side channels to trusted people via e.g. Mastodon or Facebook
  • A word of caution, though: sharing a URL on many platforms still causes the URL to be retrieved by automated mechanisms such as page unfurlers and "helpful" caching systems. For anything that requires absolute privacy, this is not a very good approach, especially if one is a target of harassment/doxing as these unfurl mechanisms can end up leaking the information to all manners of places, or even submitting pages to archive.org for "safe keeping." There is also accidental leakage from people accidentally pasting the URL into a search box instead of an address field, for example, which can end up submitting these pages to search engine indexers. Because of this, Publ's HIDDEN post type sets the X-Robots-Tag header as another level of safety net.
    • Concrete example: I wrote an invitation to my birthday party and posted it as an unlisted post, and shared it to select people on Mastodon and Facebook. Before long, my access_log showed the URL being accessed by all sorts of third parties, many of which appeared to be automated link unfurlers but many others which seemed to be "curious" instance admins or people who stumbled across the post somehow (possibly due to how privacy is implemented on Mastodon).
  • Also worth noting that if any information about hidden entries is placed in robots.txt, that can alert certain bad actors about the presence of said entries, even if the entry itself isn't listed; if a website's CMS has a mechanism for automatically finding a private post via post ID, keep in mind that bored teenagers who love drama/"lulz" have no compunctions about brute-forcing post IDs trying to find something they see as scandalous.

indiebookclub

indiebookclub supports visibility options, including unlisted, since 2021-12-03. This depends on your Micropub server indicating it supports the unlisted visibility option first. Unlisted posts do not appear on the public profile pages or when browsing posts by ISBN. When an unlisted post is viewed directly, there is an indicator at the bottom right of the post "Visibility: 👻"

gRegor Morrill

gRegor Morrill added unlisted post support to gregorlove.com 2022-11-23. I wanted to not have all of the follow posts I create appear in my main notes stream.

Add yourself!

Add yourself here… (see this for more details)

Silo Examples

Facebook

Facebook provides the ability to the user to "Hide post from feed" which can be used to make a post unlisted.

(screenshot needed)

Brainstorming

public homepage and unlisted blog

Tantek Çelik: consider designing and documenting a Getting Started that is:

Designed for:

Not for:

  • people that maintain an active public social media presence
    • For them, the existing Getting Started and POSSE approach work well to displace their social media usage

Basic setup:

  • a public home page that links to
  • a separate entirely unlisted blog (like in a /blog subfolder or maybe just /2019 to allow changing policy later by year, deciding to leave past years unlisted) that
    1. still has (monthly etc.) archives (in addition to the composite stream at the top of the subfolder)
    2. has no separate feed file (as these are sometimes separately indexed, often harvested by bots, sometimes used to seed spamblogs)
    3. is blocked from bots and search, i.e. using site robots txt to block that entire subfolder i.e. with:
User-agent: Googlebot
Disallow: /blog

User-agent: msnbot
Disallow: /blog

User-agent: *
Disallow: /blog

Alternatively if the blocking is per year like Disallow: /2019 that provides easier future flexibility, yet more (once a year?) maintenance to add an explicit disallow for each unlisted year. From my ( Tantek Çelik) experience (see #IndieWeb_Examples above), this was worth doing, and nice that I didn't have to re-review (or edit / redact) previous years when deciding to start blogging publicly, discoverably. I've kept those few starting years unlisted since.

Additional useful features:

  • one-off "public" posts that *do* get search indexed, are discoverable etc., like a position piece to reach more people (the supposed "Medium" use-case)


Goals:

  • immediately have a public presence on the web (useful for contact, payment etc. as well as IndieAuth sign-ins), and
  • a "safer" place to write your thoughts where you don't have to worry about immediate search engine indexing / discovery.
  • ability to participate in the "social web" with peer to peer comments via Webmention etc. free from the toxicity of social media silos
  • if they use private-accounts on silos, then consider/allow POSSEing from their unlisted blog/posts to those private accounts
    • Should a POSSE post to a private silo account link back to an unlisted (yet publicly "viewable") original post permalink?
      • Maybe?
      • What is the least surprising default for this?
      • Does it make sense for backfeed on private silo posts to be displayed on an unlisted (yet publicly "viewable") original post?

crawling/indexing prevention

robots.txt is not respected by all the things, but making a URL password protected with basic access authentication will block any crawlers. A commonly used way is to display a message in the "realm" field with the expected username and password, however, this functionality has been missing from Google Chrome for years.

unlisted yet in archives and nav

Brainstorm by Tantek Çelik, feel free to contribute/iterate:

Strava (which calls all their posts "activities" as in physical activity) has an option called:

Mute Activity
Don’t publish to Home or Club feeds [☑️]
This activity will still be visible on your profile

and this feels like a feature that some (e.g. Tantek Çelik) want for various use-cases.

What Strava means by "Home feed" is like other social media "home" feeds which mean, the aggregate "feed" of everyone's activities (posts) that you're following.

Their "Club feed" is what shows up when someone views a "Club", that is, an aggregation of everyone's activities (posts) who has joined the Club. There are "Clubs" for cities, brands, sports teams, fitness groups, etc. and anyone can create a new club, like we could have an "IndieWeb" club on Strava if we wanted. (this feature description could be moved to Strava).

On an IndieWeb site, such posts would

Use-cases, when:

  • you feel like you might be posting "too much" and don't want to "spam" your own home page or your feeds
  • you want to post a bunch of notes, like in a stream of consciousness way, which you’re ok having publicly viewable, but would rather not stream because they may be rougher than your "usual" notes
  • you don’t want a post (or posts) to be the most recent thing(s) on your homepage or feeds, for the moment, but might make them visible in-stream eventually, after you’ve posted some other content (which you’d prefer show up at the top of your home page / feeds)
  • ...

There is no obvious name for this feature / kind of post (which is likely worth its own page) so here are some ideas:

  • hidden from stream
  • muted — following the Strava heading, like self-muting. This one doesn't quite sound right, because "mute" is already overloaded for "don't notify me about this post / or show me this person’s posts (of a specific type)" etc.)
  • unfed — as in both not sent to any feeds, and funny enough, not federated to anyone/anywhere else either.
  • unstreamed — parallel to unlisted
  • ... more suggestions?

See Also

Retrieved from "https://indieweb.org/wiki/index.php?title=unlisted&oldid=94370"