refback

From IndieWeb


Refback is a linkback method that uses HTTP Referer headers, has an out-of-date WordPress plugin, partial usage by maybe one or two IndieWeb community members, and is no longer practical since modern browsers strip all but the domain name when sending such headers.

Browser Support

Both Firefox and Chrome send referrer headers by default, but only include the origin - stripping everything apart from the domain name. This is controlled by the referrer policy

Implementation Details

Refback gives you a single piece of data, the URL that your visitor came from. There are numerous things you can do with this URL.

  • Save it in a database or log file
    • This allows you to manually inspect and discover places that are sharing your website/sending you visitors.
    • To make the huge amount of data easier to manually inspect, consider showing only unique referrers. If you are working with plaintext log files, you can use tools like cut and sort.
  • Show it on your website
    • If you want to link back to people, you can put the linkbacks under your article in a Mentions section.
    • As with any user submissions, you should filter this for spam. Consider manually inspecting them after spam filters and modifications.
  • Analyze the data to improve your website
    • You can analyze your Referrer data in order to determine the most important sources of your traffic. This will give you a good idea of which platforms you need to focus on.
    • You can use proprietary tools like Google Analytics. It will provide valuable metrics about your visitors as well as a graph of your referrers (social networks and regular websites).
    • If you want to keep the data to yourself (which is a good idea), you can use open-source tools like AWStats and GoAccess.

Linkback Verification

Even though you don't have to do it, you may find it useful to filter your Refbacks to cut down the false positives. One easy was of doing this is downloading the referring page and checking its HTML to see if it links to you. Doing this will cut down on both false positives and spam. Doing these checks asynchronously in the background is important to keep your pages from blocking.

Here's an example code in Python that naively verifies linkbacks.

import requests
from bs4 import BeautifulSoup

def verify_linkback(source, target):
    html = requests.get(source).text
    soup = BeautifulSoup(html)
    return len(soup.find_all("a", href=target)) > 0

Software

Refback for WordPress plugin

IndieWeb Examples

Chris Aldrich

rubenwardy

rubenwardy has a script that tails his nginx logs looking for referrers to his blog pages. This currently just logs the URLs and referrers for manual investigation.

He found that most browsers strip all but the domain name, which makes this less useful for a webmentions-like feature. One option would be to crawl the source site looking for pages with such a link, but he hasn't implemented that yet.

Articles

See Also