conneg
This article is a stub. You can help the IndieWeb wiki by expanding it.
conneg is short for HTTP Content Negotiation, a method by which a browser or other web client can request content of various types from a web server, and depending on what is requested, and what the server supports, it tries to provide the best it can.
IndieWeb Examples
Aaron Parecki supports content negotiation for post permalinks to switch between returning HTML, Microformats JSON and ActivityPub JSON.
- Example:
curl -H "Accept: application/activity+json" https://aaronparecki.com/2023/03/25/2/planner
- Example:
Caching
Content negotiation can pose additional challenges when trying to add caching layers, such as nginx's fastcgi or proxy cache. Unless the cache key takes into account the requested content type of the content, the wrong representation may be returned by the cache.
Tools
Criticism
- https://wiki.whatwg.org/wiki/Why_not_conneg
- https://snarfed.org/2023-03-24_49619-2
- "Content negotiation is the classic example of an idea that sounds good in theory, but for the vast majority of web developers, turns out to be net harmful in practice. A few reasons:
Most web developers are not aware that conneg exists. This is usually fine, since it’s uncommon, but can be an unpleasant surprise when they first hit it and don’t understand it.
The Accept header is notoriously complicated to parse, generate, and fully comply with. Imo much of that complexity is unneeded in practice.
It often breaks caching when it’s first introduced. Internal and external caches generally ignore Content-Type and Accept (etc) by default, so when conneg is first introduced, clients often get the wrong response type. Developers learn the hard way that they need to add Accept to Vary and to their framework’s cache.
URLs leak across contexts. They may start out within a single service or API, but they gradually proliferate into user-visible links, object identifiers in other systems, add-on services, third party tools, etc. These generally don’t know the original conneg requirements, which causes unpleasant surprises.
I think most of this boils down to: modality generally considered harmful. When something always behaves the same way, it’s reliable and easy to use. When it behaves differently based on something far away that you may not know exists, it’s unreliable and surprising. Add in a very large ecosystem of independent tools that all need to interoperate, often in fine-grained ways, and you have a recipe for failure." @Ryan Barrett March 24, 2023
- "Content negotiation is the classic example of an idea that sounds good in theory, but for the vast majority of web developers, turns out to be net harmful in practice. A few reasons:
- https://lists.w3.org/Archives/Public/public-swicg/2023Mar/0166.html
- "We used 'content negotiation' in Solid, and unfortunately, it has been a source of bugs for me. In my opinion, it was a non optimal design decision that added a lot of complexity." Melvin Carvalho March 24, 2023
- 2023-11-18 : Why I Tend Not To Use Content Negotiation (archived)
At this point in a conversation, someone who agrees broadly with my take on REST, Hypermedia-Driven Applications, etc. will often jump in and say something like
“Oh, it’s easy, you just use content negotiation, it’s baked into HTTP!”
Not being content with alienating only the general purpose JSON API enthusiasts, let me now proceed to also alienate my erstwhile hypermedia enthusiast allies by saying:
I don’t think content negotiation is typically the right approach to returning both JSON and HTML for most applications.
The alternative is to, as I advocate in Splitting Your APIs, erm, splitting your APIs. This means providing different paths (or sub-domains, or whatever) for your JSON API and your hypermedia (HTML) API.
- 2021-09-16 : Splitting Your Data & Application APIs: Going Further (archived)
- 2023-11-20 : Cloudflare does not consider vary values in caching decisions. (archived)
This means you can’t deploy an application that uses content negotiation via the Accept header behind the Cloudflare CDN—for example serving JSON or HTML for the same URL depending on the incoming Accept header.
- Further discussion on Lobsters (archived). Talking about how it probably is not a problem for most people, as Cloudflare would not have been caching the specific URLs anyway.