Plumbing in the context of the IndieWeb, refers to all the underlying code, backend setup, protocols, formats that is all merely there to support the design and user experience of a site, the actual user visible and interactive parts.
UX and design is more important than plumbing
One of the key IndieWeb principles is "UX and design is more important than protocols."
"I don’t care about the plumbing. I care if I can take a shower." Khurt on #indieweb
Yet programmers love to talk about plumbing, in the same way that construction workers do — it's their area of expertise, hobby, and fun to talk about.
However, when you're building your own site, you realize you just want the most minimal plumbing that is reliable and will do what is necessary for the design and user experience of your site.
Thus any discussion of plumbing should always be framed as a means to an ends of good UX/design.
Discussing plumbing for plumbing's sake tends to be a distraction from actually building and creating a functional site that you want to use day to day.
Exposed Plumbing Is Bad Design
Just as you would never live in a house with exposed plumbing like this, it's considered bad design if your site makes its plumbing obvious to the user, e.g. long seemingly random numbers in URLs (as opposed to good URL design), use of acronyms and jargon in your site description, and worst of all, when code or cryptic error messages show through and make your site look ugly.
Unnecessary Plumbing Is Neglected
Caption: What happens to most sites' RSS feed(s).
Whenever you build DRY violating duplicate plumbing into a site, e.g. feed files, more often than not, such plumbing ends up getting hastily patched, ignored, neglected, and eventually non-functional, because it's not what people primarily pay attention to, which is the visible site instead.