When a support ticket arrives: “My newsletter shows & instead of &”
One common real-world symptom that triggers support tickets is text that looks wrong: literal sequences like , ", or & appearing in emails, dashboards, or printed PDFs. Users expect an ampersand (&) or a straight quote (“) but instead see entity names or numbers. That confusion often comes from mismatched escaping/decoding steps between systems — templating engines, CMS editors, API clients, or mail providers.
This article explains what character references are, where and when to escape them, how to reason about attribute vs text contexts, and how to draft a concise support-ticket and fix template examples. It also points out security and privacy limitations of automated decoding and formatter tools.
Quick primer: what are character references and why they exist
A character reference is an instruction in HTML (and XML-like formats) that represents a character using a name or numeric code: & (ampersand), " (quotation mark), or (non-breaking space). Browsers and many HTML-aware tools convert these references back to characters when rendering. They exist to disambiguate special characters that otherwise would be interpreted as markup (for example, < starts a tag) or to express characters not easily typed.
Important assumption: when I say “escape” I mean converting literal characters into their HTML character reference form (e.g., & to &). “Decode” means converting references back into literal characters. Both are necessary in different contexts.
Ampersands, quotes, and non-breaking spaces: common troublemakers
Some characters cause more trouble than others because of how HTML parses content.
Ampersands (&)
The ampersand is the entry token for any character reference. If you include a raw & in HTML text, the parser expects either a name or a numeric reference. If your text legitimately contains an ampersand (company names, query strings), you must escape it to & in normal text and attribute contexts.
Quotes (” and ‘)
Quotes are delimiters for attribute values. In attribute values that are delimited by double quotes, you should escape double quotes as " (or use the opposite delimiter). In text nodes quotes are normally safe and need not be escaped, but many templating systems will escape them by default.
Non-breaking spaces ( )
Non-breaking spaces look identical to regular spaces in many renderers, but they prevent wrapping and can result in subtle layout problems (e.g., a long string not wrapping inside a container). They also sometimes get introduced by rich-text editors and copied content. Representations include or numeric form .
Attribute context vs. text context: different escaping rules
Escaping depends on whether the character appears inside element text content or inside an attribute. Treat each context separately.
Text node example
<p>ACME & Co. sells items for $10 </p>Rendered text should be: “ACME & Co. sells items for $10” — the ampersand is escaped in the markup but shown as & in the rendered output.
Attribute example
<a href="/search?q=rock&roll" title="ACME & Co.">Open</a>Inside href or title attributes you must escape & as & so the URL or title doesn’t break the attribute syntax. If you fail to escape and the value contains a quote, the attribute might break and allow markup injection.
Decoding surprises: when decoding goes wrong
Decoding issues usually stem from double-escaping or decoding in the wrong layer. Common cases:
- Double-escaped entities: content was escaped twice and displays as &amp; instead of & or &.
- Partial decoding: an email client decodes some entities but not others, leaving mixed output.
- Context confusion: backend returns an already-escaped string to a template engine that escapes again by default.
Example: if a backend stores a title as “ACME & Co.” and a frontend template escapes variables automatically, the result may render as “ACME &amp; Co.”
Support-ticket template and developer checklist
When reporting an escaping bug, include exact inputs, where the content is stored, and what you expect. Provide a minimal reproducible case.
Quick support-ticket template
Subject: Escaped entities shown literally in newsletter (example: " &nbsp;)
Problem: When sending the weekly newsletter, subscribers see text like "&quot;Hello&quot;" and "&nbsp;" instead of quotes and spaces.
Steps to reproduce:
1) Create a draft with this title: ACME & Co. "New" release
2) Save as HTML in CMS (WYSIWYG stores as HTML)
3) Send test email to bob+test@example.com
Observed: Email shows &quot; and &nbsp; literally.
Expected: Email shows proper quotes and spaces.
Attachments: raw HTML file, screenshot, CMS storage record (JSON)
Please check: whether CMS stores escaped HTML or raw text, and whether the mailer decodes entities before delivering.
Developer checklist (use before closing ticket)
- Confirm if the CMS stores raw text or pre-escaped HTML.
- Check whether the email renderer/template engine applies automatic escaping.
- Verify mail provider’s behavior: do they modify body HTML or send as-is?
- Test the minimal reproducer in a static HTML file to isolate the issue.
Practical workflow: how to debug and fix an escaping issue
- Collect the failing sample: copy the exact stored HTML and the rendered result (screenshot and raw source).
- Use a safe decoder to inspect stored values: try an HTML entity decode on the stored string to see if it is doubly-encoded. Example tools: HTML Entity Encoder and Decoder (can encode/decode entities) and HTML to Text Extractor (can extract visible text). Note limitations below.
- Render the stored HTML in a local static file. If the static file displays correctly, the problem is likely in the emailer or transport layer.
- Check each transformation step (editor → database → API → template → mailer) for automatic escaping or sanitization.
- Fix by ensuring only the layer that outputs final HTML performs escaping for its context. Avoid double-escaping: store raw text or canonical HTML consistently.
Concrete examples
Two practical code snippets show common fixes.
Example 1: escaping an ampersand in text
// Unsafe: storing raw in HTML
<p>ACME & Co.</p>
// Safe: store escaped in HTML
<p>ACME & Co.</p>
When a template engine outputs a variable into HTML text, ensure the engine knows whether the string is already HTML-safe. In many engines the correct approach is to store raw text and let the template escape it when producing HTML, or store trusted HTML and mark it as safe so the template does not escape it again.
Example 2: attribute escaping
// Unsafe attribute (will break if title contains ")
<a href="/search?q=rock&roll" title="ACME "New"">Search</a>
// Safe attribute
<a href="/search?q=rock&roll" title="ACME &quot;New&quot;">Search</a>
Markdown-style idea in a table
The table below summarizes input, entity form, and when to use each. Think of it like a short cheat-sheet you might paste into a support ticket.
| Input | Entity / Numeric | When to use |
|---|---|---|
| & | & / & | Always escape in HTML text and attributes; leave raw in plain-text contexts. |
| ” (double quote) | " / “ | Escape inside double-quoted attributes; optional in text nodes. |
| space that should not wrap | / | Use when you need to keep words together (e.g., “$10 USD”). |
Common mistakes
- Assuming a WYSIWYG editor stores raw text — many store HTML and introduce entities.
- Double-escaping: escaping on input and again on output.
- Not considering attribute context: forgetting to escape quotes inside attributes.
- Trusting a downstream service (mailer, printer, PDF generator) to preserve HTML exactly.
- Search-and-replace fixes that change legitimate entity names (e.g., converting to a space and losing non-wrapping behavior).
Limitations and privacy considerations
Automated tools can help you inspect and clean HTML but have limits. HTML Entity Encoder and Decoder can convert between entities and characters, help detect double-encoding, and show numeric vs named forms. It cannot automatically decide whether a string should be stored as raw text or as HTML-safe content in your system: that decision depends on your app’s architecture and security policy. HTML to Text Extractor can help reveal the visible text of an HTML fragment (handy to confirm what a user will see), but it does not preserve attribute boundaries or tell you which layer introduced an encoding step.
Privacy note: pasting sensitive content (personal data, private keys, or user records) into online tools may send that content to the tool operator. For privacy-sensitive debugging, run local utilities or sanitize the sample before using an external service.
Security note: decoding entities blindly can expose you to HTML injection if you then insert decoded content into a page without proper sanitization. Escaping is a mitigation, but correct usage depends on context. Do not assume decoding an entity string removes security risk; assess input origin and apply an appropriate sanitizer for the generation context.
FAQ
Q: Should I store HTML-escaped strings in the database?
A: Prefer storing raw text if your app is mostly text-centric and escape at output. If the content is structured HTML authored by users (e.g., rich-text), store the HTML but have a clear sanitation/escaping policy and mark trusted HTML as safe in templates to avoid double-escaping.
Q: Why does my email client show &nbsp; instead of a space?
A: The email body probably contains a literal &nbsp; entity string instead of the non-breaking space character. This can happen if the email generation step encoded entities twice, or if a text-to-HTML conversion step incorrectly escaped already-escaped HTML. Inspect the raw MIME source of the email to confirm.
Q: Can a decoder tool fix double-escaped text automatically?
A: Some tools can decode twice, but automatic fixes are risky: you might inadvertently decode entities that were intentionally present. Manual inspection and adjusting the process that produces the content is safer than applying blanket decoding.
Closing notes
Understanding when to escape and when to decode is about mapping the data lifecycle in your application: where content is entered, how it is stored, what transformations it goes through, and how the final renderer treats it. A concise support ticket with the original stored HTML, the rendered output, and a static reproducer will get you faster results than guesses. When in doubt, isolate the layer by rendering the raw stored HTML in a static file; that often reveals whether the problem is storage, templating, or delivery.
Sources
Editorial note: This guide is an educational overview. Confirm the output against the documentation and workflow that apply to your project.