Syndicating web content via RSS 2.0 XML allows feed aggregators, newsletter tools, and automated readers to consume website updates without relying on proprietary APIs. While modern Content Management Systems often provide built-in RSS feeds, custom web applications, static sites, or legacy platforms frequently require an automated mechanism to build valid RSS feeds directly from a list of URLs.

In this guide, we walk through building a production-ready Bash script that reads a list of unique website URLs, extracts HTML title tags, formats RFC-822 compliant timestamps, escapes XML special characters, and outputs a syntactically valid RSS 2.0 XML feed file.

RSS 2.0 XML Schema Overview

An RSS 2.0 document consists of a root <rss version="2.0"> tag containing a single <channel> element. Inside the channel, individual content items are defined using <item> blocks. Essential RSS fields include:

  • `<title>`: The headline or title of the post or article.

  • `<link>`: The absolute URL of the article (https://example.com/post-slug/).

  • `<guid isPermaLink="true">`: A unique identifier string for the item, typically matching the article URL.

  • `<pubDate>`: Publication timestamp formatted strictly per RFC-822 specifications (e.g. Wed, 13 Aug 2026 12:00:00 +0000).

  • `<description>`: Summary snippet or full body content encoded safely or wrapped in CDATA tags.

Production Bash Script: generate_rss.sh

Here is a standalone Shell script that processes a text file containing unique website URLs (urls.txt) and generates a complete rss.xml feed file:

generate_rss.shbash
#!/usr/bin/env bash
# generate_rss.sh - Automates RSS 2.0 XML generation from a list of web URLs
set -euo pipefail
 
INPUT_FILE="${1:-urls.txt}"
OUTPUT_FILE="${2:-rss.xml}"
SITE_TITLE="Lynxbee Developer Knowledge Platform"
SITE_LINK="https://lynxbee.com"
SITE_DESC="Technical guides, Linux systems engineering, and developer documentation."
 
if [ ! -f "$INPUT_FILE" ]; then
  echo "Error: Input URL list '$INPUT_FILE' not found!" >&2
  exit 1
fi
 
# Function to escape special XML characters
xml_escape() {
  sed -e 's/&/&amp;/g' \
      -e 's/</&lt;/g' \
      -e 's/>/&gt;/g' \
      -e 's/"/&quot;/g' \
      -e "s/'/&apos;/g"
}
 
# Current RFC-822 formatted date
BUILD_DATE=$(date -R)
 
# Write RSS Header
cat <<EOF > "$OUTPUT_FILE"
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>$(echo "$SITE_TITLE" | xml_escape)</title>
    <link>$SITE_LINK</link>
    <description>$(echo "$SITE_DESC" | xml_escape)</description>
    <lastBuildDate>$BUILD_DATE</lastBuildDate>
    <pubDate>$BUILD_DATE</pubDate>
EOF
 
# Process each URL in input file
while IFS= read -r URL || [ -n "$URL" ]; do
  # Trim whitespace and skip empty lines
  URL=$(echo "$URL" | xargs)
  [ -z "$URL" ] && continue
 
  # Fetch page title via curl and sed
  PAGE_TITLE=$(curl -sL "$URL" | grep -i -o '<title>[^<]*</title>' | sed -e 's/<[^>]*>//g' | xargs || true)
  if [ -z "$PAGE_TITLE" ]; then
    PAGE_TITLE="$URL"
  fi
 
  PUB_DATE=$(date -R)
 
  cat <<EOF >> "$OUTPUT_FILE"
    <item>
      <title>$(echo "$PAGE_TITLE" | xml_escape)</title>
      <link>$URL</link>
      <guid isPermaLink="true">$URL</guid>
      <pubDate>$PUB_DATE</pubDate>
      <description>Article link for $(echo "$PAGE_TITLE" | xml_escape)</description>
    </item>
EOF
 
done < "$INPUT_FILE"
 
# Write RSS Footer
cat <<EOF >> "$OUTPUT_FILE"
  </channel>
</rss>
EOF
 
echo "Successfully generated RSS feed at $OUTPUT_FILE"

Key Script Mechanisms Explained

  • `date -R` RFC-822 Formatting: Outputs timestamps in RFC-822 format (e.g. Thu, 13 Aug 2026 12:28:00 +0000), which is strictly required by RSS 2.0 readers.

  • XML Entity Escaping: Replaces raw reserved characters (&, <, >, ") with XML character entities (&amp;, &lt;, &gt;, &quot;) to prevent XML parsing errors.

  • Robust Stream Processing: Uses while IFS= read -r loop execution to safely parse URLs even if lines contain trailing whitespace.

Validating and Automating XML Generation

After generating your rss.xml file, validate its syntax using the Linux xmllint command line utility:

XML Validation & Cron Setup Commandsbash
# 1. Install libxml2 utilities on Ubuntu/Debian
sudo apt update && sudo apt install -y libxml2-utils
 
# 2. Validate generated RSS XML file format
xmllint --noout rss.xml
 
# 3. Schedule daily automated feed regeneration via crontab
# Open user crontab editor
crontab -e
 
# Add crontab entry to run script every night at midnight:
0 0 * * * /bin/bash /home/user/scripts/generate_rss.sh /var/www/urls.txt /var/www/html/feed.xml > /dev/null 2>&1

Troubleshooting Common RSS Generation Issues

  • XML Parsing Error: Unescaped `&` in Title: Occurs when titles contain unescaped ampersands (e.g. C & C++ Programming). Ensure all dynamic strings pass through xml_escape.

  • Invalid Date Format Errors in RSS Readers: RSS 2.0 requires RFC-822 timestamps (date -R). Standard ISO-8601 dates (date -Iseconds) will cause validation failures in strict feed aggregators.

  • Empty Titles Extracted: If target pages render content via client-side JavaScript (React/Next.js SPA without SSR), static curl title extraction will fail. Use pre-rendered meta title endpoints or headless Chrome scripts for dynamic web applications.