diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..ddc17f3 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,87 @@ +# Repository Guidelines + +This file is the shared source of project instructions. Keep `CLAUDE.md` as an +`@AGENTS.md` import so both tools use the same guidance. + +## Commands + +```bash +bundle exec rake test # Run all tests +bundle exec ruby -Ilib:test test/base/base_test.rb # Run single test file +bundle exec rubocop # Lint +bundle exec rubocop -A # Auto-fix lint issues +bundle exec rbs-inline --output sig lib/ # Generate RBS from annotations +bundle exec rbs -I sig validate # Validate generated types +bundle exec steep check # Type check +bundle exec rake console # Interactive console +``` + +## Architecture + +Single-file library (`lib/simple-rss.rb`) using regex-based XML parsing for flexibility with malformed feeds. + +**Tag Syntax** (extend via `SimpleRSS.item_tags <<`): +- `tag` - simple element extraction +- `tag#attr` - attribute value (e.g., `media:content#url` → `media_content_url`) +- `tag+rel` - rel attribute matching (e.g., `link+alternate` → `link_alternate`) + +**Dynamic Accessors**: Feed tags become `attr_reader` methods at parse time. Item tags are hash keys with `method_missing` for dot notation. + +## Type Annotations + +Uses RBS inline syntax: +```ruby +# @rbs (String, Integer) -> Bool # method signature +attr_reader :name #: String # attribute type +@items = [] #: Array[untyped] # inline variable +``` + +## Changelog + +Update `CHANGELOG.md` in the same pull request as any user-visible feature, +behavior change, or bug fix. Record unreleased work under `## Unreleased` at the +top. Keep released versions newest first with headings formatted as +`## X.Y.Z - YYYY-MM-DD`. + +Write concise bullets that explain what changed for callers and why it matters. +Mention affected APIs, compatibility changes, and migration steps when relevant. +For bug fixes, describe the failing behavior and the corrected result. Link the +issue or pull request when useful. Include performance numbers only when they +were measured, and distinguish evidence from intended behavior. + +Do not copy commit logs into the changelog, claim planned work is implemented, +or add a release date before publishing. Correct inaccurate history when found; +do not silently move a change into a version that did not contain it. Routine +test, tooling, and prose-only edits do not need their own release-note bullet. + +When backfilling history, verify tag and commit boundaries, cite the source, and +distinguish source-history dates from package publication dates. Identify +intermediate source versions and the earliest repository import explicitly. + +## Release + +Before tagging a release, bump the version in both files: +- `simple-rss.gemspec` (`s.version`) +- `lib/simple-rss.rb` (`VERSION`) + +Both values must match. If they do not, CI may build and attempt to push the wrong gem version. + +Move the completed `Unreleased` notes into a section for that exact version and +the release date, then leave an empty `Unreleased` section for future work. +Check that the changelog, both version declarations, and the tag agree. Run the +test suite, RuboCop, RBS generation and validation, and Steep before publishing. +`Gemfile.lock` is ignored in this library; do not add it as part of a release. + +Then tag with `v*` to trigger automated RubyGems release: +```bash +git tag -a vX.Y.Z -m "Version X.Y.Z" +git push origin vX.Y.Z +``` + +Quick preflight checks: +```bash +rg -n "s.version" simple-rss.gemspec +rg -n "VERSION" lib/simple-rss.rb +rg '^## ' CHANGELOG.md +gem build simple-rss.gemspec +``` diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..286baf1 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,123 @@ +# Changelog + +Entries through 2.2.0 were backfilled from Git history. Their dates identify +release tags or version-changing commits; older package publication dates can +differ. Linked source boundaries identify the evidence for each entry. The +repository begins with a 1.1 import, so earlier releases are not reconstructed. + +## Unreleased + +- Use the first valid `pubDate`, `updated`, or `published` timestamp in + `latest`, `items_since`, and merge ordering. Malformed dates no longer make + `latest` raise or hide a valid fallback. Preserve the original field values, + retain source order for equal dates, and place undated entries after dated + entries, including those before 1970. Preserve merge identity rules and the + placement of unidentified entries. ([#62](https://github.com/cardmagic/simple-rss/pull/62)) +- Make documented relation accessors such as `item.link_alternate` and + `item[:link_alternate]` return the link's `href`, including when attributes + precede `rel` or the link contains nested media. Keep relation matching within + direct child links of the current entry. Preserve legacy keys such as + `item[:"link+alternate"]` and the existing `item.link` behavior; hash and JSON + serialization now include both relation keys. + ([#57](https://github.com/cardmagic/simple-rss/issues/57)) + +## 2.2.0 - 2026-03-29 + +- Add `feed_type`, feed validation through instance and class `valid?` methods, + and filtering with `items_since`, `items_by_category`, and `search`. +- Add instance and class `merge` methods to combine feeds, deduplicate identified + entries, and order them by date. Add `diff` to report added and removed entries + between feed snapshots. +- Add item `has_media?` and `media_url` helpers, feed-level `enclosures` and + `images` collections, and parsing for `media:description`, `itunes:duration`, + and `itunes:image#href`. Treat blank media URLs as absent when choosing a + usable URL. ([Source](https://github.com/cardmagic/simple-rss/compare/v2.1.0...v2.2.0)) + +## 2.1.0 - 2025-12-29 + +- Add `SimpleRSS.fetch` with timeouts, custom headers, redirects, and conditional + GET support. Expose ETag and Last-Modified values and return `nil` for a + `304 Not Modified` response. +- Add `as_json`, `to_json`, and `to_hash`, serializing time values as ISO 8601. + Add `to_xml` to generate RSS 2.0 or Atom output from a parsed feed. +- Include `Enumerable` so feeds support iteration, mapping, and filtering. + Add index access with `feed[index]` and date ordering with `latest(count)`. + ([Source](https://github.com/cardmagic/simple-rss/compare/v2.0.0...v2.1.0)) + +## 2.0.0 - 2025-12-28 + +- Add the `array_tags` option to collect multiple values for selected item tags. +- Support extracting attributes from channel/feed and item/entry elements with + the `tag#attribute` syntax. +- Normalize parsed strings to UTF-8 and skip empty or whitespace-only tag + definitions that could make parsing hang. +- Add inline RBS type annotations, Steep validation, and GitHub Actions checks + for modern Ruby versions. + ([Source](https://github.com/cardmagic/simple-rss/compare/f5e879b...v2.0.0)) + +## 1.3.3 - 2018-04-23 + +- Parse `media:content#duration`, making video duration available through + `media_content_duration`. Synchronize the gemspec and library version at + 1.3.3. ([Source](https://github.com/cardmagic/simple-rss/compare/51f0eb6...f5e879b)) + +## 1.3.2 - 2015-08-17 + +- Stop forcing parsed content to binary encoding during unescaping. Apply CDATA + removal and whitespace trimming after either unescaping branch, and add UTF-8 + fixture coverage. This date follows the version change in source; + [RubyGems](https://rubygems.org/gems/simple-rss/versions/1.3.2) records package + publication in April 2018. + ([Source](https://github.com/cardmagic/simple-rss/compare/f3768bd...76a56c6)) + +## 1.3.1 - 2013-12-16 + +- Restore Ruby 1.8 compatibility by checking for `force_encoding` before calling + it while unescaping content. + ([Source](https://github.com/cardmagic/simple-rss/commit/f3768bd)) + +## 1.3.0 - 2013-12-16 + +- Make dynamically generated feed accessors usable on Ruby 1.9 without changing + the visibility of private parser methods. +- Adjust regular expressions and encoding handling for Ruby 1.9, including + suppression of UTF-8 regexp warnings. + ([Source](https://github.com/cardmagic/simple-rss/compare/aaf86ad...0e2ce64)) + +## 1.2.3 - 2010-07-06 + +- Add `tag#attribute` mapping for item child elements. Parse Media RSS content + URLs, types, dimensions, thumbnails, titles, credits, and categories through + the corresponding attribute and element accessors. + ([Source](https://github.com/cardmagic/simple-rss/compare/0915f34...aaf86ad)) + +## 1.2.2 - 2009-06-22 + +- Replace evaluation of feed-field assignments with `instance_variable_set` + and generate feed readers directly. Copy the options hash when constructing + the parser. ([Source](https://github.com/cardmagic/simple-rss/commit/0915f34)) + +## 1.2.1 - 2009-06-22 + +- Introduce an options hash for `parse` and initialization, including a + `marshalable` option that skips feed-reader generation. This intermediate + version is recorded in Git; a package publication was not verified. + ([Source](https://github.com/cardmagic/simple-rss/commit/c9757d2)) + +## 1.2 - 2009-02-25 + +- Add `tag+rel` matching and built-in Atom link relations for alternate, self, + edit, and replies links. Add `feedburner:origLink` support. +- Remove CDATA delimiters and trim whitespace even when percent-unescaping is + unnecessary. ([Source](https://github.com/cardmagic/simple-rss/compare/40f3ca5...fa54025)) + +## 1.1 - 2006-02-02 + +- Earliest repository import: parse RSS and Atom from strings or IO objects, + expose feed and item fields through method access, provide RSS/Atom aliases, + and allow callers to extend the recognized feed and item tags. + ([Source](https://github.com/cardmagic/simple-rss/commit/437b6a6)) +- Subsequent commits retaining version 1.1 make percent-unescaping conditional + and add gem packaging metadata. These changes postdate the initial import. + ([Unescaping](https://github.com/cardmagic/simple-rss/commit/ac95fb4), + [packaging](https://github.com/cardmagic/simple-rss/commit/379f639)) diff --git a/CLAUDE.md b/CLAUDE.md index 529c522..43c994c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,55 +1 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Commands - -```bash -bundle exec rake test # Run all tests -bundle exec ruby -Ilib:test test/base/base_test.rb # Run single test file -bundle exec rubocop # Lint -bundle exec rubocop -A # Auto-fix lint issues -bundle exec rbs-inline --output sig lib/ # Generate RBS from annotations -bundle exec steep check # Type check -bundle exec rake console # Interactive console -``` - -## Architecture - -Single-file library (`lib/simple-rss.rb`) using regex-based XML parsing for flexibility with malformed feeds. - -**Tag Syntax** (extend via `SimpleRSS.item_tags <<`): -- `tag` - simple element extraction -- `tag#attr` - attribute value (e.g., `media:content#url` → `media_content_url`) -- `tag+rel` - rel attribute matching (e.g., `link+alternate` → `link_alternate`) - -**Dynamic Accessors**: Feed tags become `attr_reader` methods at parse time. Item tags are hash keys with `method_missing` for dot notation. - -## Type Annotations - -Uses RBS inline syntax: -```ruby -# @rbs (String, Integer) -> Bool # method signature -attr_reader :name #: String # attribute type -@items = [] #: Array[untyped] # inline variable -``` - -## Release - -Before tagging a release, bump the version in both files: -- `simple-rss.gemspec` (`s.version`) -- `lib/simple-rss.rb` (`VERSION`) - -Both values must match. If they do not, CI may build and attempt to push the wrong gem version. - -Then tag with `v*` to trigger automated RubyGems release: -```bash -git tag -a v1.3.4 -m "v1.3.4" && git push origin v1.3.4 -``` - -Quick preflight checks: -```bash -grep -n "s.version" simple-rss.gemspec -grep -n "VERSION" lib/simple-rss.rb -gem build simple-rss.gemspec -``` +@AGENTS.md diff --git a/README.md b/README.md index 079e1bb..749ceb8 100644 --- a/README.md +++ b/README.md @@ -15,9 +15,11 @@ A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. Design - Extensible tag definitions - Zero runtime dependencies -## What's New in 2.0 +## What's New in 2.x -Version 2.0 is a major update with powerful new capabilities: +See the [changelog](CHANGELOG.md) for release history and unreleased changes. + +The 2.x releases add: - **URL Fetching** - One-liner feed fetching with `SimpleRSS.fetch(url)`. Supports timeouts, custom headers, and automatic redirect following. @@ -237,6 +239,19 @@ SimpleRSS.item_tags << :"entry#xml:lang" | `tag#attr` | `:"media:content#url"` | `.media_content_url` | Attribute value | | `tag+rel` | `:"link+alternate"` | `.link_alternate` | Element with specific `rel` attribute | +Relation tags provide both underscore and legacy `+` hash keys. For example, +`item.link_alternate`, `item[:link_alternate]`, and `item[:"link+alternate"]` +return the same parsed value. Both keys appear in `to_hash`, `as_json`, and +`to_json`; they are ordinary hash entries, so reassigning one does not update the +other. The built-in link relations are `alternate`, `self`, `edit`, and `replies`. + +Link relation accessors read `href` from the first direct child link with the +requested explicit `rel` attribute. Attribute order and child markup do not +affect the URL. Links inside source metadata, content, or other nested elements +are excluded. A missing relation or `href` returns `nil`. Relative href values +remain relative. `item.link` retains its existing extraction behavior; it does +not select a canonical article URL across relations or feed formats. + ### Collecting Multiple Values By default, SimpleRSS returns only the first occurrence of each tag. To collect all values: diff --git a/lib/simple-rss.rb b/lib/simple-rss.rb index b364c56..5362efb 100644 --- a/lib/simple-rss.rb +++ b/lib/simple-rss.rb @@ -445,15 +445,66 @@ def parse_item_tag(item, tag, content, item_attrs = nil) # @rbs (Hash[Symbol, untyped], String, String) -> void def parse_rel_tag(item, tag_str, content) - tag, rel = tag_str.split("+") + tag, rel = tag_str.split("+", 2) return unless tag && rel - content =~ %r{<(rss:|atom:)?#{tag}(.*?)rel=['"]#{rel}['"](.*?)>(.*?)}mi || - content =~ %r{<(rss:|atom:)?#{tag}(.*?)rel=['"]#{rel}['"](.*?)/\s*>}mi + value = if tag == "link" + link_relation_href(content, rel) + else + content =~ %r{<(rss:|atom:)?#{tag}(.*?)rel=['"]#{rel}['"](.*?)>(.*?)}mi || + content =~ %r{<(rss:|atom:)?#{tag}(.*?)rel=['"]#{rel}['"](.*?)/\s*>}mi - return unless Regexp.last_match(3) || Regexp.last_match(4) + return unless Regexp.last_match(3) || Regexp.last_match(4) - item[clean_tag("#{tag}+#{rel}")] = clean_content(tag.to_sym, Regexp.last_match(3), Regexp.last_match(4)) + clean_content(tag.to_sym, Regexp.last_match(3), Regexp.last_match(4)) + end + return if value.nil? + + item[clean_tag("#{tag}+#{rel}")] = value + item[clean_tag("#{tag}_#{rel}")] = value + end + + # @rbs (String, String) -> String? + def link_relation_href(content, relation) + attributes = entry_link_attributes(content).find { |link| link["rel"]&.casecmp?(relation) } + return unless attributes + + attributes["href"] + end + + # @rbs (String) -> Array[Hash[String, String]] + def entry_link_attributes(content) + links = [] #: Array[Hash[String, String]] + depth = 0 + tokens = %r{||<\?.*?\?>|<(/?)([\w:.-]+)((?:[^<>"']|"[^"]*"|'[^']*')*)>}m + + content.scan(tokens) do + closing = Regexp.last_match(1) + tag = Regexp.last_match(2)&.downcase + attributes = Regexp.last_match(3) + next unless attributes + + if closing == "/" + depth = [depth - 1, 0].max + next + end + + links << xml_attributes(attributes) if depth.zero? && %w[link atom:link rss:link].include?(tag) + depth += 1 unless attributes.rstrip.end_with?("/") + end + + links + end + + # @rbs (String) -> Hash[String, String] + def xml_attributes(attributes) + values = {} #: Hash[String, String] + attributes.scan(/([\w:.-]+)\s*=\s*(?:"([^"]*)"|'([^']*)')/m) do + name = Regexp.last_match(1) + value = Regexp.last_match(2) || Regexp.last_match(3) + values[name.downcase] = value if name && value + end + values end # @rbs (String, String?) -> void diff --git a/simple-rss.gemspec b/simple-rss.gemspec index 0b3f750..5a97deb 100644 --- a/simple-rss.gemspec +++ b/simple-rss.gemspec @@ -5,9 +5,10 @@ Gem::Specification.new do |s| s.summary = "A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. It is designed to be backwards compatible with the standard RSS parser, but will never do RSS generation." s.email = "lucas@rufy.com" s.homepage = "https://github.com/cardmagic/simple-rss" + s.metadata["changelog_uri"] = "https://github.com/cardmagic/simple-rss/blob/master/CHANGELOG.md" s.description = "A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. It is designed to be backwards compatible with the standard RSS parser, but will never do RSS generation." s.authors = ["Lucas Carlson"] - s.files = Dir["lib/**/*", "test/**/*", "LICENSE", "README.md", "Rakefile", "simple-rss.gemspec"] + s.files = Dir["lib/**/*", "test/**/*", "LICENSE", "README.md", "CHANGELOG.md", "Rakefile", "simple-rss.gemspec"] s.required_ruby_version = ">= 3.1" s.add_development_dependency "rake" s.add_development_dependency "rdoc" diff --git a/test/base/relation_links_test.rb b/test/base/relation_links_test.rb new file mode 100644 index 0000000..f994f4d --- /dev/null +++ b/test/base/relation_links_test.rb @@ -0,0 +1,162 @@ +require "test_helper" +require "json" + +class RelationLinksTest < Test::Unit::TestCase + def test_documented_accessor_and_legacy_key_return_the_same_link + feed = parse_entry(<<~XML) + + + XML + item = feed.first + + assert_equal "https://example.com/posts/1", item.link_alternate + assert_equal item.link_alternate, item[:link_alternate] + assert_equal item.link_alternate, item[:"link+alternate"] + assert_equal "https://example.com/api/1", item.link + end + + def test_relations_accept_attribute_order_quotes_whitespace_and_paired_tags + item = parse_entry(<<~XML).first + + + Edit entry + + XML + + assert_equal "/posts/1?view=full", item.link_alternate + assert_equal "https://example.com/api/1", item.link_self + assert_equal "https://example.com/edit/1", item.link_edit + assert_equal "/posts/1/replies", item.link_replies + %w[alternate self edit replies].each do |relation| + assert_equal item[:"link_#{relation}"], item[:"link+#{relation}"] + end + end + + def test_nested_media_does_not_replace_the_link_href + source = File.read(File.join(__dir__, "../data/atom_nested_link.xml")) + feed = SimpleRSS.parse(source) + + assert_equal "https://example.com/story", feed.first.link_alternate + assert_equal "https://example.com/story", feed.first[:"link+alternate"] + assert_equal source, feed.source + end + + def test_relations_do_not_borrow_neighboring_attributes_or_content + item = parse_entry(<<~XML).first + + + + + XML + + assert_equal "/self", item.link_self + assert_equal "/replies", item.link_replies + assert_nil item.link_alternate + assert_nil item[:"link+alternate"] + assert_nil item.link_edit + end + + def test_relations_ignore_nested_source_content_comments_and_cdata + item = parse_entry(<<~XML).first + +
+ + ]]> + ?> + + + XML + + assert_equal "/entry", item.link_alternate + assert_nil item.link_self + assert_nil item.link_edit + assert_nil item.link_replies + end + + def test_relations_match_exact_element_and_attribute_names + item = parse_entry(<<~XML).first + + + + + + + XML + + assert_equal "/correct", item.link_alternate + assert_nil item.link_self + end + + def test_relations_stay_within_their_entry + feed = SimpleRSS.parse <<~XML + + Separate entries + + First + Second + + XML + + assert_equal "/first", feed.first.link_self + assert_nil feed.first.link_edit + assert_equal "/second", feed[1].link_edit + assert_nil feed[1].link_self + end + + def test_first_matching_relation_wins + item = parse_entry(<<~XML).first + + + XML + + assert_equal "/first", item.link_alternate + end + + def test_relations_preserve_case_insensitive_matching + item = parse_entry('').first + + assert_equal "/article", item.link_alternate + assert_equal "/article", item[:"link+alternate"] + end + + def test_hash_and_json_serialization_include_both_relation_keys + feed = parse_entry('') + expected = { title: "Entry", link: "/article", "link+alternate": "/article", link_alternate: "/article" } + + assert_equal expected, feed.first + assert_equal expected, feed.to_hash[:items].first + assert_equal expected, feed.as_json[:items].first + assert_equal expected, JSON.parse(feed.to_json, symbolize_names: true)[:items].first + end + + def test_custom_relations_preserve_legacy_keys_and_unrelated_tag_names + original_tags = SimpleRSS.item_tags.dup + SimpleRSS.item_tags.push(:"link+enclosure", :"custom:label+featured", :"custom:value") + item = parse_entry(<<~XML).first + + Featured + Unchanged + XML + + assert_equal "/episode.mp3", item.link_enclosure + assert_equal "/episode.mp3", item[:"link+enclosure"] + assert_equal "Featured", item.custom_label_featured + assert_equal "Featured", item[:"custom_label+featured"] + assert_equal "Unchanged", item.custom_value + ensure + SimpleRSS.item_tags.replace(original_tags) + end + + private + + def parse_entry(content) + SimpleRSS.parse <<~XML + + Relation links + Entry#{content} + + XML + end +end diff --git a/test/data/atom_nested_link.xml b/test/data/atom_nested_link.xml new file mode 100644 index 0000000..a8f90e4 --- /dev/null +++ b/test/data/atom_nested_link.xml @@ -0,0 +1,13 @@ + + Nested link media + + Story + + + + Thumbnail + + + + +