Skip to content

wordoftheday: read from official RSS feed (Merriam-Webster now returns 403) #3279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gderoo wants to merge 1 commit into
base: main
Choose a base branch
from
Open

wordoftheday: read from official RSS feed (Merriam-Webster now returns 403) #3279

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/wordoftheday/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Word Of The Day Applet for Tidbyt
---
Displays the Merriam-Webster Word Of The Day. Word and definition are scraped from the [WOTD Calendar](https://www.merriam-webster.com/word-of-the-day/calendar). Values are cached for 3 hours.
Displays the Merriam-Webster Word Of The Day. Word and definition are read from the official [WOTD RSS feed](https://www.merriam-webster.com/wotd/feed/rss2). Values are cached for 3 hours.

![Word Of The Day Applet for Tidbyt](screenshot.png)
32 changes: 21 additions & 11 deletions apps/wordoftheday/word_of_the_day.star
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ Author: greg-n

load("cache.star", "cache")
load("encoding/json.star", "json")
load("html.star", "html")
load("http.star", "http")
load("render.star", "render")
load("schema.star", "schema")
load("xpath.star", "xpath")

CACHE_KEY = "wotd"
CACHE_TTL = 10800 # 3 hours
WOTD_CALENDAR_URL = "https://www.merriam-webster.com/word-of-the-day/calendar" # Most succinct wotd definition
WOTD_RSS_URL = "https://www.merriam-webster.com/wotd/feed/rss2" # Official feed; merriam:shortdef is the most succinct wotd definition

def render_error():
return render.Root(
Expand All @@ -35,21 +35,31 @@ def main():
else:
print("Cache miss")

wotd_page_response = http.get(WOTD_CALENDAR_URL)
wotd_feed_response = http.get(WOTD_RSS_URL)

if wotd_page_response.status_code != 200:
print("Got code '%s' from page response" % wotd_page_response.status_code)
if wotd_feed_response.status_code != 200:
print("Got code '%s' from feed response" % wotd_feed_response.status_code)
return render_error()

selector = html(wotd_page_response.body())
word_parsed = selector.find(".wod-l-hover").first().text()
definition_parsed = selector.find(".definition-block").first().children().first().text()
feed_body = wotd_feed_response.body()

if word_parsed == "" or definition_parsed == "":
print("Failed to find word or definition from page")
# xpath.loads errors out on non-XML, such as a bot check page served with a 200
if "<rss" not in feed_body:
print("Feed response was not RSS")
return render_error()

# Values begin with lower cased letters on the calendar note cards
feed = xpath.loads(feed_body)

# Items run newest first, so the first one is today's word
word_parsed = feed.query("/rss/channel/item[1]/title")
definition_parsed = feed.query("/rss/channel/item[1]/merriam:shortdef")

# query returns None for a missing node
if not word_parsed or not definition_parsed:
print("Failed to find word or definition from feed")
return render_error()

# Values begin with lower cased letters in the feed
word = word_parsed[0].upper() + word_parsed[1:] + ":"
definition = definition_parsed[0].upper() + definition_parsed[1:] + "."

Expand Down
Loading