The /news endpoints look simple from the outside: ask for a player, get back recent headlines. Underneath, the hard part isn't collecting news — it's making sure one real-world event arrives as one row instead of five. This guide walks through how items are collected, matched to players, deduplicated, and aged out, and what that means for code you write against the feed.
A single roster move might surface as a reporter's short post, then a wire story, then two aggregators republishing that wire story under their own URLs, then a weekly roundup that mentions it in passing. That's five documents describing one event. A naive feed shows all five, and anything reading the feed — a lineup bot, a model, a language model asked "what's the latest?" — treats repetition as significance. Five copies of one injury report look like five injury reports.
So the pipeline is built around a single idea: an item's identity is the story it describes, not the document it arrived in.
Two kinds of source feed the pipeline, on independent schedules:
Both are reduced to the same normalized shape (title, summary, link, published_at) at the edge, so everything downstream is source-agnostic. Adding a source means writing a fetcher, not touching the matching or dedup logic.
Each source keeps a high-water mark of the newest item already ingested. A poll that finds nothing new costs one HTTP request and zero database reads. This is what makes a fast polling cadence affordable: without it, every run would re-read and re-hash the same items indefinitely.
Cursors advance only after a successful write. If a run fails partway, the next run re-reads the same window rather than skipping it — duplicate work is cheap and dedup absorbs it, whereas a skipped window is data you never get back.
Headlines refer to players inconsistently: full name in an article title, bare surname in a post ("Nabers feeling good"), nickname in a summary. Matching runs in two passes with deliberately different confidence:
This is tuned for precision over recall, on purpose. A missed headline is a gap; a wrongly attributed one is actively harmful, because a lineup decision gets made on news about a different player who happens to share a surname. Names that double as common words ("Love", "Chase", "Rice", "Sharp") are never matched on the surname alone, no matter how unambiguous they look.
/news/{player_id}/request endpoint exists for exactly this case — see the News & Injury Feed guide.Every item is tagged so consumers can weight it. Classification is rule-based and intentionally cheap — it is a sorting aid, not a judgement:
| Tag | Covers | Decays |
|---|---|---|
| injury | Designations, practice participation, availability, returns | Fast — days |
| role | Depth chart, snap share, target share, committee splits | Slow — weeks |
| transaction | Signings, trades, waivers, releases, activations | Slow — the whole season |
| general | Everything else | Fast |
The decay column isn't decoration — it drives the default lookback windows described further down.
This is where most of the engineering lives. Matching runs against the player's recent rows — scoped to one player, over a short window — in three tiers, ordered by how much each is trusted.
URLs are normalized before hashing: lowercased host, www/m/amp prefixes and AMP path suffixes removed, tracking parameters stripped, fragments dropped, remaining query parameters sorted. Two links to the same article therefore hash identically regardless of how they were decorated in transit.
This tier also does the heavy lifting across source types: when a social post links an article, that article's URL becomes the item's identity. The post and the feed item for the same story collapse into one row automatically, rather than arriving as a near-duplicate pair that some fuzzy matcher has to rescue later.
Casing, punctuation and smart quotes are stripped, then the title is hashed. This catches verbatim syndication: the same story, word for word, published at two different URLs. Before this tier existed, those reliably produced two rows.
For paraphrase and truncation — two people describing one event in their own words — items are compared by Jaccard overlap of content words, after reporter-attribution boilerplate ("per source", "sources tell", "coach X said") is removed, since that phrasing differs between outlets covering the same event.
On hand-labelled pairs, genuine duplicates score roughly 0.64–0.78 and unrelated-but-overlapping items 0.22–0.33, so the threshold sits at 0.45 in the gap.
Multi-topic items — live blogs, "winners and losers", weekly picks columns, semicolon-delimited grab bags — mention a dozen players and overlap loosely with everything. They were the dominant source of false positives in similarity testing: excluding them widened the gap between true and false pairs from a nervous 0.33/0.43 to a comfortable 0.31/0.64. They're detected, kept out of clustering entirely, and never become the primary record for a story.
Merging is non-destructive. The earliest row stays primary and keeps its headline, URL and timestamp; the re-telling is recorded on that row rather than added as a new one. When the same source re-delivers its own article with a drifted timestamp, the content is refreshed in place and the original sort key is kept, so the record doesn't flap.
News volume through an NFL week is wildly uneven, so a flat polling rate is wrong in both directions — too slow on Sunday morning, wastefully fast on Tuesday. The scheduler fires frequently and a cadence gate decides which invocations actually do work:
| Window (US Eastern) | Interval | Why |
|---|---|---|
| Sunday 11:00–20:00 | 5 min | Official inactives drop 90 minutes before the early kickoffs, then the afternoon slates run |
| Sunday early / night, Thu & Mon evening, late-season Saturday | 10 min | Pre-game ramp and primetime windows |
| Wed–Fri, 09:00–20:00 | 60 min | Practice reports and end-of-week injury designations |
| Overnight, Tuesday, offseason | 3 h | Genuinely quiet; roster churn only |
Sunday 11:00–11:30 Eastern is the highest-value window of the fantasy week — it's when a player goes from "questionable" to "out" and lineups have to change. That window is polled at the fastest rate the pipeline supports.
Keeping this logic in code rather than in a set of overlapping scheduled rules means it's unit-testable and free of the double-invocation bugs overlapping schedules produce. Windows are evaluated in US Eastern because that's how the NFL schedule is published, and the daylight-saving shift lands mid-season.
Two separate mechanisms, deliberately not conflated:
days window the API accepts. A shorter retention would silently truncate a caller asking for 90 days, which is worse than useless — it's wrong without saying so.days, the default is chosen from the impact filter:| Filter | Default lookback | Reasoning |
|---|---|---|
| injury | 10 days | A three-week-old "questionable" tag actively misleads |
| general | 14 days | Low signal, ages quickly |
| role | 30 days | Depth-chart changes explain usage for weeks |
| transaction | 45 days | Signings and trades set roster context all season |
| (no filter) | 14 days | Unchanged general-purpose default |
An explicit days parameter always wins, up to the 90-day maximum.
Three practical consequences.
Items are already one-row-per-story for a given player. If you're merging by headline on the client, you're duplicating work — and doing it with less context, since you can't see what was already folded in.
Omitting days is usually better than guessing at it, because the default already reflects how that class of news decays:
import requests
BASE = "https://api.gridirondata.com/api/v1"
HEADERS = {"x-api-key": API_KEY}
def availability_check(player_id):
"""Only current availability news — the 10-day injury default applies."""
r = requests.get(f"{BASE}/news/{player_id}",
params={"impact": "injury", "limit": 5},
headers=HEADERS, timeout=10)
r.raise_for_status()
return r.json()["news"]
def usage_context(player_id):
"""Why a player's role looks the way it does — 30-day role default."""
r = requests.get(f"{BASE}/news/{player_id}",
params={"impact": "role", "limit": 5},
headers=HEADERS, timeout=10)
r.raise_for_status()
return r.json()["news"]
for item in availability_check("Malik Nabers#WR"):
print(f"[{item['published_at']}] {item['headline']}")
Because re-tellings are folded rather than stacked, the number of items about a player is a measure of how many distinct things happened, not how loudly one thing was covered. That makes item counts usable as a signal — "three separate injury items this week" now means something. Under a naive feed it would only have meant one story got syndicated three times.