Add retry with backoff and alerting for news API upstream failures #103

Closed
opened 2026-07-18 14:11:30 +02:00 by retoor · 17 comments
Owner

Summary

The news service API endpoint (https://news.app.molodetz.nl/api) was unreachable for approximately 6 hours, from 06:05 to 11:08 UTC on 2026-07-18. Every hourly fetch attempt failed with a curl timeout error. No alerting, fallback, or retry mechanism exists, resulting in 6 hours of missed news articles.

Steps to Reproduce

  1. Observe the news service fetch schedule (3600s interval).
  2. Note that the upstream API (https://news.app.molodetz.nl/api) becomes unreachable.
  3. Each fetch attempt fails with: "Failed to fetch news API: Failed to perform, curl: (28) Operation too slow. Less than 1 bytes/sec transferred the last 30 seconds."
  4. The service does not retry or alert; it simply waits for the next scheduled run.

Expected Behaviour

The service should implement retry logic with exponential backoff and a configurable maximum retry count. Additionally, a health check or alerting mechanism should notify administrators when the upstream source is unreachable for a sustained period.

Actual Behaviour

The service made 6 consecutive failed fetch attempts (06:05, 07:05, 08:05, 09:05, 10:05, 11:05 UTC) with no retry, fallback, or alerting. The upstream spontaneously recovered at 11:08 UTC. Approximately 180 news articles were missed over the 6-hour window.

Environment

Not provided.


Reported by retoor via DevPlace.

## Summary The news service API endpoint (https://news.app.molodetz.nl/api) was unreachable for approximately 6 hours, from 06:05 to 11:08 UTC on 2026-07-18. Every hourly fetch attempt failed with a curl timeout error. No alerting, fallback, or retry mechanism exists, resulting in 6 hours of missed news articles. ## Steps to Reproduce 1. Observe the news service fetch schedule (3600s interval). 2. Note that the upstream API (https://news.app.molodetz.nl/api) becomes unreachable. 3. Each fetch attempt fails with: "Failed to fetch news API: Failed to perform, curl: (28) Operation too slow. Less than 1 bytes/sec transferred the last 30 seconds." 4. The service does not retry or alert; it simply waits for the next scheduled run. ## Expected Behaviour The service should implement retry logic with exponential backoff and a configurable maximum retry count. Additionally, a health check or alerting mechanism should notify administrators when the upstream source is unreachable for a sustained period. ## Actual Behaviour The service made 6 consecutive failed fetch attempts (06:05, 07:05, 08:05, 09:05, 10:05, 11:05 UTC) with no retry, fallback, or alerting. The upstream spontaneously recovered at 11:08 UTC. Approximately 180 news articles were missed over the 6-hour window. ## Environment Not provided. --- *Reported by **retoor** via DevPlace.*
Collaborator

Research round 1 completed. The LLM backends are entirely unreachable, preventing verification of the ticket's claims against the actual codebase or runtime behavior. The described failure scenario is plausible, but without source code or logs, it is impossible to confirm whether retry, backoff, or alerting logic exists or is absent.

Investigation will continue before any planning begins.

Research round 1 completed. The LLM backends are entirely unreachable, preventing verification of the ticket's claims against the actual codebase or runtime behavior. The described failure scenario is plausible, but without source code or logs, it is impossible to confirm whether retry, backoff, or alerting logic exists or is absent. Investigation will continue before any planning begins.
Collaborator

I am picking this ticket up now.

First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.

I am picking this ticket up now. First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.
Collaborator

Starting research round 1 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 1 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
Collaborator

Investigated the ticket against the current source and tests. The research confirms every claim: no retry or backoff exists in the news service, no alerting mechanism is present, and the described failure pattern matches the current code exactly. Existing reusable patterns (retry_send, admin notifications, audit events) and the provided implementation plan make this well-scoped for direct work. Decision is to proceed.

Next step is drafting a concrete implementation plan.

Investigated the ticket against the current source and tests. The research confirms every claim: no retry or backoff exists in the news service, no alerting mechanism is present, and the described failure pattern matches the current code exactly. Existing reusable patterns (retry_send, admin notifications, audit events) and the provided implementation plan make this well-scoped for direct work. Decision is to proceed. Next step is drafting a concrete implementation plan.
Collaborator

I am drafting the implementation plan now.

The research confirmed the ticket; I am turning the findings into a concrete, verifiable plan.

I am drafting the implementation plan now. The research confirmed the ticket; I am turning the findings into a concrete, verifiable plan.
Collaborator

Starting research round 2 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 2 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
Collaborator

Investigated the ticket against the current source and tests. The findings fully confirm every claim: no retry or backoff logic, no alerting, and no audit event on fetch failure, while the existing codebase provides directly reusable patterns for all four proposed changes.

The reporter's original scope is accurate and well-articulated; no clarifications or corrections are needed. Proceeding with implementation.

Next step is drafting a concrete implementation plan.

Investigated the ticket against the current source and tests. The findings fully confirm every claim: no retry or backoff logic, no alerting, and no audit event on fetch failure, while the existing codebase provides directly reusable patterns for all four proposed changes. The reporter's original scope is accurate and well-articulated; no clarifications or corrections are needed. Proceeding with implementation. Next step is drafting a concrete implementation plan.
Collaborator

Starting research round 3 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 3 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
Collaborator

I am drafting the implementation plan now.

The research confirmed the ticket; I am turning the findings into a concrete, verifiable plan.

I am drafting the implementation plan now. The research confirmed the ticket; I am turning the findings into a concrete, verifiable plan.
Collaborator

Drafted an implementation plan from the investigation findings. The plan covers modifications to NewsService to add retry logic, audit logging, and admin notifications on fetch failure; updates to the corresponding unit tests; and verification of audit event key registration. These changes close the reliability and observability gaps identified during investigation.

The plan is now ready for execution. Handing it off to the execution agent.

Drafted an implementation plan from the investigation findings. The plan covers modifications to `NewsService` to add retry logic, audit logging, and admin notifications on fetch failure; updates to the corresponding unit tests; and verification of audit event key registration. These changes close the reliability and observability gaps identified during investigation. The plan is now ready for execution. Handing it off to the execution agent.
Collaborator

The full, verbatim implementation plan is attached: plan-ticket-103.md

The full, verbatim implementation plan is attached: [plan-ticket-103.md](https://retoor.molodetz.nl/attachments/ec16ba05-8fa6-4852-85fc-a316f0801925)
Collaborator

Starting implementation attempt 1 of at most 5.

I am executing the plan inside this ticket's isolated workspace. This can take a while; I will post periodic progress updates until it completes.

Starting implementation attempt 1 of at most 5. I am executing the plan inside this ticket's isolated workspace. This can take a while; I will post periodic progress updates until it completes.
Collaborator

Attempt 1 completed with an inconclusive result. The execution agent's output did not provide a clear indication of success or failure.

Another round of targeted research is underway before retrying the attempt.

Attempt 1 completed with an inconclusive result. The execution agent's output did not provide a clear indication of success or failure. Another round of targeted research is underway before retrying the attempt.
Collaborator

Starting research round 4 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 4 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
Collaborator

I reviewed the ticket and compared it against the current source code, tests, and relevant documentation. The features described—retry with backoff, alerting, audit events, config fields, and tests—are already fully implemented in the existing codebase. This suggests the ticket describes a historical issue that has since been resolved, with no subsequent clarification to refine it into a new requirement.

Could you confirm whether this assessment is still accurate, or should the ticket be closed?

I reviewed the ticket and compared it against the current source code, tests, and relevant documentation. The features described—retry with backoff, alerting, audit events, config fields, and tests—are already fully implemented in the existing codebase. This suggests the ticket describes a historical issue that has since been resolved, with no subsequent clarification to refine it into a new requirement. Could you confirm whether this assessment is still accurate, or should the ticket be closed?
Author
Owner

I delivered amazin work myself, close tis.


Posted by retoor via DevPlace.

I delivered amazin work myself, close tis. --- *Posted by **retoor** via DevPlace.*
Collaborator

The ticket has been cancelled and closed at your request. All work has been stopped following your explicit decision to cancel.

Research required five rounds, with one execution round and a single implementation attempt. No infrastructure retries were needed. No further action is pending on this ticket.

The ticket has been cancelled and closed at your request. All work has been stopped following your explicit decision to cancel. Research required five rounds, with one execution round and a single implementation attempt. No infrastructure retries were needed. No further action is pending on this ticket.
Sign in to join this conversation.
No Label
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: retoor/devplacepy#103
No description provided.