Google Read Aloud bot repeatedly crawls auth-gated endpoints (/messages, /notifications, /game) #75

New Issue
Open
opened 2026-07-13 00:37:23 +02:00 by retoor · 8 comments
retoor commented 2026-07-13 00:37:23 +02:00
Owner
Copy Link

Summary

Google's "Read Aloud" accessibility crawler is repeatedly hitting protected endpoints (/messages, /notifications, /game) that require authentication. The bot receives a 403 response but continues retrying, generating unnecessary audit log entries and server load.

Steps to Reproduce

  1. Observe server logs for requests from the Google-Read-Aloud user agent.
  2. Note repeated requests to /messages, /notifications, and /game from Google IPs (e.g., 74.125.208.x).
  3. Confirm that each request returns a 403 response.

Expected Behaviour

The crawler should either be blocked from requesting these paths entirely or receive a lightweight response that does not trigger audit logging or excessive server load.

Actual Behaviour

The bot receives a 403 response but continues retrying the same endpoints multiple times per day, generating unnecessary audit log entries and server load.

Environment

  • User-Agent: Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (compatible; Google-Read-Aloud; +https://support.google.com/webmasters/answer/1061943)
  • Affected endpoints: /messages, /notifications, /game
  • Source IPs: 74.125.208.x (Google IPs)

Reported by retoor via DevPlace.

## Summary Google's "Read Aloud" accessibility crawler is repeatedly hitting protected endpoints (`/messages`, `/notifications`, `/game`) that require authentication. The bot receives a 403 response but continues retrying, generating unnecessary audit log entries and server load. ## Steps to Reproduce 1. Observe server logs for requests from the Google-Read-Aloud user agent. 2. Note repeated requests to `/messages`, `/notifications`, and `/game` from Google IPs (e.g., 74.125.208.x). 3. Confirm that each request returns a 403 response. ## Expected Behaviour The crawler should either be blocked from requesting these paths entirely or receive a lightweight response that does not trigger audit logging or excessive server load. ## Actual Behaviour The bot receives a 403 response but continues retrying the same endpoints multiple times per day, generating unnecessary audit log entries and server load. ## Environment - User-Agent: `Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (compatible; Google-Read-Aloud; +https://support.google.com/webmasters/answer/1061943)` - Affected endpoints: `/messages`, `/notifications`, `/game` - Source IPs: 74.125.208.x (Google IPs) --- *Reported by **retoor** via DevPlace.*
plan-ticket-75.md
2.4 KiB
typosaurus commented 2026-07-20 00:04:17 +02:00
Collaborator
Copy Link

I am picking this ticket up now.

First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.

I am picking this ticket up now. First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.
typosaurus commented 2026-07-20 00:04:37 +02:00
Collaborator
Copy Link

Starting research round 1 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 1 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
typosaurus commented 2026-07-20 00:08:13 +02:00
Collaborator
Copy Link

I reviewed the ticket against the current source and tests, and checked outside documentation. The middleware now blocks the described paths for known crawlers with a lightweight 204 response and omits audit logging — directly matching the expected behaviour. The ticket’s claims of a 403 with continued audit logging no longer reflect the code. This does not appear to match the current state of the code. Could you confirm whether this is still accurate, or should the ticket be closed?

I reviewed the ticket against the current source and tests, and checked outside documentation. The middleware now blocks the described paths for known crawlers with a lightweight 204 response and omits audit logging — directly matching the expected behaviour. The ticket’s claims of a 403 with continued audit logging no longer reflect the code. This does not appear to match the current state of the code. Could you confirm whether this is still accurate, or should the ticket be closed?
typosaurus commented 2026-07-23 01:15:43 +02:00
Collaborator
Copy Link

I am picking this ticket up now.

First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.

I am picking this ticket up now. First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.
typosaurus commented 2026-07-23 01:16:41 +02:00
Collaborator
Copy Link

I need input to continue.

What I tried: Retried this step 3 times.
What I found: Each attempt failed for internal reasons unrelated to this ticket. The details are in the operator's logs; this is not a judgment about the ticket.

The system needs operator attention before I can continue. Reply here once things are healthy and I will resume from where I left off.

I need input to continue. What I tried: Retried this step 3 times. What I found: Each attempt failed for internal reasons unrelated to this ticket. The details are in the operator's logs; this is not a judgment about the ticket. The system needs operator attention before I can continue. Reply here once things are healthy and I will resume from where I left off.
typosaurus commented 2026-07-23 03:01:05 +02:00
Collaborator
Copy Link

I am picking this ticket up now.

First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.

I am picking this ticket up now. First I will bring up an isolated workspace with a fresh checkout and investigate whether the ticket matches the current state of the code before planning any change. I will announce every step here before I take it.
typosaurus commented 2026-07-23 03:01:25 +02:00
Collaborator
Copy Link

Starting research round 1 of at most 5.

I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.

Starting research round 1 of at most 5. I am now reading the relevant source, tests, and prior work in the live checkout to verify the ticket and gather the context needed for a concrete plan. This can take a while; I will post periodic progress updates until it completes.
typosaurus commented 2026-07-23 03:02:49 +02:00
Collaborator
Copy Link

After comparing the ticket against the current source and tests, and consulting outside documentation, the middleware already returns a lightweight 204 response for the Google-Read-Aloud bot on the listed paths. No audit logging or server load is added, which matches the ticket’s stated requirements. The reported behavior is therefore already in place.

This does not appear to match the current state of the code. Could you confirm whether this is still accurate, or should the ticket be closed?

After comparing the ticket against the current source and tests, and consulting outside documentation, the middleware already returns a lightweight 204 response for the Google-Read-Aloud bot on the listed paths. No audit logging or server load is added, which matches the ticket’s stated requirements. The reported behavior is therefore already in place. This does not appear to match the current state of the code. Could you confirm whether this is still accurate, or should the ticket be closed?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: retoor/devplacepy#75
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?