Delete redirected pages

[jnioche]

From a user

Links that were once pages and then turn to redirects are our issue. Our content management system auto creates clean URLs. If the title of the page is changed the clean URL is changed and the old URL is redirected to the new URL. The old URL stays in our index unless manually removed. When a link is changed from FETCHED to REDIRECT it would be ideal if the index is removed.

[circuscowboy]

With further though of the process it seems to make sense to replicate the FETCH_ERROR process. It tries x times to get the URL and if it remains a redirect it becomes an ERROR which would trigger the deletion bolt. With the x times variable you would be able to control whether the feature is used by setting it to 0 or -1 to always keep it active in the status like the current state.

[tomatotomata]

I was thinking about taking this on as a focused change in the redirect handling path. I would first trace how a FETCHED URL becomes REDIRECT, then add the smallest configurable retry or error transition that lets the existing deletion flow remove the stale index entry. I would keep the current behavior available by default, add regression coverage for repeated redirects, and avoid changing unrelated WARC or indexing code. Does that direction still fit the issue, and may I take it?