Also considering working on a traefik plugin + helm chart for sending LLMs that ignore robots.txt to a tarpit like iocaine/nepenthes
Also considering working on a traefik plugin + helm chart for sending LLMs that ignore robots.txt to a tarpit like iocaine/nepenthes