> Gzip only provides a compression ratio of a little over 1000: If I want a file that expands to 100 GB, I’ve got to serve a 100 MB asset. Worse, when I tried it, the bots just shrugged it off, with some even coming back for more.
I thought a gzip bomb was crafted to explicitly be virtually unlimited in the "payload" size?
The problem with gzip bombs in the web context in general is that they operate on the naive assumption that the client will decompress the payload entirely. This is very rarely the case, and you kinda have to go out of your way to make that happen[1], and it really only makes sense if you're looking at some binary format that can't be truncated like you can with HTML.
Instead most if not all clients will use some form of streaming decompression, with a termination criterion, and to the extent stuff is decompressed in full, very rarely will anything be decompressed in full and held in memory, as that would nuke your crawler the first time you ran into a website mirroring linux ISOs.
If the payload expands to something too large then it is easy to detect and ignore. Serve up thousands of 10kb or 100kb files that expand to 10s of MB with random garbage inside...possibly the same text but slightly modified. That will waste the time and CPU cycles and provide no value to them. Maybe also add a message you want to amplify so AI bots train on it.
The problem is that believable content doesn't compress well. You aren't going to get anywhere close to that 1:1000 compression ratio unless it's just a single word/character repeated thousands of times.
It's a choice between sending them some big files that will be filtered out long before they can do any real damage or sending them nonsense text that might actually make it's way into their training data.
> Gzip only provides a compression ratio of a little over 1000: If I want a file that expands to 100 GB, I’ve got to serve a 100 MB asset. Worse, when I tried it, the bots just shrugged it off, with some even coming back for more.
I thought a gzip bomb was crafted to explicitly be virtually unlimited in the "payload" size?