Also, both Firecrawl and Firecrawl Simple are really simple, and most importantly don't have proxy service which is the heart of any crawler and scraper
It seems that there's a need for a benchmark to compare all solutions available in the market based on the quality and price
The majority of comments are related to prices and qualities
Also, is there any movements about product detection?
These days I'm looking for solutions that can recognize goods in high accuracy and show [brand][product_name][variant]
It's an implementation of the 'Library of Babel', as described by Borges in his short story of the same name. It is a library of every possible permutation of a limited alphabet, of books 1,312,000 characters long. The FAQ page [1] and the GitHub readme [2] explain it in much more detail.
AFAICT, this is a more advanced way of using Embeddings (which can encode for the vibes similarity (not an official term) of prompts) to determine where you get the most "bang for your buck" in terms of testing.
For instance, if there are three conversations that you can use to test if your AI is working correctly:
(1) HUMAN: "Please say hello"
AI: "Hello!"
(2) HUMAN: "Please say goodbye"
AI: "Goodbye!"
(3) HUMAN: "What is 2 + 2?"
AI: "4!"
Let's say you can only pick two conversations to evaluate how good your AI is. Would you pick 1 & 2? Probably not. You'd pick 1 & 3, or 2 & 3.
Because Embeddings allow us to determine how similar in vibes things are, we have a tool with which we can automatically search over our dataset for things that have very different vibes, meaning that each evaluation run is more likely to return new information about how well the model is doing.
My question to the OP was mostly about whether or not this "vibe differentiated dataset" was constructed prior to the evaluation run, or populated gradually, based on each individual test case result.
That's probably the intent, but I don't know if this actually achieves this (I have another comment that's about the use of bayesopt here). But even if it did, bayesopt operates sequentially (it's a Sequential Model-based Optimizer or SMBO) and so the trajectory of queries different LLMs evaluate would be different. Unless there is something to correct this cascading bias I don't know if you could use this to compare LLMs. Or obtain a score that's comparable to standard reported numbers.
On a different note, if all we want is a diverse set of representative samples (based on embeddings), there are algorithms like DivRank that do that quite well.
If you write something for one startup, you can use it in other startups too
So, some people like him fit easily for them all