I've also found that sometimes, to maximize options, I have to first commit to an option. I've had situations before where I deliberated too long and missed the opportunity altogether.
Hi, I'm Chip, the author of the post. I spent waaaay too much time doing research for this. The AI engineering layer was especially hard because so many tools have similar and/or overlapping features. It was also a lot of pain trying to understand the repos that only have Chinese in the README do.
great list! as someone who's also trying to map the ai engineering landscape... i wonder what u think of adding other parts of the AI stack (https://www.latent.space/p/dec-2023). right now you have 4 categories and those are all in the text/code-heavy RAG/Agent world, but i think the space has broadened out a bit as i see it. for example, you could add:
- finetuning/other post-pretrain model tools (axolotl, mergekit <- all made and used by people without traditional ML engineer/researcher background)
- multimodal models/frameworks like vocode and comfyui
- AI UX tools like vercel ai sdk
- synthetic data generation tooling? whatever the nous pple have made
open question whether inference frameworks like llama.cpp/ollama or vllm and tgi count as AI Eng tools? again given the background of ggeranov and the students behind the other projects, arguably yes but ofc it starts to bleed into classical mlops here. (update: i see u have them in the "model development" category, ok fair)
IMO, the classical mlops is closer to the genai stack than most people think. E.g. experiment tracking is the same: with classical mlops, you experiment with hyperparams, with genai, you experiment with prompts. Similarly, finetuning is just an extension of training. Even vector databases for RAG is just vector search + databases, both of which have been around forever.
The post-train world is what I find to be the most fun. Techniques like model merging, constrained sampling, and all the new creative techniques for inference optimization and faster decoding are super cool!
Hey Chip, thanks for your contributions along the years and the amazing book.
2 questions: From what you researched, how many of those solutions are ready for production? and Regarding this mortality, what are the let's say top 5 things someone needs to think even before to do a PoC over those tools?
Production is a spectrum. Many of the repos I see are still demowares, but at the same time, most companies I've seen are also still at the PoC phase instead of massively scaling up their GenAI use cases.
I don't think the considerations for adopting a tool has changed. It starts from what problem you want to solve, the money/time budget you have for the solution, ROI of each solution.
I know it sounds generic, but without more detail, it's hard to give a more concrete answer!
Claypot AI | Founding Engineer (Infra / ML / Frontend) | REMOTE (US & OUTSIDE US) | Full-time | https://claypot.ai/careers.html
Claypot AI is a platform for real-time machine learning. Our platform unifies streaming and batch systems for online prediction, continuous evaluation, and continual learning.
We’re looking for engineers excited about machine learning and streaming tech to be the foundation of our engineering team.
What makes Claypot AI special?
* A culture of transparency, collaboration, ownership, and learning
* A very high bar for engineering craftsmanship
* Expertise in both distributed systems and ML
* An opportunity to win over a large, growing, yet untapped market for fast ML delivery
What will you get?
* Competitive compensation package
* Flexible remote-first culture with options for in-person collaboration
* Learn how to build a startup from the ground up
* Public speaking opportunities
* An environment for you to grow into the career you want
Reach out if you're interested, even if you don’t find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
Claypot AI is a platform for real-time machine learning. Our platform unifies streaming and batch to make it easier and cheaper for companies to do online prediction, continuous evaluation, and continual learning.
We’re looking for engineers excited about ML and streaming tech to be the foundation of our engineering team.
What makes Claypot AI special?
* A culture of transparency, collaboration, ownership, and learning
* A very high bar for engineering craftsmanship
* Expertise in both distributed systems and ML
* An opportunity to win over a large, growing, yet untapped market for fast ML delivery
What will you get?
* Competitive compensation
* Flexible remote-first culture with in-person component
* Learn how to build a startup from the ground up
* An environment for you to grow into the career you want
Reach out if you're interested, even if you don’t find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
Claypot AI | Founding Engineer (Infra / ML / Frontend) | REMOTE (US & OUTSIDE US) | Full-time | https://claypot.ai/careers.html
Claypot AI is a platform for real-time machine learning. Our platform unifies streaming and batch systems to make it easier and cheaper for companies to do online prediction, continuous evaluation, and continual learning.
We’re looking for engineers excited about machine learning and streaming tech to be the foundation of our engineering team.
What makes Claypot AI special?
* A culture of transparency, collaboration, ownership, and learning
* A very high bar for engineering craftsmanship
* Expertise in both distributed systems and ML
* An opportunity to win over a large, growing, yet untapped market for fast ML delivery
What will you get?
* Competitive compensation package
* Flexible remote-first culture with options for in-person collaboration
* Learn how to build a startup from the ground up
* Public speaking opportunities
* An environment for you to grow into the career you want
Reach out if you're interested, even if you don’t find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
Claypot AI is a platform for real-time machine learning. Our platform unifies streaming and batch systems to make it easier and cheaper for companies to do online prediction, continuous evaluation, and continual learning.
We’re looking for engineers excited about machine learning and streaming tech to be the foundation of our engineering team.
What makes Claypot AI special?
* A culture of transparency, collaboration, ownership, and learning
* A very high bar for engineering craftsmanship
* Expertise in both distributed systems and ML
* An opportunity to win over a large, growing, yet untapped market for fast ML delivery
What will you get?
* Competitive compensation package
* Flexible remote-first culture with options for in-person collaboration
* Learn how to build a startup from the ground up
* Public speaking opportunities
* An environment for you to grow into the career you want
Reach out if you're interested, even if you don’t find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
Claypot AI | Founding Engineer (Infra / ML / Full-stack) | REMOTE (US & OUTSIDE US) | Full-time | https://claypot.ai/careers.html
Claypot AI is a platform for real-time machine learning. Our platform unifies streaming and batch systems to make it easier and cheaper for companies to do online prediction, real-time evaluation, and continual learning.
We're looking for engineers excited about ML and streaming tech to be the foundation of our engineering team.
What makes Claypot AI special?
* A culture of transparency, collaboration, and ownership
* A very high bar for engineering craftsmanship
* Expertise in both distributed systems and machine learning
* An opportunity to win over a large, growing, yet untapped market for fast ML delivery
What will you get?
* Competitive compensation package
* Flexible remote-first culture with options for in-person collaboration
* Learn how to build a startup from the ground up
* Public speaking opportunities
* An environment for you to grow into the career you want
Reach out if you're interested, even if you don't find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
Claypot AI | Founding Engineer (Infra / ML / Full-stack) | REMOTE (US & OUTSIDE US) | Full-time | https://claypot.ai/careers.html
Claypot AI is a streaming-first platform for real-time machine learning. Our platform unifies streaming and batch systems to make it easier and cheaper for companies to do online prediction, real-time evaluation, and safe automated model retraining.
We're looking for engineers excited about ML and streaming tech to be the foundation of our engineering team. We're small, well-funded, with a healthy number of inbound interests from the market.
Reach out if you're interested, even if you don't find a job description that fits you. We believe in creating an environment for people to do their best, not squeezing people to fit into job descriptions. As the company grows, you can define the role that you want with us.
