I've been at a company that internally sends out fake links that log the user and links to an educational page on internet safety.
I honestly don't mind too much since it's a once a year thing (hacktober) and honestly companies should be trying to catch out employees who click any and all links.
Ok maybe you have the opinion that it's all crap right now. That's fine. But pretend it gets good. Pretend that instead of bothering with bands at some point in the future you just generate music to your tastes on the fly all the time.
Where does that leave Bandcamp? Do they market themselves as "fresh organic music" and live in that niche? What good does all the rights music companies own do if music generates on the fly?
I suspect a huge amount of lobbying incoming asap to stop this. Perhaps a law against AI generated music that's not owned by the RIAA? You might not like AI generated music but you should be very very cautious of those fighting it.
you just generate music to your tastes on the fly all the time
This might work for people who like music as wallpaper to help them study or drown out external distractions, but none of it is going to evoke much feeling or stir memories. It's like calling chewing gum food.
> Pretend that instead of bothering with bands at some point in the future you just generate music to your tastes on the fly all the time.
As someone who enjoys live music, I would still need a live band to play this on-the-fly generated music. I guess then you'll trot out AI holograms! but that sounds still as unappealing as your base case.
Typed a prompt and hit generate. No response after waiting some time. I scrolled down to existing sample music to get a sense of what it creates and hit play. Not one of the play buttons worked. Ok load up Chrome instead of Firefox, maybe they did some Chrome specific thing? Nope site's still broken and none of the samples under "Suno AI Music Gallery" actually work. There's a javascript error "invalid client" on clicking it. I'm not logged in i guess?
It did work on mobile but that seems like it presents a completely different site.
I think the URL should be suno.com, the link you posted is a different thing? Suno.com is the one I've used, I generally use it for DND type campaigns when I need custom music for scenes or background noises. It does pretty good sound effects and spoken word so sometimes I use it for that as well.
I specifically do remember comics poking fun at diversity initiatives. A quick search of "Dilbert comic about diversity" brings up some examples.
At the time i read those i probably thought they were on point. I've changed my views over the years. You can't keep them or you end up like Adams. That's probably the key to understanding him. He grew up in an era where black students were not allowed to attend white schools. The world changed. He didn't.
Funny enough, to get to actual representative diversity you need to explicitly hire underrepresented candidates and pass up on white dudes. Which Scott famously complained about.
> you need to explicitly hire underrepresented candidates and pass up on white dudes
If the initiatives that promoted diversity explicitly said that, they probably wouldn't have passed. The whole argument was about whether that was true because proponents would never be honest about that part so the public debate never got past that.
Arithmetic coding takes a prediction of the next bit and writes out exactly as many bits as needed to correct that prediction. The amazing part is that you can write out fractional bits. Eg. You predict the next bit is '1' with 75% probability? If it is 1 you only need to write out 1/2 of a bit (correcting that 25% portion). If it's 0 you need to write out 2.4bits. It may seem strange to work with 1/2 a bit but it works! (essentially the other half of the bit represents other future correction required). You might have heard of huffman coding which can't deal with fractional bits, arithmetic coding is a generalization of huffman coding that can deal with this.
Arithmetic coding is mathematically perfect at what it does. You will not waste a single bit using this algorithm to encode data given a prediction of that data.
So the entirety of modern compression techniques don't deal with the encoding/decoding side at all. What they deal with is modelling the data so far and making the most accurate prediction possible on the next bit of data (next byte also works, but working 1 bit at a time is easier to comprehend when learning arithmetic coding).
Incidentally the encoders and decoders essentially work exactly the same. Given the data read or data decoded so far predict the next bit. This part is exactly the same either way. The decoder would read the compressed file for the correction and the encoder would read the input file and write out the correction. The important part is "predict the next bit". This is what separates all the different compressors.
This is also where those of us experienced in this area try to correct people on the wrong track. A compression algorithm is never about the encoding side but instead 100% always about the prediction of the data. Can you build a model that can accurately predict what the next data to come is? That's what you need to do to make a better file compressor. The entropy encoding part is a completely solved problem already, don't bother re-solving that.
I don't understand. On average, for every 4 input bits we will get it right 3 times writing 0.5 bits each time and get it wrong once writing 2.4 bits once. So we write a total of 3 * 0.5 + 2.4 bits = 3.9 bits. The compressed output is 3.9/4 = 97.5% as big as the input. Not very compelling. What am I misunderstanding?
It's -log2(0.75) for getting a 75% chance right and -log2(0.25) for getting it wrong. I should have stated .4 bits and 2bits respectively not 0.5 and 2.4. Sorry! Good catch.
It's 3.2 vs 4bits. Now that may not seem huge but the probabilities tend to be at the more extreme ends if the predictor is any good. Once you start going towards the 99% range you get extreme efficiency.
Lossy and lossless are essentially fundamentally the same actually. All state of the art lossy compressors use arithmetic coding as an example and they still do prediction. Eg. your favourite video codecs predict not only the next bit in the 2D frame, but also the next bit when modelling past frames (becomes a 3D cube of data at that point) and they also do things like motion prediction of individual objects in frame to help make a better prediction. They all use arithmetic encoders to encode the data.
Where the lossy part comes in is the point at which humans notice/don't notice data being thrown away. Got a bit that was waaay out of line in the prediction and going to cost you 10bits to correct? Perhaps to humans it doesn't matter? Can we throw it away? This throwing away of data is often done before the prediction+compression stage (eg. maybe quantizing the color space to 8bits from 32bits?) but from there on it's the same thing.
To simplify: lossy compression = lossless compression + a perception model that can tell you what aspect of the data you can safely toss away without anyone noticing.
Thanks! That's really enlightening. Maybe this can help me settle a question I've had. If I have 4k video and I want a smaller file size (but suppose I still have a 4k tv), I always felt like I should get better quality at that file size by compressing it further than by reducing the resolution. Rather than deciding for myself what data to throw away, why not let the compression algorithm do it?
Lowering the resolution to match the typical TV resolutions is sensible but beyond that trusting the codec is always the better way. The codecs will adaptively compress portions of the video differently. The top left area that's in shadow for the next 30seconds? Maybe that areas effective resolution can be lowered? Etc. They can do things like this!
If you change the resolution or color space of the entire file you do that without consideration to where the extra details might have been needed.
So resolution should match typical output resolutions exactly and from there it's all on the codec.
The results at https://www.mattmahoney.net/dc/text.html explicitly add the size of the compressor itself to the result. Note the "enwik9+prog" column. That's what it's ranked on.
The reason to do this is that it's trivial to create a compressor that 'compresses' a file to 0 bytes. Just have an executable with a dictionary of enwik9 that writes that out given any input. So we always measure what is effectively the Kolmogorov complexity. The data+program as a whole that produces the result we want.
So those results add in the compressor size. The programs there generally have no dictionary built in or in the case of LLM based compressors, no pre-trained data. They effectively build the model as they process data. Not compressing much at all at the start and slowly compressing better and better as they go. This is why these programs do better and better with larger data sets. They start with 0 knowledge. After a GB or so they have very good knowledge of the corpus of human language.
This program here however is pre-trained and shipped with a model. It's 150MB in size! This means it has 150MB of extra starting knowledge over those models in that list. The top models in that list are the better compressors, they'll quickly out learn and overtake this compressor but they just don't have that headstart.
Of course measuring fairly this should be listed with that 150MB program size added to the results when doing a comparison.
The complexity of a simple turing machine is itty bitty, and you can bootstrap that into an x86 emulator in a matter of kilobytes, so when we're messing with 100MB files it's not a big factor.
I wonder if they gave the chess bot X seconds of thinking time in an era when computers were slower?
The way you set difficulty for turn based game ai is that you limit how far ahead the algorithm searches. If you set the lookahead based on compute time your difficulties will be way out of line if someone upgrades the CPU.
Something similar happened to the macOS chess game, which has always been bundled with OSX/macOS. Once upon a time it was easy to beat in easy mode, which restricted how long it could thing in advance.
When Big Sur rolled out around 2020, Apple introduced a bug which disabled the difficulty slider: no matter what it was set to, it was hard or impossible to beat. In macOS Sequoia, the Chess app got updated again, and supposedly they fixed the difficulty slider, but in the interval silicon improved so much that the old restraints (like think for only a second) mean little. The lowest levels play like a grand master.
is there some reason to implement it as a time limit instead of iterations or something else deterministic? it being affected by CPU speed or machine load seems obvious.
or whatever makes sense if “iterations” isn’t a thing, I know nothing about chess algorithms
It’s simpler. Chess is a search through the space of possible moves, looking for a move that’s estimated to be better than the best move you’ve seen so far.
The search is by depth of further moves, and “better” is a function of heuristics (explicit or learned) on the resulting board positions, because most of the time you can’t be sure a move will inevitably result in a win or a loss.
So any particular move evaluation might take more or less time before the algorithm gives up on it—or chooses it as the new winner. To throw a consistent amount of compute at each move, the simple thing to do is give the engine consistent amounts of time per move.
> To throw a consistent amount of compute at each move, the simple thing to do is give the engine consistent amounts of time per move.
The simple thing to do is give it a limit on the total number of states it can explore in its search.
If your goal is consistency, wall-clock time makes no sense. If I run 'make -j20', should the chess computer become vastly easier because the CPU is being used to compile, not search? Should 'nice -n 20 <chess app pid>' make the chess computer worse?
Should my computer thermal-throttling because it's a hot day make the chess computer worse, so chess is harder in winter?
If the goal is consistency, then wall-clock isn't the simple way to do it.
> It’s simpler than doing a limit on number of states
According to who?
A counter that you ++ each move sounds a lot easier to me than throwing off a separate thread/callback to handle a timer.
> Doing a time limit also enforces bot moving in a reasonable time.
It's designed for specific hardware, and will never have to run on anything significantly slower, but might have to run on things significantly faster. It doesn't need a time cutoff that would only matter in weird circumstances and make it do a weirdly bad move. It needs to be ready for the future.
> It puts a nice limit to set up a compromise between speed and difficulty.
Both methods have that compromise, but using time is way more volatile.
A time limit is also deterministic in some sense. Level settings used to be mainly time based, because computers at lower settings were no serious competition to decent players, but you don't necessarily want to wait for 30 seconds each move, so there were more casual and more serious levels.
Limiting the search depth is much more deterministic. At lower levels, it has hilarious results, and is pretty good at emulating beginning players (who know the rules, but have a limited skill of calculating moves ahead).
One problem with fixed search depth is that I think most good engines prefer to use dynamic search depth (where they sense that some positions need to be searched a bit deeper to reach a quiescent point), so they will be handicapped with a fix depth.
> One problem with fixed search depth is that I think most good engines prefer to use dynamic search depth (where they sense that some positions need to be searched a bit deeper to reach a quiescent point), so they will be handicapped with a fix depth.
Okay, but I want to point out nobody was suggesting a depth limit.
For a time-limited algorithm to work properly, it has to have some kind of sensible ordering of how it evaluates moves, looking deeper as time passes in a dynamic way.
Switch to an iteration limit, and the algorithm will still have those features.
Getting more thinking time tends to give surprisingly small improvements to playing strength. For a classical alpha-beta search based engine, for a given ply (turn) you might have ~20 moves to consider each depth of the search tree. If you're trying to brute force search deeper, a 10x increase in compute time or power doesn't even let you search an extra ply.
Elo gains for engines tend to come from better evaluation, better pruning, and better search heuristics. That's not to say that longer search time or a stronger CPU doesn't help, it just doesn't magically make a weak engine into a strong engine.
There is a strategy called alpha beta pruning meaning you can discard a lot of move options quickly based on the results of similar branches. That and caching similar board states means 20x options does not mean 20x CPU time.
True, although better pruning can massively lower the effective branching ratio compared to pure alpha-beta, making the algorithm benefit more from longer search time again (which is why pruning is so important).
Naming it the "Turbo" button rather than making "turbo mode" the default and then pressing a button for "slow" mode, IMO, was marketing genius, even though the results are the same.
Blizzard did a similar thing in World of Warcraft during the beta. After playing for a while, your character would get "exhausted" and start earning half experience for killing mobs. The only way to stop being exhausted would be to log off or spend a LONG time in an inn. At some point, they flipped the script. They made the "exhausted" state the default, and while offline or in an inn, you would gain a "rested" experience buffer, where you would earn double experience.
The mechanic worked exactly the same, but by giving it different terms, players felt rewarded for stepping away from the game occasionally, rather than punished for playing too long. They also marketed it as a way of giving players a way to "catch up" after spending a day or two offline.
The original intention behind the turbo button was to give a way to set the clock speed something closer to a 4.77 MHz Intel 8088 for the benefit of games that relied on CPU cycle timing. Therefore turbo was the default and slow mode the exception.
For some reason this feature persisted in PC compatibles long past having any useful purpose, e.g. toggling a 386 between 33 MHz and 25 MHz. Perhaps manufacturers feared any PC without such a button would be perceived as slower, even though as you say, it's really a slow-down button not a turbo button.
Yes of course you'll keep it on the fast speed as much as you can, not the slow speed. But it's still presented as fast being a bonus rather than slow being a malus.
Alternatively, since there's only one difficulty provided ("easy"), I wondered if the programmer have selected say, DifficultyLevels array index 0 meaning the easiest, but it was actually sorted hardest first.
Those Teslas in Australia are Chinese made too of course as are the majority of Teslas globally. USA made really doesn’t exist at all in Australia. It’s merely USA branded. Even the Ford Ranger that’s sold in Australia is made in Thailand.
Meanwhile visible light based tech is going up in price due to competing with ai on the extra gpu need while lidar gets the range/depth side of things for free.
Ideally cars use both but if you had to choose one or the other for cost you’d be insane to choose vision over lidar. Musk made an ill timed decision to go vision only.
So it’s not a surprise to see the low end models with lidar.
I wonder if ubiquity doesn’t effect the lidar performance? Wouldn’t the systems see each other’s laser projections if there are multiple cars close to each other? Also is
LIDAR immune to other issues like bright 3rd party sources? At least on iPhone I’m having faceid performance degradation. Also, I suspect other issues like thin or transparent objects net being detected.
With vision you rely on external source or flood light. Its also how our civilization is designed to function in first place.
Anyway, the whole self driving obsession is ridiculous because being driven around in a bad traffic isn’t that much better than driving in bad traffic. It’s cool but can’t beat a the public infrastructure since you can’t make the car dissipated when not in use.
IMHO, connectivity to simulate public transport can be the real sweet spot, regardless of sensor types. Coordinated cars can solve traffic and pretend to be trains.
LIDAR systems use timing, phase locking, and software filtering to identify and eliminate interference from other units. There is still risk of interference, resulting in reduced range, noise, etc.
I'm not a self-driving believer (never had the opportunity to try it, actually), but I'd say bad traffic would be the number one case where I'd want it. I don't mind highway driving, or city driving if traffic is good, but stop and go traffic is torture to me. I'd much rather just be on my phone, or read a book or something.
Agreed that public transportation is usually the best option in either case, though.
To me any kind of driving is torture. I don't want the responsibility, the risk, the chance of fines if I miss a speed sign somewhere. And if my car could self drive I could spend the time usefully instead of wasting it on driving. It would be amazing.
Right now I don't even have a car but for getting around outside of the city it's difficult sometimes.
Yeah, I feel ya. I don't mind it, but I'm far from loving it. What particularly stresses me out is how I can be screwed even doing everything correctly, if someone else screws up.
All reasons why I think public transit is the better solution over self driving cars. They're generally much safer, and also you get to do something while you're on the go. Pretty nifty, I think.
Yes that's why I don't own a car. In a big city public transit is amazing. I spend 20 bucks a month on unlimited travel. That won't even buy me a headlight bulb for a car these days lol. When I still owned one I had to pay for the car, insurance, road tax, fuel, maintenance, parking, tolls. It felt like it was dragging me down the whole time. It's insane how much costs add up.
I love public transport and an added benefit is: I don't have to go back to where I left it. I often take a metro from A to B, walk to C and then get a bus back to A or something. Can't do that with a car, as such I tend to walk a lot more now. Because it's a hassle-free option now. The world seems more open for exploration when I don't have to worry about returning to the car, or having a drink, or the parking meter expiring. I really don't get that people consider cars freedom.
Of course once you go outside the city it's a different story, even here in Europe. Luckily I don't need to go there so much. But that's something that should be improved. On the weekend here in the city the metro runs 24/7 and the regional trains really should too but they don't.
Unfortunately in my region highway traffic is quite congested, and so called "adaptive cruise control" is a game changer. I find it reduces fatigue by a lot. Usually the trucks are all cruising at the speed limit and I just hang with them. I only change lanes if they slow down or there's an obstruction etc.
There are regular 100+ car pileups in the central California valley due to fog. Cars crash in a lot of situations because the driver simply can't see. We need something better than vision to avoid these kinds of accidents.
Coordinated cars won't work unless all cars are built the same and all maintained 100% the same and regularly inspected. You can't have a car driving 2 inches from the car in front, if it can't stop just as fast as the car in front. People already neglect their cars, change brake compounds, and get stuck purchasing low quality brake parts due to lack of availability of good components.
Next time you see some total beater driving down the road, imagine that car 2 inches off your rear bumper, not even a computer can make up for poor maintenance. Imagine that 8000lb pickup with it's cheap oversized tires right in your rearview mirror with it's headlights in your face. It's not going to be able to stop either.
A combination of cameras, lidar, ridar, ultrasonic fused together have a strong sense of perception since they fill in each other's gaps. (short, long, different spectrums of electro-magnetic spectrum / sound).
The good news is they're all commodity hardware prices now.
Tesla removing radar and parking ultrasonic sensors was a self own. Computer vision inference is pretty bad when all the camera sees is a while wall when backing up.
Fog - Radar will perceive the car. Multi car crash, long range radar picks it up.
Bright glare from sun, lidar picks it up. Lidar misses something, camera picks it up.
Waymo has the correct approach on perception. Jam with sensor so they have superhuman vision of environment around.
They're wideband EM devices, so the problem of congested spectrum can be dealt with by the same sort of techniques used by WiFi and mobile phone services.
i imagine seismic has already well solved a lot of that.
you know a lot about the light you are sending, and what the speed of light is, so you can filter out unexpected timings, and understand multiple returns
If you have to choose one over the other, it has to be vision surely?
Even ignoring various current issues with Lidar systems that aren’t fundamental limitations, large amounts of road infrastructure is just designed around vision and will continue to be for at least another few decades. Lidar just fundamentally can’t read signs, traffic lights or road markings in a reliable way.
Personally I don’t buy the argument that it has to be one or the other as Tesla have claimed, but between the two, vision is the only one that captures all the data sufficient to drive a car.
For one, no one is seriously contemplating a LIDAR-only system, the question is between camera+LIDAR or camera-only.
> Lidar just fundamentally can’t read signs, traffic lights or road markings in a reliable way.
Actually, given that basically every meaningful LIDAR on the market gives an "intensity" value for each return, in surprisingly many cases you could get this kind of imaging behavior from LIDAR so long as the point density is sufficient for the features you wish to capture (and point density, particularly in terms of points/sec/$, continues to improve at a pretty good rate). A lot of the features that go into making road signage visible to drivers (e.g. reflective lettering on signs, cats eye reflectors, etc) also result in good contrast in LIDAR intensity values.
It's like having 2 pilots instead of 1 pilot. If one pilot is unexpectedly defective (has a heart attack mid-flight), you still have the other pilot. Some errors between the 2 pilots aren't uncorrelated of course, but many of them are. So the chance of an at-fault crash goes from p and approaches p^2 in the best case. That's an unintuitively large improvement. Many laypeople's gut instinct would be more like p -> p/2 improvement from having 2 pilots (or 2 data streams in the case of camera+LIDAR).
In the camera+LIDAR case, you conceptually require AND(x.ok for all x) before you accelerate. If only one of those systems says there's a white truck in front of you, then you hit the brakes, instead of requiring both of them to flag it. False negatives are what you're trying to avoid because the confusion matrix shouldn't be equally weighted given the additional downside of a catastrophic crash. That's where two somewhat independent data streams becomes so powerful at reducing crashes, you really benefit from those ~uncorrelated errors.
"In the camera+LIDAR case, you conceptually require AND(x.ok for all x) before you accelerate."
This can be learnt by the model. Let's assume vision is 100% correct, the model would learn to ignore LIDAR, so the worst case scenario is that LIDAR is extra cost for zero benefit.
This is not going to be true for a very long time, at least so long as one's definition of "vision" is something like "low-cost passive planar high-resolution imaging sensors sensitive to the visual and IR spectrum" (I include "low-cost" on the basis that while SWIR, MWIR, and LWIR sensors do provide useful capabilities for self-driving applications, they are often equally expensive, if not much more so, than LIDARs). Camera sensors have gotten quite good, but they are still fundamentally much less capable than the human eyes plus visual cortex in terms of useful dynamic range, motion sensitivity, and depth cues - and human eyes regularly encounter driving conditions which interfere or prohibit safe driving (e.g. mist/ fog, heavy rain/snow, blowing sand/dust, low-angle sunlight at sunrise/sunset/winter). One of the best features of LIDAR is that it is either immune or much less sensitive to these phenomena at the ranges we care about for driving.
Of course, LIDAR is not without its own failings, and the ideal system really is one that combines cameras, LIDARs, and RADARs. The problem there is that building automotive RADAR with sufficient spatial resolution to reliably discriminate between stationary obstacles (e.g. a car stalled ahead) and nearby clutter (e.g. a bridge above the road) is something of an unsolved problem.
The worst case scenario is that LIDAR is a rapidly falling extra cost for zero benefit? Sounds like it's a good idea to invest into cheap LIDAR just in case the worst case doesn't happen. Even better, you can get a head start by investing in the solution early and abandon it when it has obsolete.
By the way, Tesla engineers secretly trained their vision systems using LIDAR data because that's how you get training data. When Elon Musk found out, he fired them.
Finally, your premise is nonsensical. Using end to end learning for self driving sounds batshit crazy to me. Traffic rules are very rigid and differ depending on the location. Tesla's self driving solution gets you ticketed for traffic violations in China. Machine learning is generally used to "parse" the sensor output into a machine representation and then classical algorithms do most of the work.
The rationale for being against LIDAR seems to be "Elon Musk said LIDAR is bad" and is not based on any deficiency in LIDAR technology.
If you're on a desert island and you have 2 watches instead of 1, the probability of failure (defined as "don't know the time") within T years goes from p to p^2 + epsilon (where epsilon encapsulates things like correlated manufacturing defects).
So in a way, yes.
The main difference is that "don't know the time" is a trivial consequence, but "crash into a white truck at 70mph" is non-trivial.
It's different because the challenge with self-driving is not to know the exact time. You win for simply noticing the discrepancy and stopping.
Imagine if the watch simply tells you if it is safe to jump into the pool (depending on the time it may or may not have water). If watches conflict, you still win by not jumping.
I was responding to the parent who said if you had to make a choice between lidar and vision, you'd pick lidar.
I know there are theoretical and semi-practical ways of reading those indicators with features that are correlated with the visual data, for example thermoplastic line markings create a small bump that sufficiently advanced lidar can detect. However, while I'm not a lidar expert, I don't believe using a completely different physical mechanism to read that data will be reliable. It will surely inevitably lead to situations where a human detects something that a lidar doesn't, and vice versa, just due to fundamental differences in how the two mechanisms work.
For example, you could imagine a situation where the white lane divider thermoplastic markings on a road has been masked over with black paint and new lane markings have been painted on - but lidar will still detect the bump as a stronger signal than the new paint markings.
Ideally while humans and self driving coexist on the same roads, we need to do our best to keep the behaviour of the sensors to be as close to how a human would interpret the conditions. Where human driving is no longer a concern, lidar could potentially be a better option for the primary sensor.
> For example, you could imagine a situation where the white lane divider thermoplastic markings on a road has been masked over with black paint and new lane markings have been painted on - but lidar will still detect the bump as a stronger signal than the new paint markings.
Conflicting lane marking due to road work/changes is already a major problem for visual sensors and human drivers, and something that fairly regularly confuses ADAS implementations. Any useful self-driving system will already have to consider the totality of the situation (apparent lane markings, road geometry, other cars, etc) to decide what "lane" to follow. Arguably a "geometry-first" approach with LIDAR-only would be more robust to this sort of visual confusion.
Everyone is missing the point, including Karpathy which is the most surprising because he is supposed to be one of the smart ones.
The focus shouldn't be on which sensor to use. If you are going to use humans as examples, just take the time to think how a human drives. We can drive with one eye. We can drive with a screen instead of a windshield. We can drive with a wiremesh representation of the world. We also use audio signals quite a bit when when driving as well.
The way to build a self driving suite is start with the software that builds your representation of the world first. Then any sensor you add in is a fairly trivial problem of sensor fusion + Kalman filtering. That way, as certain tech gets cheaper or better or more expensive and worse, you can just easily swap in what you need to achieve x degree of accuracy.
> ...just take the time to think how a human drives...
We truly have no understanding of how the human brain really models the world around us and reasons over motion, and frankly anyone claiming to is lying and trying to sell something. "But humans can do X with just Y and Z..." is a very seductive idea, but the reality is "humans can do X with just Y, Z, and an extremely complex and almost entirely unknown brain" and thus trying to do X with just Y and Z is basically a fool's errand.
> ...builds your representation of the world first...
So far, I would say that one of the very few representations that can be meaningfully decoupled from the sensors in use is world geometry, and even that is a very weak decoupling because the ways you performantly represent geometry are deeply coupled with the capabilities of your sensors (e.g. LIDAR gives you relatively sparse points with limited spatial consistency, cameras give you dense points with higher spatial consistency, RADAR gives you very sparse targets with velocity). Beyond that, the capabilities of your sensors really define how you represent the world.
The alternative is that you do not "represent" the world but instead have that representation emerge implicitly inside some huge neural net model. But those models and their training end up even more tightly coupled to the type of data and capabilities of your sensors and are basically impossible to move to new sensor types without significant retraining.
> Then any sensor you add in is a fairly trivial problem of sensor fusion + Kalman filtering
"Sensor fusion" means everything and nothing; there are subjects where "sensor fusion" is practically solved (e.g. IMU/AHRS/INS accelerometer+gyro+magnetometer fusion is basically accepted as solved with EKF) and there are other areas where every "fusion" of multiple sensors is entirely bespoke.
Sorry if this is obvious, but are there actually any systems that "choose one over the other"? My impression's always been it was either vision + LIDAR, or vision alone. Are there any examples of LIDAR alone?
Don't ultimately even the ones which are vision + LIDAR ultimately have to choose priority in terms of one or the other for "What do you do if LIDAR says it is blocked and sight says it is clear' or visa-versa?" Trying to handle edge-cases where say LIDAR thinks that sprinker mist is a solid object and to swerve to avoid it and say vision which thinks that an optical illusion is a real path and not a brick wall.
Since the current traffic infrastructure was built for human drivers with vision, you’ll probably need some form of vision to navigate today’s roads. The only way I could picture lidar only working would be on a road system specially made for machine driving.
Roomba (specifically the brand of the American company iRobot) only added lidar in 2025 [1]. Earliest Roombas navigated by touch (bumping into walls), and then by cameras.
But if you use "roomba" as a generic term for robot vacuum then yes, Chinese Ecovacs and Xiaomi introduced lidar-based robot vacuums in 2015 [2].
> Earliest Roombas navigated by touch (bumping into walls)
My ex got a Roomba in the early 2010s and it gave me an irrational but everlasting disdain for the company.
They kept mentioning their "proprietary algorithm" like it was some amazing futuristic thing but watching that thing just bump into something and turn, bump into something else and turn, bump into something again and turn again, etc ... it made me hate that thing.
Now when my dog can't find her ball and starts senselessly roaming in all the wrong directions in a panic, I call it Roomba mode.
For full self driving sure but the more regular assisted driving with basic ‘knows where other cars are in relation to you and can break/turn/alarm to avoid collisions’ as well as adaptive cruise control lidar can manage well enough.
I think fsd should be both at minimum though. No reason to skimp on a niw inexpensive sensor that sees things vision alone doesn’t.
Given a good proportion of his success has rested on somehow simplifying or commodifying existing expensive technology (e.g. rockets, and lots of the technology needed to make them; EV batteries) it's surprising that Musk's response to lidar being (at the time) very expensive was to avoid it despite the additional challenges that this brought, rather than attempt to carve a moat by innovating and creating cheaper and better lidar.
> So it’s not a surprise to see the low end models with lidar.
They could be going for a Tesla-esque approach, in that by equipping every car in the fleet with lidar, they maximise the data captured to help train their models.
It's the same with his humanoid robot. Instead of building yet another useless hype machine, why not simply do vertical integration and build your own robot arms? You have a guaranteed customer (yourself) and once you have figured out the design, you can start selling to external customers.
> why not simply do vertical integration and build your own robot arms?
Robot arms are neither a low-volume unique/high-cost market (SpaceX), nor a high-volume/high-margin business (Tesla). On top of that it's already a quite crowded space.
The ways in which Musk dug himself in when experts predicted this exact scenario confirmed to me he was not as smart as some people think he was. He seemed to have drank his own koolaid back then.
And if he still doesn’t realize and admit he is wrong then he is just plain dumb.
I think there’s room for both points of view here. Going all in on visual processing means you can use it anywhere a person can go in any other technology, Optimus robots are just one example.
And he’s not wrong that roads and driving laws are all built around human visual processing.
The recent example of a power outage in SF where lidar powered Waymo’s all stopped working when the traffic lights were out and Tesla self driving continued operating normally makes a good case for the approach.
Didn't waymo stop operating simply because they aren't as cavalier as Tesla, and they have much more to lose since they are actually self driving instead of just driver assistance? Was the lidar/vision difference actually significant?
The reports I’ve read said that some continued to attempt to navigate with the street lights out, but that the vehicles all have a remote confirmation where they try to call home to confirm what to do. That ended up self DDoSing Waymo causing vehicles to stop in the middle of the road and at intersections with their hazards on.
So to clarify, it wasn’t entirely a lidar problem it was an need to call home to navigate.
> roads and driving laws are all built around human visual processing.
And people die all the time.
> The recent example of a power outage in SF where lidar powered Waymo’s all stopped working when the traffic lights were out and Tesla self driving continued operating normally makes a good case for the approach.
Huh? Waymo is responsible for injury, so all their cars called home at the same time DOS themselves rather than kill someone.
Tesla makes no responsibility and does nothing.
I can’t see the logic the brings vision only as having anything to do lights out. At all.
Yes... but people can only focus on one thing at a time. We don't have 360 vision. We have blind spots! We don't even know the exact speed of our car without looking away from the road momentarily! Vision based cars obviously don't have these issues. Just because some cars are 100% vision doesn't mean that it has to share all of the faults we have when driving.
That's not me in favour of one vs the other. I'm ambivalent and don't actually care. They can clearly both work.
They do, but the rate is extremely low compared to the volume of drivers.
In 2024 in the US there were about 240 million licensed drivers and an estimated 39,345 fatalities, which is 0.016% of licensed drivers. Every single fatality is awful but the inverse of that number means that 99.984% of drivers were relatively safe in 2024.
Tesla provided statistics on the improvements from their safety features compared to the active population (https://www.tesla.com/fsd/safety) and the numbers are pretty dramatic.
Miles driven before a major collision
699,000 - US Average
972,000 - Tesla average (no safety features enabled)
2.3 million - Tesla (active safety features, manually driven)
5.1 million - Tesla FSD (supervised)
It's taking something that's already relatively safe and making it approximately 5-7 times safer using visual processing alone.
Maybe lidar can make it even better, but there's every reason to tout the success of what's in place so far.
No, you're making the mistake of taking Tesla's stats as comparable, which they are not.
Comparing the subsets of driving on only the roads where FSD is available, active, and has not or did not turn itself off because of weather, road, traffic or any other conditions" versus "all drivers, all vehicles, all roads, all weather, all traffic, all conditions?
Or the accident stats that don't count an accident any collision without airbag deployment, regardless of injuries? Including accidents that were sufficiently serious that airbags could not or were unable to deploy?
The stats on the site break it into major and minor collisions. You can see the above link.
I have no doubt that there are ways to take issue with the stats. I'm sure we could look at accidents from 11pm - 6am compared to the volume of drivers on the road as well.
I wonder how much of their trouble comes from other failures in their plan (avoiding the use of pre-made maps and single city taxi services in favor of a system intended to drive in unseen cities) vs how much comes from vision. There are concerning failure modes from vision alone but it’s not clear that’s actually the reason for the failure. Waymo built an expensive safe system that is a taxi first and can only operate on certain areas, and then they ran reps on those areas for a decade.
Tesla specifically decided not to use the taxi-first approach, which does make sense since they want to sell cars. One of the first major failures of their approach was to start selling pre-orders for self driving. If they hadn’t, they would not have needed to promise it would work everywhere, and could have pivoted to single city taxi services like the other companies, or added lidar.
But certainly it all came from Musk’s hubris, first to set out to solve the self driving in all conditions using only vision, and then to start selling it before it was done, making it difficult to change paths once so much had been promised.
> And if he still doesn’t realize and admit he is wrong then he is just plain dumb.
The absolute genius made sure that he can't back out without making it bleedingly obvious that old cars can never be upgraded for a LIDAR-based stack. Right now he's avoiding a company-killing class action suit by stalling, hoping people will get rid of HW3 cars, (and you can add HW4 cars soon too) and pretending that those cars will be updated, but if you also need to have LIDAR sensors, you're massively screwed.
> The ways in which Musk dug himself in when experts predicted this exact scenario confirmed to me he was not as smart as some people think he was.
History is replete with smart people making bad decisions. Someone can be exceptionally smart (in some domains) and have made a bad decision.
> He seemed to have drank his own koolaid back then.
Indeed; but he was on a run of success, based on repeatedly succeeding deliberately against established expertise, so I imagine that Koolaid was pretty compelling.
To be frank, no one had a crystal ball back then, and stuff could go either way with uncertainty in both hardware and software capabilities. Sure Lidars were better even back then, but the bet was on catching up on them.
I hate Elon's personality and political activity as much as anyone, but it is clear from technical PoV that he did logical things. Actually, the fact that he was mistaken and still managed to not bankrupt Tesla is saying something about his skills.
Fair. So in a sense, the lidar vs camera argument ultimately can be publicly assess/proven through human babysitter (regulation permit) and accident rates. or maybe user adoptions.
Between anti-Musk sentiment, competition in self driving and the proven track record of Lidar, I think we’ll start seeing jurisdictions from Europe to New York and California banning camera-only self-driving beyond Level 3.
Nah, you don't need to ban anything. Just force the rule, that if company sells self driving, they are also taking full liability for any damages of this system.
It’s not a conundrum as much as an implementation detail. We’ve decided to hold Waymo accountable. We’re just ticking the boxes around doing that (none of which involve confusion around Waymo being responsible).
Personally, I'd argue that if the AI killed someone due to being incompetent (as in, a human in a fit state to drive would not have made this mistake), the punishment should go to the corporation that signed off on the AI passing all relevant tests.
The nature of the punishment does not necessarily follow the same rules as for human incompetence, e.g. if the error occurs due to some surprising combination of circumstances that no reasonable tester would have thought to test, which I can't really give an example of because anything I can think of is absolutely something a reasonable tester would have thought to test, but for the sake of talking about it without taking this too seriously consider if a celebrity is crossing a road while a large poster of their own face is right behind them.
Let me re-iterate my original caution: human drivers are really bad: more than 40,000 people die in car crashes every year! If a self driving cars makes mistakes that humans would not in some cases, but overall they would only cause 30,000 deaths per year then I want self driving required. Thus I want liability to reflect not perfection is required but that they are better than humans.
Don't get me wrong, perfection should be the long term goal. However I will settle for less than perfection today so long as it is better.
Though better is itself hard to figure out - drunk (or otherwise impaired drivers) are a significant factor in car deaths, as is bad weather when self driving currently doesn't operate at all. Statistics do need to make sure self driving cars are better than non-impaired drivers in all situations where humans driver before they can claim better. (I know some data is collected, but so far I haven't seen any independent analysis. The potentially biased analysis looks good though - but again it is missing all weather conditions)
The AI's benefits should be irrefutable, but this isn't as simple as "at least x10 better than human drivers", or any fixed factor, it's that whatever mistakes they do make, if you show the video of a crash to the general public, the public generally agrees they'd have also crashed under those conditions.
Right now… Tesla likes to show off stats that suggest accidents go down while their software is active, but then we see videos like this, and go "no sane human would ever do this", and it does not make people feel comfortable with the tech: https://electrek.co/2025/05/23/tesla-full-self-driving-veers...
Every single way the human vision system fails, if an AI also makes that mistake, it won't get blamed for it. If it solves every single one of those perception errors we're vulnerable to (what colour is that dress, is that a duck or a rabbit, is that an old woman close up facing us or a young woman from a distance looking away from us, etc.) but also brings in a few new failure modes we don't have, it won't get trusted.
By this logic, then we should also create a rule for regular, non-self-driving that says, if you have a car accident that kills someone, all your wealth is taken away and given to the victim's family. If we had a rule like this, then "you'd probably see much safer driving". Are you willing to drive under those circumstances? I am sure you will say yes, but it does not make your suggestion any less ridiculous.
I can think of one example where something similar works. The requirements from insurance companies on airline pilots are considerable tougher than the government ones because they are on the hook for ~$200m if they crash.
A big reason car companies don't worry much about killing pedestrians at the moment is it costs them ~$0.
About half our road fatalities are pedestrians. About 80% of those are intoxicated with alcohol. When you're driving at 40mph, at night, and some drunk guy chooses to cross the road, no amount of safety features or liabilities can save him.
Sure, cars can be safer for light collisions with pedestrians where the car is going slowly. Especially in the US where half the cars have a very high hood. But where I live the problem is not safer cars, it's drunk pedestrians.
I wonder how a Waymo would do with your drunks? Really the answer for that is probably more a different road layout so the drinking is separate from the traffic. I live near Soho in London which is full of drunk people in the streets but most traffic is blocked off there or doing 10 mph.
I’ve been paying more attention to Waymos recently.. and noting that it stops to let people cross that i didn’t even see first.
And sometimes at places that aren’t even a cross walk.
Im in DTLA frequently and I am almost even developing a secondary instinct to cover my brake and have an extra look around when a Waymo stops in a street.
Because it may be dropping off or picking up a rider or it saw something or someone I didn’t. Just happened Saturday in fact. I saw it do an abrupt stop when I was yielding to it at a “T” intersection and expected it to have the right of way and keep going. I didn’t proceed until I could figure out WHY it had just stopped, like “okay WHERE’S the passenger”
and then five or so people started running across the street in front of it that I would not have seen if that Waymo wasn’t there and I was clear to turn left.
As an added bonus it stayed stopped after they all crossed and I decided to be a jerk and turn left in front of it. It stayed stopped for me too. There’s no driver in it. It ain’t mad. XD
I have a good eye for spotting uber drivers who are about to load or unload too,
Especially if they have some common sense and are trying to line up to do that so their passenger can get on or off curbside. A Waymo is just.. way more immediately identifiable that I can react that much faster to it or just be like.. alright. I’ll take a cue from it, it’s usually right.
And hell even if it’s wrong, maybe this isn’t a good time to pull out in front of it anyway!
We cannot even properly ban asbestos, expecting people to die first is just having a realistic perspective on how the US government works WRT regulations.
That's a legal non-starter for all car companies. They would be made liable for every car incident where self-driving vehicles were spotted in close vicinity, independently of the suit being legit. A complete nightmare and totally unrelated to the tech. Makes would spend more time and tech clearing their asses in court than building safe cars.
It wasn't ill timed. Any sane leader would understand both size and cost of tech always comes down rather quickly over time. He's just refused to accept having lidar uglify his cars or wait for it to get smaller. He instead fabricates about humans don't have lidars so cars shouldn't have them and sold "no lidars on Teslas" as an advantage instead of the opposite and refuses to accept the truth due to needing to feed his ego. Firing all non-yesmen didn't help either.
That strikes me as a weak argument. We have vehicles with legs, but wheels are just better, practically speaking.
The argument is that humans provide a proof-of-concept that vision + neural net can drive a car, because (for some reason) some people doubt this is possible. However there's no need for a "proof of concept" that mechanical legs can be built, because everyone already know that building mechanical legs is possible.
Maybe, but those claims have yet to be shown in the real world.
Waymo has certainly had its share of issues lately on the "practicality" axis. The cost of (actually good enough) LiDAR hardware doesn't help practicality either.
> Maybe, but those claims have yet to be shown in the real world.
They surely have as Waymo have a much better safety record than Tesla and that's even with Tesla hiding and obscuring their crash data - probably outright lying about most crashes if Musk has any input to their processes.
The big advantage of Lidar is that it can easily pick out obstructions and that's a good ability for a driver to have, whereas Teslas can get fooled by shadows etc. It seems obvious to me that having cameras and Lidar allows for much better analysis of traffic and pedestrians.
>They surely have as Waymo have a much better safety record than Tesla
Tesla is already closing in on Waymo to within a factor of 2[0], and this is less than 6 months in, with Waymo having 125 million autonomous miles and Tesla having under a million. Bet on whichever horse you want, but this trend is not looking good for Waymo.
>It seems obvious to me
Oh thank goodness! Fortunately this style of argument has never been wrong before. ;)
Some companies work on reducing the size of it so manufacturers will be able to put it inside the car behind the mirror. Innoviz is one example https://techtime.news/2025/11/14/innoviz-27/
Is that really true? Extraordinary claims require extraordinary proof.
Ars cites this China Daily article[0], which gives no specifics and simply states:
>A LiDAR unit, for instance, used to cost 30,000 yuan (about $4,100), but now it costs only around 1,000 yuan (about $138) — a dramatic decrease, said Li.
How good are these $138 LiDARs? Who knows, because this article gives no information.
This article[1] from around the same time gives more specifics, listing under "1000 yuan LiDARs" the RoboSense MX, Hesai Technology ATX, Zvision Technologies ZVISION EZ5, and the VanJee Technology WLR-760.
The RoboSense MX is selling for $2,000-3,000, so it's not exactly $138. It was going to be added to XPENG cars, before they switched away from LiDAR. Yikes.
The ATX is $1400, the EZ5 isn't available, and the WLR-760 is $3500. So the press release claims of sub-$200 never really materialized.
Furthermore, all of these are low beam count LiDARs with a limited FOV. These are 120°x20°, whereas Waymo sensors cover 360°x95° (and it still needs 4 of them).
It seems my initial skepticism was well placed.
>if you had to choose one or the other for cost you’d be insane to choose vision over lidar
Yes lidar has limitations, but so does machine vision. That’s why you want both if you can have it. LIDAR is more reliable at judging distance than stereo vision. Stereo vision requires there to be sufficient texture (features) to work. It can be thrown off by fog or glare. A white semi trailer can be a degenerate case. It can be fooled by optical illusions.
Yes, humans don’t have built in lidar. But humans do use tools to augment their capabilities. The car itself is one example. Birds don’t have jet engines, props, or rotors… should we not use those?
Most practical navigation systems employ some form of sensor fusion. It’s more the norm than not. I’m sure even Tesla’s FSD does too for fusing vision, inertial sensors, wheel speed, multiple etc.
I believe traffic lights currently use three bulbs, red, yellow and green. Even without color a computer system can easily determine when each light is lit.
If there are single bulbs displaying red, green and yellow please give clear examples.
What do you mean by proportion? They are different data sources, and their usage is determined by system design.
eg A driving decision system needs to know object distances AND traffic light colours. It doesn't particularly need to know the source of either.
You could have a camera-only system that accurately determines colour and fuzzy-determines distance. Or you could have a LIDAR-only system that accurately determines distance and fuzzy-determines colour.
Or you use both, get accurate LIDAR-distance and accurate camera-colour and skip all the fuzzy-determination steps.
Or keep the fuzzy stuff and build a layer of measurement-agreement for redundancy.
So then the question becomes, what's your proportion when deciding whether to stop at a traffic light? Is it mostly light colour or mostly distance to other objects? Or 50/50?
Because it really feels like you're just JAQing off, either to come to some "gotcha" moment or a pseudo-intellectual exercise. In either case, the questions feel bad faith.
> At what proportion? Is it mostly lidar or mostly cameras? Or 50/50?
What proportion of your vision is rods or cones? Depends on the context. You can do without one. But it’s better with both.
> How about when you come 4 way stop. LIDAR is useless as it wouldn't recognize anyones turn signals
Bad example. 99% of a 4-way stop is remembering who moved last, who moves next by custom and who may jump the line. What someone is indicating is, depending on where you are, between mildly helpful and useless.
Something I've seen noises about is time of flight systems for traffic. I think the idea is you can put those systems on traffic lights, cars, bicycles, and pedestrians and then cars can know where those things are.
Show the cost differences and do the math then come back to us before you can suggest what decisions were ill timed. Otherwise it's just armchair engineering.
Camera's are not the issue, they are dirt cheap. Its the amount of progressing power to combine that output. You can put 360 degree camera's on your car like BYD does, and have Lidar. But you simply use the lidar for the heavy lifting, and use a more lighter model for basic image recognition like: lines on the road/speed plates/etc ...
The problem with Tesla is, that they need to combine the outputs of those camera's into a 3d view, what takes a LOT more processing power to judge distances. As in needing more heavy models > more GPU power, more memory needed etc. And still has issues like a low handing sun + white truck = lets ram into that because we do not see it.
And the more edge cases you try to filter out with cameras only setups, the more your GPU power needs increase! As a programmer, you can make something darn efficient but its those edge cases that can really hurt your programs efficiency. And its not uncommon to get 5 to 10x performance drops, ... Now imagine that with LLM image recognition models.
Tesla's camera only approach works great ... under ideal situations. The issue is those edge cases and not ideal situations. Lidar deals with a ton of edge cases and removes a lot of the progressing needed for ideal situations.
The issue isn't just the cost of the lidar units off the shelf. You have to install the sensors on the car. Modifications like that at the scale that Waymo does them (they still have less than 10K cars) are not automated and probably cost almost as much as the price of the car itself. BYD is getting around this by including them in a mass produced car, so their cost per unit is closer to the $130 off the shelf price. This is the winning combination IMO.
Waymo already has an automated integration line, and the new vehicles from Zeekr will come partially assembled from the factory as a semi-custom design so there's no modifications in the sense that you're talking about.
Tesla uses their own chips. Chips which you can’t skip by using lidar because you still need to make decisions based on vision. A sparse distance cloud is not enough
They have no fabs. They're using nvidia chips on server side last I checked, and what tsmc thei own design for in car? Those aren't cheap anyway. The markup on nvidia chips is high, but it's not _that_ high.
I honestly don't mind too much since it's a once a year thing (hacktober) and honestly companies should be trying to catch out employees who click any and all links.
reply