John Carmack's twitter feed has focussed on head-mounted displays quite a bit as well. Seems like a popular next step if they sort out the problems. Latency's a big one, especially if you're doing head-tracking or augmented reality.
I wonder if we'll see an "i-Eye" from Apple in a few years' time.
Numbers: E2E latency of key press->VGA signal or keyboard LED toggle: both ~23.3ms. Measured with 100MHz analog oscilloscope+300fps cam.
For that experiment he used a little custom circuit for input (button) and output (LED) that was wired to the serial port of a computer running Windows. In addition to the output LED, the button was also directly wired to another LED so as to get precise measurements of the delay. If you were to have full end-to-end control of the system and you chose to make it a top property, you could obviously get much lower latency. But to even get those numbers I believe he had to use a special driver that granted his test program privileged access so he could do direct port IO with INB and OUTB rather than delegate the work to the kernel.
I think the big latency issue here isn't so much with analog input delay of the user as it is the speed of video processing algorithms.
Currently researched augmented reality solutions look great, but they're plagued by delay that make usability an issue. As processing power becomes cheaper and faster, it makes sense that we'll eventually be able to achieve almost-realtime augmented reality. At such a time in the near future, head mounted displays will definitely become popular.
My point was more that the baseline latency on a modern computer and operating system, without really doing anything on the application side, is already very significant--it took simplifying assumptions and a good deal of hacking to get to that figure of ~23.3 ms. So, yes, latency is a big issue with practical augmented reality, on several levels of the hardware/software stack. Computer vision algorithms that rely on temporal coherence can easily introduce many frames of latency, so even if the per-frame processing time is manageable, the latency may not be.
I agree. The problem of latency definitely exists on more than the software level. From my anecdotal experience however, the majority of current latency in augmented reality does seem to lie in the vision processing.
Interestingly, previous research[1] indicates the maximal latency for a mobile augmented reality device to be fully immersive is 5-10 ms. While I don't think this target is reachable with modern technology, as you've obviously made clear, I feel we'll eventually get to a point where it's Good Enough for practical use.
I can't help but feel excited about what kinds of applications such technology will yield in the future.
[1]Pasman, W., Schaaf, A. van der, Lagendijk, R. L., & Jansen, F. W. (1999). Low latency rendering for mobile augmented reality. Proceedings of the ASCI'99 (Heijen, the Netherlands, June 15-17), 372-376.
Actually if you do want to "feel" the latency at all in most situations, you'd rather be at 1ms than 10 ms. 10ms still carries a huge lag when you are moving you head around in several directions. 1 ms brings it way closer to being real. We need improvements of a number of magnitudes to reach high resolution, processed graphics and display at 1ms rate to work. We may have some working prototypes within one decade, with commercial applications 10-15 years from now. It could go faster than that, but not very likely.
I wonder if "Good Enough" really is. One big problem with latency is that it may well lead to motion sickness, since your perception of motion from the visual system and your inner ear is out of sync.
Disclaimer: It's been a _long_ time since I worked with heads-up/AR displays. Let's make it 15 years. (I feel ancient now ;)
The main delay issues are in head movement, hand tracking etc. is much more forgivable. Thankfully head tracking can be augmented with low-latency accelerometer data.
Anyone have investment ideas in this area (virtual reality and/or augmented reality headsets/glasses and software) among publicly traded stocks? Google is supposedly coming out with a display this year. Sony has the best consumer microdisplay through investment in camcorder viewfinder tech... Whoever hits on this first could be the next Apple (or Apple may just do it like you say), ultimately software seems like it is going to be the key determinant. Microsoft has Kinect.. Sony, Google, Apple, Microsoft... anyone have some more underdog guesses?
Newell's reactive/empowering mgmt style: "We’re a very flat organization, so we expect everybody to manage themselves...[employees] need to know when to broadcast to me when something’s important...We’re optimized for people who are very experienced and have been working for a long time and don’t really need someone looking over their shoulder or second-guessing their decisions."
I wish Gabe would get behind the Open Pandora project, fund them for mass production and help get them out there. It seems like the perfect machine for their console needs ..
I wonder if we'll see an "i-Eye" from Apple in a few years' time.