Three Months with Apple Vision Pro

Back in 2017, I wrote:

While everyone is talking about next-gen smartphones like the very impressive Galaxy Note 8 and the highly-anticipated iPhone X, I'm more interested in what's being built on top of smartphones that will bridge us to the next big thing — Augmented Reality.

At the time, Apple was on the verge of launching ARKit for iOS developers, which was Apple's first public building block towards the next big (hardware) thing that comes after the smartphone1.

Fast forward seven years to today, we finally have Apple Vision Pro (AVP).

The hype cycle has completely come and gone these past few months, but I've been using my AVP every day since launch and I've collected all my thoughts on the hardware, software, and user experience into this post.

Let's get into it.

Comfort

Dual Loop Band. (Source: Apple)

AVP comes with two straps: the Solo Knit Band and the Dual Loop Band.

The Solo Knit Band is the one that is in all of the marketing material and is used when you get a demo at the Apple Store. It's beautifully crafted, soft, stretchy, and is clearly meant to minimize messing up your hair. Unfortunately for me, the Solo Knit Band gave me a headache within 5 minutes of my demo...

Thankfully, Apple includes the Dual Loop Band in the box, which is dramatically more comfortable, allowing me to wear AVP for 3-hour stretches without a hitch.

The Passthrough Screen

One key feature for AVP is the ability to see the world around you while wearing the device. But to be clear, this is not a transparent piece of glass; this is a passthrough screen, which means it is using cameras to show you a real-time video feed of the world around you.

The general consensus among big tech reviewers like MKBHD and Nilay Patel of The Verge says that AVP's passthrough screen is the best in the industry. But because these are digital cameras with displays and not our actual eyes looking through transparent glass, we run into some limitations:

  1. The field-of-view is a tad smaller than expected, especially when compared to the marketing material.
  2. When you try to read small text on physical objects while wearing AVP, it is extremely difficult. For example, when you get a push notification on your iPhone, you have to hold it super close to your face and squint to read the text. This gets even harder when you're in a low-light environment.
  3. When walking around or turning your head, you will see a lot of motion blur on the real world environment around you. Ignore all those viral videos of AVP users walking around in public, randomly pinching at things in the air; real AVP users will not be doing that because it is not worth the literal headache from all the motion blur.

AVP is primarily meant for stationary use, whether that be sitting down, standing in place, or working within the same general area. The passthrough screen is simply there to allow awareness of your physical surroundings and give you a certain level of freedom to move around while wearing the device.

EyeSight

In addition to the passthrough screen that faces inward, AVP has a external screen that faces outward. While you are immersed in an app, the outer screen will display a purple aurora animating just like the northern lights.

When someone walks within your proximity, two things happen:

  • For you, the person will slowly fade into view.
  • For them, the aurora will slowly break apart to reveal your eyes, or rather a real-time digitally-rendered version of your eyes.

AVP is the only headset device to do this.

While critics chalk this up as pointless and superficial, Apple went great lengths to pull this off for one reason — eye contact is important for in-person human interaction. By allowing people to see your eyes, you maintain a certain level of approachability instead of appearing completely isolated.

I think it's a clever solution — like Transparency Mode for headphones — but only to a certain degree:

If someone next to you has a quick question, it feels perfectly natural to turn towards them and have that quick conversation without having to take off the device. But if you need to have a more serious conversation, it only makes sense to take it off out of courtesy, just like headphones.

One thing I will note about EyeSight is it currently does not work with dogs. When I'm fully immersed and my dog walks up to me, sadly, he does not fade into view nor is he able to see my eyes. (Yet, strangely, he looks at me as though he can.) I am hopeful this will come eventually; iPhone's Portrait Mode and the Photos app originally did not recognize dogs either until later OS updates.

visionOS

Home View on Apple Vision Pro
Home View in the Mount Hood environment on Apple Vision Pro (Source: Apple)

I can't even begin to tell you how unbelievably cool it is to use Vision apps. AVP's screen has 50x more pixels-per-inch than iPhone, making 3D objects look real and colors look vibrant. It is truly mind-blowing to see digital things just floating in space as if they are physically there. When you pull a window closer or push it farther from you, you'll even notice a shadow on the table or floor, moving in unison with the window.

With visionOS, Apple introduces Gaze & Pinch as the primary interaction. It is so simple, yet absolutely brilliant, and downright feels like magic. As long as the cameras can see your hands (e.g. there is enough lighting in the room and your hands are not under a blanket), you can click, drag, and resize things without having your hands leave your lap. (It's not like Minority Report where you have to make grand arm gestures, which would get very tiring very quickly.)

But that's not all — if you pull a window up close to your face, you can tap and swipe just like a multitouch screen!

One of my favorite things to do is open the Photos app, pull the photos as close as possible, and swipe through them as if I'm using an iPad that is floating in mid-air. Reliving memories from my photo library like this is truly breathtaking. It's so good, I'm seriously questioning if I should go back to shooting all my photos & videos in landscape again, just like the pre-Snapchat days.

Now for the downside. While Gaze & Pinch feels magical and swiping feels familiar...typing, is a big fat pain-in-the-ass. I can't even begin to tell you how much of a struggle it was to log into each of my apps when I first set up my AVP.

There are three ways of typing on the virtual keyboard:

  1. Good 'ol fashioned pecking on each key, one at a time.
  2. Gaze-and-Pinch on every single key, one-by-one.
  3. Use Siri dictation.

Within a couple weeks, I got fairly comfortable with the pecking method for typing, but only for very short things like domain names and search keywords in Safari. I've had this thing for almost three months yet to this day, I still don't know how to move the text cursor around when I need to fix a typo.

If you plan on doing a significant amount of typing — or do any serious work general — I highly, highly suggest connecting a bluetooth keyboard and trackpad. A bluetooth mouse can be used, with limitations, by enabling some hidden Accessibility settings.

An Infinite Canvas

With visionOS, windows can be moved closer or farther, left or right, above you or below you. You truly have a full 360-degree canvas to place windows wherever you want. But that's not all! When a 360-degree canvas isn't enough…you can place windows in other rooms too! Place your calendar app on the wall. Keep Slack open next to your desk. Put the Reminders app in front of your refrigerator or Safari in front of your toilet. AVP does a remarkable job of keeping windows in the same position, even when you take off the device and put it back on again.

Here is a great video by Himels to give you an idea:

I should note though that sometimes the position tracking can be confusing. There have been multiple times when I opened an app and I'm wondering, "Where the hell did it go??" like Ben Stiller in Something About Mary, only to realize that I left my Safari window open downstairs in front of the couch.

In those situations, to summon a specific window — no matter how far away — simply gaze-and-pinch on the window's handlebar, and the window will be summoned right to you like Thor's hammer. It truly feels like magic and I don't say that lightly.

To summon all windows, simply press-and-hold on the Digital Crown and all windows will appear right in front of you, perfectly centered.

Mac Virtual Display

By far my most favorite part about living in the Apple ecosystem is how all of its products work together. I'm a big believer in "the whole is greater than the sum of its parts," and this feature makes AVP an incredible addition to the Apple ecosystem.

With AVP, you can easily bring your Mac into your virtual space as a virtual 4K monitor. No wires. No stand. Just a glorious 2560-by-1440 pixel display — only visible to you — that you can float anywhere in space.

If 2560-by-1440 isn't enough, there is an option to increase it to a whopping 3840-by-2160 display, which I find incredible useful for certain projects that require a ton of multi-tasking. I love that my screen real estate can be scaled up or down depending on what I'm working on.

After three months, my virtual workspace has settled into something like this:

  • All work-related apps floating front-and-center in my Mac Virtual Display, such as Chrome, VSCode, Slack, and Toggl.
  • All my personal stuff floating on the sides as Vision apps, such as iMessage, Twitter/Mastodon, Discord, podcasts, and/or streaming apps.

I love this setup because whenever I need to take off my AVP, all my personal stuff disappears but all of my work stuff pops up on my MacBook Pro as normal; my productivity never skips a beat. And since this is all within the Apple ecosystem, typing, using the trackpad, and copy-pasting between Mac apps and Vision apps works seamlessly.

For those who use multiple screens for their Mac setup, there have been reports that Apple engineers have been testing this for quite a while2. In the meantime, third-party apps Splitscreen and Universal Desktop enable multiple Mac Virtual Displays.

Overall, Mac Virtual Display truly opens up opportunities for working remotely from anywhere, with a full multi-display workstation setup.

Environments

We've long-known that the beauty of virtual reality is that you can transport yourself anywhere. One moment, you're sitting in a cramped economy seat on a plane with very little legroom. Put on VR headset and you are transported to a wide open beach.

But sometimes you don't want to be fully immersed. Sometimes you just want to block out physical objects in front of you, like that reclined airplane seat that's 10 inches away from your face. Maybe instead of being 100% immersed, dialing it back to about 40% would give you the "space" you need in front of you while still allowing you to see what's going on beside you.

AVP allows for exactly that. With a quick turn of the Digital Crown, you can control exactly how much you want the virtual environment to wrap around you.

I love that I'm able to do work 40% immersed in the Haleakalā environment in the morning, Joshua Tree in the afternoon, and Mount Hood in the evenings. Having this granular control of the immersion lets me transform my boring wall into an office with a killer view while allowing me see my dog walking around or lounging right next to me:

One awesome benefit that I did not anticipate — I can now totally rethink my home office furniture and desk accessories. For example:

  • There's less need for a desk lamp since the virtual lighting in AVP is always perfect.
  • There's no need to angle my external monitor away from sunlight to reduce glare since there is no glare on a Mac Virtual Display.
  • Likewise, there's less need to continue saving up for a $1,599 Apple Studio Display with the optional $300 Nano-Texture (anti-glare) coating.
  • There's no need to orient my monitor a certain way to minimize people spying over my shoulder.
  • There's less need for a 16-inch display on my next MacBook Pro; I can downgrade to the 14-inch model and use Mac Virtual Display for work.

Even for non-work related things:

  • There's no need to align my oversized bean bag chair with my gaming TV; I can get rid of my gaming TV altogether.
  • There's no need for a TV in my bedroom.

What I originally thought would be a gimmick, Environments with immersion control turned out to be AVP's most underrated feature, for both work and play.

Entertainment

Speaking of non-work related uses, entertainment has been my biggest use-case by far with AVP, especially when combined with Environments. In the past month, I've watched:

Apple's official TV app has a special mode called Cinema Environments:

As someone who's wanted a $400+ Philips Hue Sync Box setup for the longest time, this is a pretty sweet alternative. And it's not hard to imagine Apple adding their own Philips Hue Sync lighting effects in a future visionOS update.

I love the flexibility of sizing and positioning video any way I want. My video will always be perfectly aligned with me, no matter which end of the couch I'm sitting on or whatever angle my head is in bed. Sometimes I set the immersion to 0%, make the video small, and put it right up to my face so I can watch a movie on a small screen while playing fetch with my dog. Other times, I set the video to be the size of my bedroom wall:

Watching a 4K travel vlog in my bedroom using Juno for YouTube

YouTube

Sadly, YouTube is the biggest downer — there is currently no official YouTube app for Vision Pro. The default option available is to point Safari to youtube.com and use the website.

I gotta say…using the website fucking sucks.

The video controls are close together, so it's way too easy to accidentally click the wrong thing. When I try to hit the Full Screen button on the bottom-right corner, I often accidentally click on the end of video. Or when I try to hit the Play/Pause toggle button, I accidentally move the tracker to the the very beginning of the video. It truly is the worst.

YouTube has officially said that a native Vision Pro app is "on the roadmap". In the meantime, the best workaround is using Bluetooth trackpad and/or the third-party app, Juno for YouTube, which you can see in the screenshot above.

I'm personally in the middle of researching travel destinations for my next solo adventure and it's exhilarating to watch 4K travel vlogs stretched out to the width of my bedroom wall.

Gaming

As for gaming, it's still very early days so there's not much to write about. Although, I do expect Apple to work with third-party developers to add more AR games to its Apple Arcade library in the coming years.

For console-level games, there are several third-party ways to play PlayStation and PC games on AVP. When I inevitably hand down my 55-inch TV to one of my family members, I can see myself using the NDI streaming method.

For retro console games, Apple recently lifted their policies to allow for retro game emulators like the popular Delta app, which has been a part of the jail-breaking community for 10 years. This is currently only available on iOS but it's not hard to imagine this coming to AVP in the near future.

VR Porn

For the guys in the back, I'm sad to report to say that VR Porn does NOT work out-of-the-box on AVP…But don't worry, Redditors are determined to find workarounds.


Overall, when it comes to entertainment, AVP makes for an incredibly immersive personal home theater. While the living room TV will always be an essential for the household, I can see a future where I personally will never buy secondary TV for myself ever again.

Looking ahead, I'm incredibly excited to see how Apple continues to make inroads with live sports and the entertainment industry. Recently, NBA Commissioner Adam Silver met with Tim Cook to bounce around ideas for the future of NBA viewing:

As Tim said it this is in many ways better than sitting courtside. […] It can take you anywhere on the floor. It can give you the perspective of a player, it can bring you places that you could never otherwise go and absorb it. […] We saw it for soccer, we saw it for baseball [in the Vision Pro demo], and I think as Tim said, this is of course early days, but this is transformative. […] This to me, will be how people over time experience sports through the media.

Personas

If there's one thing we learned from the pandemic, it's that video calls are a big deal and are here to stay. But because the device covers our faces, we still need a way to show up in video calls.

Meta launched with the fun, cartoony Meta Avatars. Apple, on the other hand, decided to go with realism:

It's definitely not perfect. But if you had to choose between AVP Personas or cartoony Meta Avatars for a Zoom call with coworkers, it's clear that this is the right direction.

Admittedly, I haven't had much time to play around with Personas. I suggest watching this 14-minute video of Brian Tong getting on a FaceTime call with iJustine and MKBHD. They have a fun time testing it out and discovering some really clever nuances that Apple included in the experience:

Apple still markets this as a beta feature that is still in the works. I'm happy to report that with every visionOS update, Apple is constantly showing improvements, like this new Spatial Personas that makes immersion more social:

Price

As a bachelor who works from home as a web developer, I can sum up my AVP usage like this:

  • During work hours, AVP is an awesome Mac accessory that lets me work with a big-ass screen, in an immersive environment, with all my personal apps floating on the sides.
  • During off-hours, AVP is the ultimate personal home theater experience.

But does that make it worth it? Well, let’s first take a look at the price:

Model Difference Price
Apple Vision Pro (256GB) - $3,499 + tax
Apple Vision Pro (512GB) +$200 $3,699 + tax
Apple Vision Pro (1TB) +$400 $3,899 + tax

Since this is a cutting-edge device that is very complicated to manufacture, repairs are very, very expensive:

Incident Repair Fee
Cracked Cover Glass $799
Other Component $2,399

For paranoid customers like me, there's AppleCare+ extended warranty:

Item Cost
AppleCare+ $24.99/mo or $499 for 2 years
Damage to Included Accessories $29 with AppleCare+
Other Accidental Damage $299 with AppleCare+

One interesting thing to note: the one-time $499 AppleCare+ option provides coverage for AVP for only 2 years, like iPhone and iPad, while AppleCare+ for Macs provide coverage for 3 years.

Overall, I paid about $4,500 for the 512GB model with AppleCare+, which is an insane amount of money for a first-generation product that has so many questions. But as a lifelong techie, a hardcore Apple fan, and a big believer in AR, there was one question in particular that I had to explore for myself — how many of my devices can this replace?

After nearly three full months of using AVP every day, here's what I found that could be replaced (for me):

  • $1,899 Apple Studio Display with Nano-Texture? YES. Screen glare is a non-issue with Mac Virtual Display.
  • $1,099 iPad Pro (12-inch)? ALMOST. Since I mainly use my iPad for watching YouTube and streaming apps, this makes sense for me. I'm just waiting for YouTube and Twitch to support visionOS.
  • $800 55-inch TV and $149 Apple TV for personal use? YES. I'll always have a TV in my living room, but I'm convinced I'll never need a secondary TV just for myself ever again.
  • $400 Philips Hue Sync Box? Kind of! Not exactly the same effect but AVP's Cinema Environments feature is a pretty sweet alternative.
  • $500 savings by downgrading my MacBook Pro from 16-inch to 14-inch? I think so! A 14-inch MacBook screen will be just fine for casual use and light work. Then when I need more screen real estate for more serious work, I'll just bust out AVP.

Crunching those numbers with a little bit of Boy Math, it looks like value I get from AVP indeed justifies the cost! I personally am confident enough in the Vision platform to phase out my iPad, external monitor, and home office TV and budget for a new AVP every 3 years.

But that's just me.

For 99% of my friends, I would never recommend spending this much money on a cutting-edge v1.0 product. AVP today is not meant for the common consumer. It is here today in its bulky, overpriced form for three reasons:

  • Get developers to research and develop cool shit.
  • Get enterprises to start thinking of innovative business applications.
  • Get feedback from real people so the AVP engineering team can quickly iterate on the platform.

And then for the tiny percentage of consumers who are early adopters like me...Apple will gladly take our money lol.


Looking Ahead

As groundbreaking as this product is, it is very much a v1.0 product with a long runway and ton of opportunities to improve. This is very reminiscent of the first few years of iPhone, which didn't have third-party apps, push notifications, or even copy-and-paste.

In the short-term, I expect Apple to improve comfort, increase the field-of-view, improve passthrough mode, offer more affordable models, and continue to foster the Vision app ecosystem.

In the long-term, it all comes down to one thing — launch AR glasses. That has always been the goal.

Just look at what Tim Cook has said over the years:

2016: Few people are going to view that it’s acceptable to be enclosed in something.

2017: Unlike Virtual Reality which closes the world out, AR allows individuals to be present in the world.

2017: I also like the fact that [AR] doesn’t isolate […] I’ve never been a fan of VR like that because I think it does the opposite.

2020: I think [AR is] something that doesn’t isolate people. We can use it to enhance our discussion, not substitute it for human connection, which I’ve always deeply worried about in some of the other technologies.

Today, AVP is a bulky mixed reality headset with passthrough technology. But Tim Cook has clearly signaled for years that the real goal is AR glasses. Why? Because glasses are already normalized and socially acceptable. Glasses can be worn indoors or outdoors, daytime or nighttime, stationary or while walking. They won’t mess up your hair or your makeup and there's no need for a passthrough screen and dealing with its limitations.

AVP is not meant to usher in a dystopia of people walking around with big headsets, nor will it be about living our digital lives in the "metaverse" like Mark Zuckerberg originally envisioned. Rather, it's the evolution of the computer screen that we carry with us everywhere. Instead of tapping and swiping on 6-inch smartphone screens, we will use gaze-and-pinch, hand gestures, and AI-powered voice commands to control digital experiences that float around us. You will have the freedom to make AR apps as big or as small as you need them to be; you will no longer be restricted by the size of your smartphone screen.

While I can't recommend the first-gen AVP to anyone, to completely dismiss the future of this product would be like completely dismissing the future of cellular telephones back in 1983 when the first mobile phone, the Motorola DynaTAC, was a 4.4 lbs plastic brick that sold for $3,995. (Adjusted for inflation, that would be a whopping $12,010 in today's dollars.)

It took 23 years to go from the first cell phone to the first iPhone. And we are now 17 years into the modern smartphone era. If you're a non-developer who is crazy enough to buy an AVP like me, what you're really paying for is the a glimpse into the future; a future that won't be realized for another decade or two.3 AVP is big, expensive, and has a ton of limitations…but that's exactly how all hardware innovations start out.

Apple Vision Pro 1.0 is just the beginning.


  1. The smartphone will not be going away anytime soon, especially not in second-world countries. But in thriving first-world countries, it's very possible that AR glasses will eat away at the smartphone market. 

  2. Industry analyst Ben Thompson writes, "I have heard through the grapevine that Vision Pro users at Apple headquarters can project two Mac screens […]" 

  3. Smaller companies like XREAL have their own AR glasses today, but it's hard to see such a small company getting any serious traction with developers when Apple and Meta have a lot more credibility.