Comment by isk517
2 days ago
Yes, the company I work for has started using Hololens 2. We have a program that can overlay the 3D models from our CAD program onto the physical steel assemblies for QC. When it works, it works well and enables our quality checkers to perform checks faster and more accurately than using tape measures while going back and forth looking at a 2D drawing printed on 11 x 17 paper.
The biggest hurdles is that none of the large companies think there is enough profit to be made from AR. The Hololens 2 is the only headset on the market both capable of running the program required while also being safe to use in a active shop enviroment (VR with passthrough is not suitable). Unfortunately the Hololens 2 is almost 6 years old as is being stretched to the absolute limits of its hardware capabilities. The technology is good but feels like it is only 90% of the way to where it needs to be. Even a simple revision with double the RAM and faster more power efficient processor would alleviate many of the issues we've experienced.
Ultimately from what I've seen, AR is about making the human user better at their job and there are tons of industries where it could have many applications, but tech companies don't actually want to make things that could be directly useful to people that work with their hands, so instead we will just continue to toss more money at AI hoping to make ourselves obsolete.
The biggest issue comes from area mapping. In order to keep the hologram steady and anchored you need to perform a mapping process so that the helmet recognizes both the physical steel assembly and a bit of the surrounding area to keep it steady when moving around. The 8GB of RAM puts a limit on the amount of mapping data that can be stored putting a limit of the size of assembly you can work with, and since the mapping process relies on using the helmets own software that has not been work on in years it is extremely sensitive to any sort of background movement, which means it works best in as controlled of a environment as possible.
Right now we are just using it for special projects that are complex and have little margin for error. We'd like to be able to use it for everything but that isn't feasible with where the tech is currently stuck at.
I wish I could edit CAD models (in Solidworks or similar) while in a VR environment.
Like, real work modifying sketches and dragging points around in 3D.
When VR first came out (well, around the time HTC Vive was first launched) I searched madly for something to do this but all the apps felt like toys.
Have you used something you would recommend toward this end, on the design portion of things?
Thank you very much for sharing your experience.
Quick question about your use case - is the 3D overlay really that important, or would you get most of the value simply seeing the blueprints in your heads-up display, maybe doing a quick finger swipe or voice command to switch between pages/images?
Yes, the 3D overlay is the entire point. A heads up display is just looking at the blueprint on a piece of paper with an additional layer of complexity, it wouldn't remove the need to manually measure, nor would it provide any assistance in spotting missing attached pieces (or some extra pieces). Once the model is overlayed QC goes from having to measure the placement of every pieces and the location of every hole to just walking around the finished assembly and ensuring that every conforms to the civil engineer approved model. A half hour process can be done faster and more precisely in 5 minutes, you notices very quickly when there is solid steel where the hologram has a hole, or thin air where the hologram shows that a plate was suppose to be welded on.
Have you looked into OpenSplat type of post-processing? You take a bunch of pictures and then let hardware create a 3d model. It's really competent and could easily create a rectified model for measurements. To get actual values, you'd need some control points, but beyond that, a pipeline that continiously creates models could be feasible.
Then your QC guys are mostly behind computers and rotated to the floor when things are identified.
Ultimately, your VR isn't doing anything more technically accurate than this.
Very interesting, and I agree with your assessment of the difficulties using the aging HoloLens.
I am curious, what size of clients are you working with and how many contracts has it realistically turned into?
I also believe proper AR hardware/software can revolutionize the QA and inspections industry.
What I am noticing is a chicken/egg problem where companies want proof it works, while also reluctant to put their money where their mouth is and invest in the R&D. Which then leads to Microsoft and similar refusing to fully invest in new AR tech.
As such, it all stays mostly in experimental and drawing board land, never quite fully reaching the market.
Thoughts?
We work with all of the large general contractors in the steel construction industry. Right now, it's turned into one contract, but we are the second company the client has employed since the first company they hired failed to produce a single assembly that met their requirements. The client was the one that originally was using this tech since they wanted a way to do their own QC after the first experience, and we decided it was worth while pursuing ourselves since successfully pulling this project off while be a HUGE boost to our reputation. The construction industry is all about your portfolio of past projects.
QA is the big sales point of the software we are using, but there are many other potential applications for the same product. It should be possible to overlay the model on the main assembly prefab then use that to quickly mark where holes should be drilled and additional pieces attached. The other potential application that is being explored is using the holographic overlays to construct things out of the usual order, instead of building part 1 then starting part 2 since it needs to be built to conform to the first part you can instead build around the hologram so that your not relying on the previously built parts to ensure your angles are correct.
I agree about the chicken/egg problem. Its an emerging technology where the payoff might be a decade away, customers need software that will actually benefit them, developers need reliable hardware capable of running software that has practical uses, and hardware companies want to know there is a customer base. The issue is AR falls under the category of product that the customer does not know they actually want, so the only way it is going to be developed is if one of the hardware manufactures takes a leap of faith and makes the long term investment. Sadly, I feel like AR is a million dollar idea with practical uses that has to contend with a business climate where you can make billions making some doodad that collects private data then displays ads to the masses.
An additional layer of insight to the chicken/egg problem: the developer of the software we are using was founded by someone in the construction industry, not software. I think one of the issue with the adoption of AR is that there is currently a disconnect between the people who have a problem and the people who could produce a solution. Compared to 'a solution in search of a problem', AR seems to be 'a solution that is failing to introduce itself to the problems it can solve'
There are thousands of ways companies can invest to make their employees more efficient. My guess is companies are choosing to invest in lower hanging fatter fruits.
Companies have put billions into R&D, but still haven't delivered a product that surpasses the hurdle rate.
The problem is there's no growth market, there's no easy to canibalize existing market, and the tools are not "we'll buy a new one every year".
The world in which capitalism has taken hold is not one that produces incrementally better products for niche markets.
You end up mostly with passionate people improving niche markets, and if it involves hardware, we're just at the beginning of small time custom hardware makers to make a dent into this type of market need.