Introducing HoloLens Remote Assistance

November 1, 2018

Remote assistance is one of the key use cases for Mixed Reality.

With HoloMeeting's Life Feed function, Remote assistance is a reality with a wide-ranging potential impact and a number of different use case scenarios. In HoloMeeting, remote assistance can be seen through the Live Feed feature.

Imagine being an engineer on an oil platform needing to conduct repairs. You encounter a challenge during the repair process and require assistance. Currently, you would call a remote expert for support.

However, given the fact that the expert can't see what you are working on, it is hard to provide efficient and detailed support. The most common solution to issues that cannot be solved via remote support would be for an expert engineer to be transported to the location. This clearly presents its own problems, not least of all the loss of output and earnings during the downtime period.

From oil rigs to factories, manufacturing facilities to research laboratories these periods of downtime and lost output hours cost businesses millions of pounds every year.

Thankfully, the HoloLens and Mixed Reality technology has allowed for an elegant and effective solution to be developed. We call our remote maintenance solution "HoloLens Remote Assistance" which is packaged within our HoloMeeting application in the form of the Live Feed Feature.

Introducing HoloLens Remote Assistance in HoloMeeting

Our determination when we developed HoloMeeting was for the application the be a facilitator of remote collaboration and communication with Mixed Reality. to be an application for communication and collaboration, we realise that we could think in broader terms than this. This is because the video Live Feed aspect of HoloMeeting which can be used for training, design, collaoration and communication can be used as a tool to bring a remote expert into the space of an in-field engineer.

The HoloLens allows the engineer to share his or her point of view through the camera being mounted directly above the line of sight. This enables the expert to provide valuable and efficient support.

At the moment, the closest solution available to this is for HoloLens. Having discussed this extensively with clients, we found that this is not a suitable solution for actual business use due to several factors:

  • No control over bandwidth consumption (limiting the data being used)
  • Calls can be received via desktop Skype app but requires installation of a plugin to use interaction features (many business do not allow the installation of plugins)
  • No automatic call recording
  • Can't take notes during the call

These are only but a few limitations of the currently available Skype solution.

A real remote maintenance solution for commercial purposes

Kazendi's HoloLens Remote Assistance solution via HoloMeeting provides the following features:

  • Fine grained control of the bandwidth used: 1 FPS (frame per second), 5 FPS, 15 FPS, 320x450, 620x350, 1080x720 (HD resolution) therefore the solution can consume anything from 50 kilobyte per minute to 15 megabyte, depending on your needs
  • Holographic annotations (see screenshot below)
  • Send files (FBX 3D models, PDF, images)
  • Apps available for Windows and iOS (Android & Web version in progress)
  • Record sessions
  • Expert collaboration by inviting other experts to join the call

This is a screenshot taken through HoloLens, showcasing the settings options. When not in a call, these can be accessed handsfree using voice commands. We are considering implementing gaze based control to allow for handsfree control during a call.

Return on Investment

Clients want to be able to use HoloLens during maintenance operations. Therefore, an expert could remotely assist the repair of a machinery within a factory setting. Considering that often when an assembly line of a car manufacturer has to be stopped due to a single fault machine, this can often run into the £100,000+ per day or even per hour cost.

A HoloLens remote assistance solution that reduces time-to-repair, can potentially have a significant impact on machine reliability and manufacturing floor profitability.

Another use case is in remote locations - an oil rig or a mine for example - where the necessary expertise can't easily be provided (e.g. due to travel) - our HoloLens remote repair solutions allows your experts to dial in and support the user like they are there.

Additionally, through the ability to record support calls, we can use the footage for more efficient training of future remote experts. This allows for better documentation as part of complex Service Level Agreements as well as overall process improvements.

Outlook

For the future, we are planning to release a feature that allows the HoloLens to 3D scan the machine/environment in question and for the expert to view it on their HoloLens in life size.

Using the built-in scanners of HoloLens, we will create a 3D model of the object in question and using the RGB camera and create the colour information (texture). The remote expert using a HoloLens can then see  the object in question in front of them in real time and in a life-size format. This would allow the expert to be "teleported" to the support scene and virtually stand next to the engineer.

Furthermore, we are planning to expand the HoloLens Remote Assistance availability to other platforms such as iOS, Android & Web as part of our larger HoloMeeting rollout.

If you are interested in exploring HoloMeeting and remote assistance for your organisation do not hesitate to get in touch.

Back
© Kazendi Ltd, company registration number 09234614  The information contained in this website is for general information purposes only. It is provided by Kazendi Ltd and while we endeavour to keep the information up to date and correct, we make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the website or the information, products, services, or related graphics contained on the website for any purpose. Any reliance you place on such information is therefore strictly at your own risk. In no event will we be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this website. Through this website you are able to link to other websites which are not under the control of Kazendi Ltd. We have no control over the nature, content and availability of those sites. The inclusion of any links does not necessarily imply a recommendation or endorse the views expressed within them.Every effort is made to keep the website up and running smoothly. However, Kazendi Ltd takes no responsibility for, and will not be liable for, the website being temporarily unavailable. Kazendi Ltd is registered in England and has the company number 09234614.