Augmented Reality Becomes a Reality for Trading

By Ivy Schmerken, Editorial Director

As consumer technologies rapidly evolve, some are speculating that augmented reality could be the next big thing to transform the trader’s work space.

Financial services firms are already looking at bringing augmented reality into the trading room to help traders to interact with complex data sets and to collaborate with clients remotely.

While virtual reality immerses the person in a virtual world, augmented reality allows digital concepts to interact with a person’s real physical world.

Ever since the game Pokémon Go became a global sensation in July 2016 on the smartphone, there has been more interest in the commercial potential of AR, not only in gaming, but also in ecommerce and in mobile commerce.

What Lies Ahead?

Apple’s launch of the iPhone 8, iPhone 8 Plus and the 10^th anniversary iPhone X on Sept. 12 with augmented reality and facial recognition software for unlocking the phone, is going to spur the development of a huge variety of AR apps in the consumer space, said Daniel Gleeson, senior analyst covering consumer technology for Ovum in London.  [see sidebar]

With the emergence of the Microsoft HoloLens AR headset in 2016, some banks and third party developers have already begun to explore this new paradigm.

In March of 2016, Citi was the first bank to release a proof of concept for an augmented reality trading application for stock trading, as reported by Quartz. Using the Microsoft HoloLens headset, Citi worked with virtual reality design firm and Microsoft affiliate 8Ninths.

In a YouTube demo, Citi trader “Jason” checks his news on the traditional 2D monitors flanking his workstations. He then puts on his HoloLens and activates his 3D trading tools workstation. Using voice commands like “select” and “cancel,” the trader pulls up volume trends in the energy sector and simultaneously collaborates with a hedge fund client “Dan” who is remote. In what seems more like science fiction than fact, the client calls up an immersive chart, and the Citi trader is standing next him represented as a hologram.

“The trader is no longer confined to the trading desk and could project graphs and maps the size of the room in the AR space, enabling the more intuitive input and analysis of information,” wrote Citi GPS in the October 2016 report Virtual and Augmented Reality: Are you sure it isn’t real?”

In the same report Citi predicts that “VR/AR will be used in a wide range of applications and in a number of different industries going forward.”  Citi estimates the total market, including infrastructure like VR/AR headsets, content and services for the devices, and e-commerce that leverages AR will grow to $80 billion by 2020 and $569 billion by 2025.

“In 5-10 years VR/AR will probably be part of our daily life in areas like shopping, travel, and leisure and entertainment,” citing films, music, sports, amusement parks, zoos, and other activities,” wrote Citi GPS.

Meanwhile, banks are moving forward with AR applications to keep up with the millenials who are a growing part of the workforce and are more accepting of new technologies.

Citi is said to be exploring how virtual and augmented reality might be applied to other areas of finance like wealth management. And Wells Fargo has been working with Facebook’s Oculus Rift to improve the customer experience with bank tellers.

In terms of other verticals or enterprises, AR is getting a lot of attention in the high-end retail sector, said Gleeson.  “AR with a head-mounted display has huge potential benefits for all sorts of enterprise systems,” said Gleeson. Someone could walk around a house or showroom when they are buying a high-end sports car and view the colors of a car when they are not near the showroom. Someone could also fix a pipe while instructions are overlaid by the HoloLens.

But to gain momentum on trading desks, where there is a demand for high accuracy and high reliability, AR will have to offer tangible benefits.

Augmenting Desktop Real Estate

For decades traders have set up their screens to monitor real time bids and offers, market news, tweets and other social media streams. Today, they are incorporating email, chat and messaging applications. They are also working orders in algorithms, and monitoring executions routed to exchanges and dark pools.

Many have complained about the scarcity of real estate on the trader’s desktop, which can be an excuse for not adding another app or trading venue.

“While a traditional trader workspace would have six to eight monitors, AR technology could allow the trader to have a whole room as their monitor,” said Andy Mahoney, head of sales at FlexTrade in London. “We can basically create our own trading world that isn’t actually constricted to a number of monitors.”

In April 2017, FlexTrade demonstrated an augmented reality trading application with Microsoft HoloLens for FX. Using existing technology and application programming interfaces (APIs) within the EMS, the firm’s FlexAR application offers an interactive order blotter, trade ticket and charting in the virtual space.

Initially, the firm had coded with virtual reality but it was too immersive, said Mahoney. “It shut off all external events and it was difficult to conceive of being used in real life. Then we used the same code to port to augmented reality and it worked quite well,” he added.

“You can create charts, on a wall or in a free space, or you could have a different room where you place things,” said Mahoney,

With the HoloLens, the trader is using hand gestures or key words to interact with virtual objects, which could be placed on a finger. Or, the trader can point to a cursor, which depends on head movements, or point to where they want to click, or say the word “select” or “cancel,” said Mahoney.

One goal is to expand the blotter into three-dimensions. Normally a trader has the ability to graph stock prices on an x-and y-axis, said Mahoney. “If you graph it in the free space, you get an extension to the third dimension which is the z axis depicting depth,” he noted. As an example, someone could plot percentage of volume done on one axis, profit & loss displayed on a second axis and percentage of average daily volume (ADV) on a third axis.  “That way it will give you a completely different way to visualize and interact with that data,” he said.

Is Augmented Reality the Next Stage of Visualization?

Apple Moves into AR

Apple’s big push into the augmented reality market with its new iPhones could open up new opportunities for app developers, but it’s unclear how this AR development will impact the enterprise market.

Just as Apple revolutionized the way that consumers interface with software through the app store, Daniel Gleeson, senior analyst for consumer technology at Ovum in London, suggests that Apple could do the same for virtual reality and augmented reality apps.

At its recent event, Apple’s marketing executive said that the new iPhone’s camera was “calibrated for AR” to help people display digital images on a physical world. The downside is that Apple’s AR solution runs on a smartphone, which is a huge limitation for rolling out business applications for the enterprise, said Gleeson. “You have to hold a handset in front of you to avail of the AR system,” said the analyst.

Traders need their mobility, so holding a smartphone in front of their faces while they glance up to decipher complex charts and data patterns, isn’t ideal.

However, the upside is that developers will have an open AR system that all kinds of third-party developers can work with. This could spur “a huge variety of different applications, some of which can be used for trading and get more developers thinking AR,” said Gleeson.

When devices like the Microsoft HoloLens and other head mounted displays become more affordable and more commonplace, there won’t be a huge delay in software being developed for business.

Apple’s new iPhones incorporate image recognition and facial recognition capabilities for unlocking the phone, which Gleeson said, is very important for AR to work in real time for any sort of application.

Not only is this relevant for facial recognition, but this is key to recognizing text or an image on the screen, or to recognize what a person is figuring out contextually, said Gleeson.

“The image recognition system is probably one of the most significant pieces of technology that was announced on the iPhone,” said Gleeson, who equates this with artificial intelligence. Image recognition is an extremely complex task and this is what the iPhone X will be trained to do, he said. From an algorithmic point of view, the iPhone could also be trained to learn based on if/then rules to retrieve information based on a set of behaviors.

Apple has been rumored to be entering the AR space for months. In July Business Insider reported that Apple “filed a patent for a method that overlays computer-generated virtual objects over a real world environment.”

And in June, Apple released a developer platform called ARKit, which lets developers create iPhone apps and tools that use augmented reality, noted Business Insider.

Despite all of these moves, Gleeson says the eventual future is still a few years away, but that Apple’s involvement can influence how much engagement there is from developers and consumers in AR.

According to Brad Bailey, research director at Celent , there has been a history of visualization tools that brought the trader’s desktop to this point. “We went from people looking at numbers, to charts, and then all of this technical analysis evolved to give us visual insight. People really couldn’t just look at a bunch of numbers. Heat maps are just another point on that evolution. Visualization tools have combined with business intelligence and analytics to give powerful tools. The next transition could be 3D visualization with mixed reality, he suggested.

“For the last 15 years, we’ve been talking about trader real estate. If we could take the data, put all the [currency] pairs and tie them together in a way to visualize it, that would make sense,” said Bailey.

But the question is will traders feel comfortable wearing these devices?  For instance, “Google Glass didn’t take off the way people thought,” said Bailey. It was the first rudimentary step in this direction, noted the analyst. “It was disruptive.” That said, the Google glass concept in specialized fields is growing, said Bailey.

However, the HoloLens has some limitations, one of which is that it’s extremely expensive even for most enterprises, said Ovum’s Gleeson.  The cost is $2,000 or $3,000 per device. While HoloLens is not bulky, a trader would definitely know they are wearing it, said Mahoney.  The resolution is also not high enough for traders.

However, Gleeson believes the headset is the natural place for AR in trading.  “You don’t have to hold something in front of your face.”

AI Adds Value to AR

The critical ingredient propelling AR is going to be artificial intelligence, asserts Gleeson. “Having AI with its image recognition and facial recognition capabilities is very important for AR to work in real time for any sort of application,” said Gleeson.

Microsoft is working on an AI chip to include in HoloLens, said the analyst. It’s called an AI core processor for the HoloLens to handle image recognition.  That sort of technology is going to be common across all of the AR systems, said Gleeson.

With a head mounted display, AI could add in contextual information. If a trader is looking at a certain company or trade, the AR system might immediately recognize what they are doing and bring up all the contextual information without being asked for it. Or, it might highlight the most important elements, such as price or volume changes in the last 24 hours in real time to the trade.

Many of the top tech companies, including Microsoft, Google, and Facebook, are investing a lot of money in AR technology, so it is bound to evolve over time. Apple’s recent iPhone X release contains augmented reality technology, which is a sign that Apple plans to push into the AR market for consumer apps. (In April, Apple said it was going to start working on a set of smart glasses.)

But there can be resistance to wearing glasses with an always-on front-facing camera, said Gleeson, noting that was the case with Google Glass.

While there are still challenges for mobile augmented reality, Bailey believes the trader’s desktop will be using more holistic communications, such as chat, sharing data and the phone. The ability to integrate it all could be appealing.  “You can imagine sitting in a three-dimensional trading desk with your colleagues all over the world, and they holographically appear,” said Celent’s Bailey.

But the eventual future could still be many years away, according to Gleeson.  “It is really at a developer education stage of AR in trying to get developers onboard,” he said referring to consumer apps.  When the headmounted display units become more popular “then you will have a large developer community to roll with it,” said Gleeson.

But Celent’s Bailey said augmented reality is likely to materialize in other areas of society before it shows up on trader desktops.

As far as where adoption will occur, Baily sees potential in AR when it comes to visualizing massive amounts of data. “There is a limitation to how much information a brain can take in.  If there is a way to view price/time data as a blob in three dimensions,” and gain an edge, then Bailey thinks traders will be more willing to wear AR devices.