Multiconnection support for Tundra

We implemented a multi-connection framework for Tundra 2. Most 3D viewers support a single client server connection with a single presence in the virtual world. This is often done so that the virtual world owner can control the users experience. In some of our use cases we saw a need to maintain multiple viewers operating concurrently e.g.  the need to maintain separate private/public spaces, the ability to survive a wireless disconnection. Furthermore, we wanted to have a simple user interface mechanism to to easily move data between different worlds, e.g. to share data by drag and drop. As a result, rather than opening up separate viewers, we made one viewer and allowed it to open up new ‘tabs’. This should provide us with some benefits.

1. Data from one world can be dragged and dropped to another open tab as a simple user action.
2. Redundant information from multiple viewer windows can be removed saving memory
3. Users can have a presence in multiple different virtual worlds
4. Switching between worlds should be much faster.

Here we show snapshots of our viewer running with multiple client server connections. Each client server link can be accessed by clicking a tab at the top of the scene.

Figure 1 shows the current User Interface. Note the 5 tags, of which 3 are concurrent client server connections.

Figure 2. A connection to our test server (tundraclient0) with 4 users.

Figure 3. A connection to the local tundra server (tundraClient1) on the client device. This option is used for 2 reasons, a) provide the user with a private work space and b) maintain a working space if the wireless link is disconnected.

Figure 4. A third connection (tundraClient2) with an embedded QML application.

To implement the multi-connection strategy we addressed 2 problems, the communication architecture and the memory handling.

We needed to alter the architecture of RealXtend to support multi-connection. There are various challenges in the protocol layer, synchronization layer, storage and handling multiple 3D scenes.

When the client makes a new connection there has to be unique synchronization manager for each 3D scene created. These so called syncmanagers keep track of all the objects in the scene and update the positions and animations. 3D scenes have to be stored somewhere and brought up at will, so connecting the scenes to web browser style tabs would initiate the switch to the 3D scene that will be rendered. Currently rendering is done using a single Ogre scene so we need a container for multiple Ogre instances which all render one of the multiple 3D scenes.

In the protocol layer, Tundra only stores and processes one connection at given time. An alteration of the connection procedure needs to be made. The client only needs the update packages from the currently rendered 3D scene, but the protocol layer needs to keep alive all the connections that the client has made so far.  Connection properties has to be saved for every connection and the update procedure has to be extended to cover all the created connections.

Memory management is handled by dividing each virtual space into regions of interest, in our case there are 2 ROI. One region includes components in the Field Of View(FOV) and the second contains the rest (non-FOV). When we switch from one virtual space to another, the non-FOV assets are removed from memory while the FOV assets are kept. Hence, when we switch back to the previous virtual space the FOV objects are readily available and displayed while the non-FOV object are reloaded.

User Experiences of Hybrid 2D/3D UIs on Tablets and Design Guidelines

Abstract: We found that hybrid 2D/3D user interfaces for touch screen tablet devices result in a poor user experience. We recommend minimizing the amount of 2D overlaid icons or even avoiding 2D/3D hybrid solutions. To support  smooth user interaction, we recommend to focus more on using direct on screen touch gestures as part of the  design.

Introduction

Hybrid 2D/3D user interfaces incorporate 2D UI elements in a 3D environment. The 2D UI elements can be embedded in the environment (e.g. information panels). However, the most common approach is to overlay the 2D GUI components on the 3D Space. Sometimes called a Head Up Display, these components move with the camera and are not dependent upon ones position within the 3D space. Hybrid 2D/3D UIs are very common in most MMO’s and virtual environments.

What particularly interested us is the user experience with hybrid 2D/3D UIs on tablet devices and its affect on 3D UI design in general. High in our priority was the intuitive nature of the hybrid and users first reactions to them.

To study this, we selected four 3D applications for 2 different tablet screen sizes, a 7 inch (Samsung Galaxy Tab) and 10 inch(iPad). We collected user experiences from the 12 participants, who used these applications based on our tasks: move the avatar, familiarize with the game space, and interact with objects. The applications selected were Order and Chaos, Pocket Legend, Dungeon Defender, and YesCitiz Barcelona. Three of these applications made heavy use of overlaid 2D icons controls while the fourth (YesCitiz Barcelona 3D map) was primarily controlled by gestures.

Order and Chaos  Description

                             Figure 1. Order and Chaos UI (10″ screen)

The Order and Chaos game’s overlay UI has 10 visible 2D icons and 1 hidden icon on the left for avatar control. The hidden icon is for navigation control and appears when the user touches the bottom left space of the screen. The embedded UI has interactive 3D objects e.g. in the form of other online gamers and enemies, but there are also 2D icons to indicate interaction possibilities in the embedded game space. Other kind of visual cues include colored circles below avatars and text above important objects. Occasionally one can use a ‘targeted’  navigation’ by tapping a yellow 2D arrow icon on the game space, although these are not that common.

Pocket Legends Description

Figure 2. Pocket Legends UI (7″)

The Pocket Legend game’s overlay UI has 14 visible 2D icons. The interactive 3D UI objects embedded in the 3D space are in the form of magic portals, other online gamers and enemies. Text appears on important objects to indicate interaction. The avatar is controlled by pressing a yellow star icon the left side of the UI, and then dragging the icon. Another control option is to simply tap on game space, and the avatar runs to this point. Camera adjustment works by sweeping left and right.

Dungeon Defender Description

Figure 3. Dungeon Defenders UI (on 7″)

The overlay UI for Dungeon Defenders game  has 17 visible 2D icons. The embedded 3D UI objects are in form of portals, caskets, local game avatars, online gamers and enemies. Text appears on important objects to indicate interaction. In addition, animation and color cues are often used.

Avatar control happens by pressing a grey control icon on the left side of the UI, and then dragging the icon. In the upper right side there is an icon which has five actions in one. The largest symbol is a map, which opens on game view with little transparency. Plus and minus symbols indicate zooming in and out. The arrows on the icon have the camera control from left to right.

YesCitiz Barcelona 3D map Description

YesCitiz Barcelona 3D map has an overlay UI with a single small 2D icon for a compass on the top of the right corner. The application use is dependent on touch gesture use, because the icons provide merely location information, a search menu and settings.

Figure 4  A still image of the YesCitiz Barcelona 3D map application. (10″)

Navigation happens by pressing and dragging on the screen in the desired direction. Zooming in and out is performed with spread and pinch. To rotate the view, two fingers are used at the same time, other one still and the other moving to the left or the right to turn in the desired direction. To pan the street view, two fingers are placed on the screen at the same time. Other one is kept still and the other is moved upwards or downwards to tilt the street view.

Findings

The results showed a number of interesting affects.
The fewer 2D UI controls in a design, the better the experience.
YesCitiz Barcelona was the preferred application as it was considered meaningful and the UI was viewed positively.
First, as these were touch screen devices, users first started to interact with touch gestures directly to the screen, but usually nothing happened. Most of the UI controls for navigation and actions are contained in the 2D overlay and users were  eventually forced to use these. The overlay icons demonstrated the standard problem of affordance. Users simply did not know what the 2D UI icons did by looking at them. This required users to interact and experiment with the icons to find out their meaning. For example, the boot icon in Order and Chaos was the most problematic; it was interpreted as a kick, walk and weapon to name a few. Consequently, the more icons an application had resulted in confusion and had the participants more focused on the overlay UI icons than using what is available in the embedded UI directly. It took users quite a lot of time to learn the game UI and user experience was negative, because they found them confusing to use.

Another problem that occurred was that many objects in the 3D space are non-interactive, while some are interactive. This proves problematic as a user might interact with a door, only to find its part of the scenery, and then totally ignore a door that is marked for interaction.

Users held the device in a number of ways. It was not uncommon for users to change the way they held the device during usage. Sometimes they put it on the table, resting it on the table edge, holding it in one hand while using the other hand for interaction and holding it with two hands. One could say that the design of the UI for two games (Pocket Legends, Dungeon Defenders) forces users into a 2 handed hold position with the users thumb used for the UI interaction.

Figure 5. Thumb interaction with a 3D object in the 3D space.

This however, was not how users tried to use the device at first. Generally users used their dominant hand for all interactions, including the navigation. Forcing the users into a thumb based position as the most effective way to control the application made it difficult to interact with 3D objects embedded in the screen as the users would extend their thumbs to try and reach the object, as shown in figure 5. This was ergonomically uncomfortable and the same task could be achieved by the overlaid 2D UI. This problem gets worse as the screen size increases.

Design Guidelines

We conclude that the use of the overlaid 2D UI actually inhibits the use of a 3D UI elements. In short, we summarize that hybrid 2D/3D UIs are not a good solution when designing 3D applications. Rather we propose that when designing UIs for 3D applications one should

• promote the use of direct and on screen touch gestures as the main user interaction technique.
• design location of interactive elements by taking ergonomics into account.
• design quick, easy and interesting guidance to principal gesture use.
• Overlaid UI elements should be minimized and restricted to non-core functionality.

We believe that these guidelines we believe this will result in a better user experience.

More Details

A paper on this work has been accepted by ACHI 2012, Valencia Spain and should be available in Feb 2012. Salo, K., Arhippainen L, Hickey S., (2012) Design Guidelines for Hybrid 2D/3D User Interfaces on Tablet Devices – A User Experience Evaluation.

Bandwidth optimization : Part 2 : Protocol traffic.

Table showing the compression of the Knet protocols

Datagram stacking together with zlib compression revealed that we have an opportunity to lower both amount of transmitted data as well as packet count for both up and down link. Judging the  attached table of figures, the savings we gained ranged from ~61% to ~87% for different cases.  Savings are always a good thing, but in reality this showed jerkiness in avatar movement and some level of inaccuracy in actions. Again, the test showed that blindly compressing the traffic may not be the optimal solution, but instead first classifying the traffic and making selections on what to compress may be a better solution.

Developer Diary 3 – The office space

As an alternative to the clubhouse teams, we developed an office space . The process of designing the office space was very similar than with the clubhouse. This time I started with the benchmarking: I searched for different kind of pictures of office and conference rooms to see what kind of offices had already been designed.

Then I made the list where I gathered some styles describing words and their opposites, like: small-roomy, modern-traditional and casual-businesslike. Based on this list I started to sketch the office ideas. I drew ten sketches from which three were again chosen.

The most liked ideas were a traditional home office and two modern work stations. The traditional office was liked because of it’s cozy and classy feeling. Also people liked that there were chairs for the guests too.

One of the workstation pictures was liked because it had lots of space in the table to work in and because the space looked clear. The most popular sketch was also a work station, but the people liked this more because of its different user interface. They thought that it was more interesting to have something else than the old computer and a table. Both of the ideas were also liked for their big windows and city view. People thought that it made the space feel bigger. There was also an idea that I could combine the nature view from another sketch to these designs.

Based on these three designs and the feedback I got I made again ten more sketches. From those three were chosen and two of them I modeled in Blender. The models had the same difficulty as the outdoor clubhouse, because the view from the windows had to be quite large.

Rendered pictures of the traditional office scene.

Rendered pictures from the Workstation Office.

Developer Diary 2 – Outdoor clubhouse

After the indoor clubhouse, we wanted to try out an outdoors version of the club, which had also had positive comments from the test group. So I modeled this space that I had previously sketched, with a few little changes like making the space bigger again.

But a problem with an outdoor model was the sky, the ground and the trees. The problem with the sky and the ground was that the models needed to be really big to create the illusion of a limitless space. And as the size of the models grew, so did the amount of the vertices needed to make it look good. Also the textured area came out really big. It doesn’t increase the filesize no matter how big the object using the texture is, but it takes more power for devices to run the preview with such big texture areas.

The problem with the trees wasn’t the size, but the amount. And to make it so that each leaf didn’t had to be modeled separately, I had to use a texture of leaves that used an alpha channel for them to appear partly transparent. But at the moment we haven’t been able to export the scene so that the leaves would appear right.

Rendered pictures from the outdoor scene.

Developer Diary 1 – The Clubhouse model

The idea was to design a 3D-space with a music theme, where music listeners and musicians could hang out, listen and share music, and chat with other people. There were some interactive features that were to be included, like a stage for bands, a dance floor and some kind of jukebox for music and video sharing. Otherwise the idea was open at that stage.

I started the clubhouse design process by going through the preference photos that were gathered from the internet in a benchmarking process. After that, I made a list of different styles that a clubhouse could be. Also to find a lot of variation, I wrote opposites for those words. Like for example: old-modern, cozy-stylish and dark-light.

After making the word list I started to make sketches. I drew 10 different kind of room ideas and with the help of our test group we selected three of the most interesting ones from those sketches. Most people liked the cellar styled club and the cozy look. People also liked the ideas were there was different spaces for each activity. But some people pointed out that even though they wanted the space to be intimate, they didn’t want it to be too small. So that if there was a lot of people online the space wouldn’t get too crowded.

The sketches that were chosen from the first ten ideas.

Next I started to develop those three sketches based on the feedback I got. I made ten more sketches from which three were chosen for the next stage.

After the sketches I started to work with the Blender. I made rough sketch-like models of the three drawings so that I could get a basic idea how they would look as three dimensional spaces. It turned out that the rooms seemed much more narrow in 3D than they had seemed in the sketches, so I had to change their original design.

From these “sketches” I chose one that I started to model. I modeled the room and the items in it and placed some textures to the models. After that I made four different versions of that model where I used different amount of vertices and textures, so that we could compare the power usage.

Finally I finalized the version where I had the most vertices and textures. After trying out the scene in tundra, I realized that I had to make the room even larger, because the space seemed very small when the avatar was placed in there.

Rendered pictures from the final clubhouse.

What people do on their Tablets

Google’s AdMob subsidiary has carried out a survey in the US about peoples tablet usage. The results are interesting. Most people use the tablet at home (82%), only 7% use it for work. Additionally, only 11% use it while travelling. Gaming is the leading service used on the device (84%) followed by information searching(78%), emailing (74%). Social networking comes in at 51%. The report from the Guardian newspaper is here. The original report can be found here. Of course, this study does not tell us a lot about why people make these choices or what kind of games they play.

Additionally, there is a separate report in the guardian on how people use tablets for work here. A rather interesting quote is “The problem with laptops in the classroom is the battery life, and the size and weight. When Apple said that it would last for 10 hours, and we realized it actually did, that was really important. And the size and weight matters too for younger children.”