Art Spotlight blog – Mixed Reality Storytelling

My name is Dimitri Darzentas and I am a Researcher at the Mixed Reality Lab of the University of Nottingham. For the last couple of years I have been working on a project that explores the effects of technologies such as the Internet of Things and Social Media.

Specifically I have been researching the ways by which communities of practice, such as those that form around a hobby, record and share information about their activities. Especially those hobbies that are built around collecting or crafting objects.

We chose to focus on the pastime of Tabletop Miniature Wargaming as it encompasses a wealth of artful crafting practices, where hobbyists design, assemble and paint their armies of miniatures. Furthermore it features a wealth of narrative and storytelling, as players create detailed backgrounds for their armies and characters, especially in the specific game we chose to focus on, namely Games Workshop’s Warhammer 40,000.

This choice was further reinforced by our proximity to GW’s Nottingham-based headquarters and their extensive visitor’s centre, the well known ‘Warhammer World’.

The project has now culminated in an upcoming exhibit, the “Mixed Reality Storytelling” project, which you can find out more about here. The exhibit will be publicly displayed at the National Videogame Arcade in Nottingham city centre. It will feature an Augmented Reality – enabled Diorama, and more pertinently, a 3D scanning booth that we have put together which generates high-quality 3D models of wargaming miniatures. Visitors will be able to bring along their own miniatures and have them turned into 3D models for their own use.

Ravasch Cario’s Creation

This brings us to the real subject of this article! The 3D model of Ravasch Cario! He was one of the first miniatures that were scanned and the result truly showed the quality that the scanning process was capable of.

Our scanning method is based on photogrammetry, instead of laser or structured light scanning. Thus it uses only still images of the subject. This choice was made for a number of reasons. First, most laser or SL scanners are considerably expensive with relatively questionable applicability for our purposes. They also tend to be rendered obsolete and thus require the purchase of newer equipment. Furthermore, final image quality was paramount to us, meaning that in order to show the detail of how a model was painted, we needed the best image quality possible in order to generate the final textures. Laser and SL scanners usually have quite low resolution cameras and the final textures are not the best they can be. Photogrammetry on the other hand scales very well as the images improve, and benefits greatly from a higher megapixel count. The images that are used to generate the model are also used to generate the textures, and are thus effectively photorealistic.

With the above in mind we chose to use Agisoft Photoscan, a very well known and respected photogrammetry software application. We shot the images of each model using a lightbox and a bluetooth controlled turntable. The most crucial element of the scanning setup was the camera. As a high pixel count is beneficial to the photogrammetry process we decided on the use of a Full-Frame DSLR camera, specifically a Pentax K-1. A variety of lenses were used, including wide angle lenses and Macro Primes, depending on the size of the miniature that was being scanned.

In the case of Ravasch Cario, and most of the miniatures, we shot 36 photos of the miniature per rotation of the turntable, so effectively one per 10 degrees. We repeated this process for three different angles, low, mid and high, in order to capture as much of the miniature as possible in a total of 108 images. All the images where shot in RAW format to preserve the detail as much as possible, however they were also converted to JPG for demonstration purposes.

From this point onwards, the work was done almost entirely within Agisoft Photoscan. Following the normal process, each set of photos was aligned, thus generating a sparse cloud of the model. Once this was done it was easy to check the alignment of the photos as they would form concentric circles for each height angle as seen in the images. Next was the lengthiest process, which was the generation of the dense point cloud. Once that was done, the mesh could be created and if needed decimated, as on average each scanned model had several million vertices.

The final step prior to exporting was to generate the textures, a task easily accomplished within photoscan with the shot images.

Sketchfab

It was at this stage that Sketchfab came into play. The models that resulted from the scanning process were impressive, however they were by no means in an easy state to present. All of them were much too detailed, with millions of vertices and faces, far beyond the point of diminishing returns. While they could be decimated and simplified within the software, we wanted to preserve and demonstrate their highest quality. Furthermore, they were all misaligned with highly offset rotations and origins. Fixing these issues and preparing the models for use within our Augmented Reality applications would require importing them into dedicated 3D modelling software.

However, photoscan included in its options a direct upload to Sketchfab through its API. This meant that immediately after a scan had finished, I could upload the results to my Sketchfab account. There, the model’s origin points were automatically corrected, and a 2-click process in the 3D settings view sorted out the rotation as well. From that point on I could showcase the model in any way I liked, using the excellent in-built editor to set the environment, lighting and materials.

And I have to stress that these were the original un-decimated 3D scans. Sketchfab’s viewer loaded them without a single hiccup. We imported the same models into dedicated Game Engines that had trouble displaying them.

Instead of having to develop or purchase a 3D model viewing solution for our website,

Sketchfab’s embedding options worked right out of the box. From WordPress portal to Facebook pages and post, it was straightforward to share our models online.

And here he is:

And in closing, the nicest surprise was how welcoming the Sketchfab community has been. Within hours of the first model being uploaded, we had a wealth of positive comments and opportunities. I couldn’t have asked for more.