Generating 3D Models for Augmented Reality Projects with Xbox Kinect – Part 1.

Creating 3D models often requires hours of labor and knowledge of complex 3D modeling software. There is no way around it; if you want to create a 3D model of a specific object, you have to search for the model, download it and tweak it, pay a 3D modeling expert to create it, or spend numerous of hours learning software such as Blender, 3D MAX or Google SketchUp to create the model yourself. Simple models such as Earth, a building, or anything rectangular may not take a lot of time to create, but when creating complex models (e.g. a buffalo), the creation process becomes more grueling.

These three articles link 1, link 2, link 3 which deal with 3D printing and using Microsoft Kinect to scan the physical models to convert them to 3D models, got us thinking about using Kinect to create 3D models for Augmented Reality applications.

Tony Buser explains how to use Kinect as a 3D model scanner to create a 3D model in this video: 3D scan cleanup project. We have followed his scanning procedure to create 3D models. After several experiments, we have identified a way to use Kinect for 3D model scanning more effective and efficient. Below is the breakdown of our procedure:

  1. Buffalo

    Figure 1

    Prepare the model to be scanned. We used a wooden buffalo (Figure 1) and placed it on a  “lazy susan” (rotating circular tray, placed on top of a table to aid in moving food on a large table).  Rotating the buffalo with hands would create an inaccurate and deformed scan, so to get the most precise scan, we used the lazy susan. While rotating, you need to pay close attention to the rotating speed. Too slow or too fast will result in a deformed 3D model.

  2. Before you install ReconstructMe, make sure you satisfy the hardware requirements (see device compatibility matrix for graphics cards) and have the necessary software installed.  After you checked for software and hardware requirements and installed ReconstructMe, follow these instructions on how to use it properly.
  3. Scan the model. Make sure that the model you are scanning is positioned a minimum of 40cm from the device and placed in an area of 1 square meter.
  4. Once you are done with the scanning, you will be asked to save the model. The file is in  OBJ or STL format and will need to be touched up.

Obtaining the scanned 3D model generated via Kinect and ReconstructMe is the first step to creating your own 3D models. There are more steps to follow to have a completed 3D model. For instance, the scanned 3D model may have some missing areas (e.g., holes), rough surfaces, extraneous surfaces, and lack colors. Fixing the model and preparing it for the final use will be the subject of the second part of this tutorial, which we plan to publish by the end of November 2012.

Here is the video we produced to help you visualize the scanning process:

Posted in 3D Modeling, Augmented Reality, Education, Kinect, Technology | Tagged , , , , | 2 Comments

Occupational Safety Scaffolding

We just posted another video produced with the help of AR. Professor Ron Dotson reviews OSHA regulations for scaffolding in this AR video. This video serves as a preview session for an assignment in OSH 379: Construction Safety. Check out the video below and the project page here.

Download the project files here.

Posted in Augmented Reality, Safety & Security, Technology | Tagged , , | 1 Comment

Current State of AR

Three years have passed since GE astonished us with their Smart Grid flash based AR commercial. Since then, AR has been slowly being embraced as mainstream technology and was deployed on various devices and platforms. During the GE campaign, we saw the release of first Web based AR implementation – FlartoolKit (ArtToolKit AR libraries for Flash). Three years later, there are several Web AR implementations.  In2AR recently released Flash based AR SDK, and SLARToolkit was developed for the Microsoft Sliverlight platform.

One problem many novice users come across is the creation of 3D models for their AR projects. In general, 3D objects are difficult to create. Depending on the complexity, it may take several days to several weeks to master various 3D modeling software packages. In our research, we discovered that Google 3D Warehouse when used with Google Sketchup, offers myriad of free 3D models that can be used to create other 3D models, scenes and scenarios. Additionally, Blender has proven to be a great developmental and supplemental tool for creation of 3D content.

Microsoft Kinect 3D scanner may be an option for those who want to create complex 3D models or to digitize existing physical 3D models. For example, you could scan a 3D object (e.g. tea kettle) with Microsoft Kinect and ReconstructMe, prepare it in MeshLab, MeshMixer, and Blender, and export it as a full 3D object to be used in AR project. Tony Buser‘s 3D scan cleanup project with exclusion of 3D printing is a good example of this concept. This workflow has the potential to simplify the process of creation of 3D objects and make it easier for novices to create content for their AR projects.

In the area of fiducial marker recognition and processing, there are indications that fiducial marker based AR is being  replaced with image recognition algorithms. Fiducial markers have to be created with great care and are unsightly when compared to image based markers. It is easier to associate an image of a monkey with a 3D model of a monkey than associating the same 3D model with hiro fiducial marker.

And lastly, AR interaces such as see-through HMD’s are getting more advanced. In the past three years, several companies (Vuzix, Brother, Epson) introduced commercial grade AR see-through goggles. In the Spring of 2012, Google has also entered the AR interface market by announcing project Glass. Almost all of these interfaces are experimental or well above the $1000 price tag, but as this technology becomes more available, we can expect a price drop within a year or two.

The future for AR seems bright and it will be exciting to see what the next three years bring.

Posted in Augmented Reality, Flash, Technology | Tagged , , , , , , , , | Comments Off on Current State of AR

Augmented Reality (AR) Solar System Magic Book

Updated version of this project can be found here: http://www.arined.org/?p=964

We created this project as an idea generator for further AR projects at Critical Thinking session at EKU. The content for this AR solar system was taken from NASA Web site and the concept of AR Magic Book was originally created by Dr. Billinghurst. All project files are free and available to download below.

Project Files:

Posted in Augmented Reality, Education, Magic book, Technology | Tagged , , , | 4 Comments

Safety Audit and Document AR Project

This is another example of educational video produced with AR. Dr. Dunlap used previously created warehouse model to develop a student assignment around it (Safety audit and document).  There will be no posted files for this project as we already posted project files and “how to” here. This time we are only posting the video to the project.

Posted in Augmented Reality, Education, Safety & Security | Tagged , , , , , , , | 1 Comment

Consumer grade see through HMD glasses from Vuzix

We are a step closer to a consumer grade see through HMD’s this year with the upcoming product from Vuzix. Sean Buckley interviewed Vuzix engineers at CES 2012 and brought you this video:

Posted in Augmented Reality, Technology | Tagged , , | Comments Off on Consumer grade see through HMD glasses from Vuzix

GM Opens a New Window on Entertainment (and learning)

Anyone who’s driven through a god-forsaken wasteland like, say, west Texas with young children has glimpsed a corner of hell. The road is long and the scenery boring, a combination that can have your kids pushing you to the limits of sanity.

General Motors wants to help you out with that.

The automaker asked the dreamers and designers at Future Lab at Israel’s Bezalel Academy of Art and Design to develop a futuristic suite of apps that effectively turn car windows into a tablet. Read the rest of the article on Wired.

Posted in Augmented Reality, Entertainment | Tagged , , , | Comments Off on GM Opens a New Window on Entertainment (and learning)

Doable, Affordable, and Sustainable Augmented Reality

It has been almost a year since we started developing Augmented Reality (AR) projects and implementing this technology into our online courses. During the past 10 to 12 months, we have conducted an AR research project and developed several AR instructional materials. In addition, we have identified a more efficient workflow for creating AR projects for our online courses and found resources that would make our AR design more sophisticated and robust. As we are reviewing what we have accomplished this year, we would like to share our thoughts about AR in education.

People often ask: “What is Augmented Reality?” The more we explain what AR is and what we are developing, the more we realize there is a need to give our AR productions an identity in order to distinguish it from other AR applications (e.g., mobile AR). Also, we often find people enthusiastic about this technology, but feeling intimidated by it because it appears to be “difficult.” We hope this essay will ease the technological anxiety and encourage more people to adopt this technology.

Doable

The essential elements for developing AR applications are 3D models. People are often intimidated by the thought of creating 3D models. Creating a 3D model from scratch is certainly not easy. However, it is not that difficult to find 3D models and modify the models to fit your needs. Google 3D Warehouse is a good place to start finding the 3D models you need. Web sites such as Turbo Squid also offer free 3D models. You will be surprised by what you can find on the Internet. There is no need to re-invent the wheel!

How to deploy the 3D model would be the second question that people ask. We have experimented with two different ways of deploying 3D models. One is BuildAR Pro; the other is Flartoolkit. If you are using BuildAR Pro, you don’t need to code to create AR applications. However, you need to learn the basic functions of this particular software such as resizing the model, creating scene, etc. There are tutorials for you to quickly become proficient with BuildAR Pro. If you want to create AR projects that could be deployed on the Internet, then you might want to consider Flartoolkit. Using Flartoolkit might sound daunting to people who are not familiar with Flash. It is true that you need to learn how to use Flash AS3, but you don’t need to be an expert of Flash AS3 to create AR projects. The first Flartoolkit is a great starting point for you to try Flash-based AR. What you really need to do is to replace the 3D model with your own. As for the multiple-markers, FlarManager will be a good place for you to explore how to create a multiple markers AR.

Regardless which AR software or platform you choose to develop your own project, there are plenty of tutorials and templates for you to learn AR and ask for helps. It is certain there will be a learning curve, but the AR community will help you to get over it.

Affordable

Cost is often the variable that determines whether a school or an individual adopts a technology (hardware and/or software). We layout the software and hardware you need to have in order to develop AR projects in the following table. It costs approximately around $ 1500 or less to create AR projects. Considering how many AR learning materials an individual would be able to create with these hardware and software, investing in AR is quite cost-effective.

Hardware PC $ 400-700
Web Cam $ 25-75
Software BuildAR Pro $ 200
Flash Professional $ 120
Fraps $ 37
Google SketchUp Free
Blender Free

Sustainable

As we have mentioned in the very beginning, creating 3D models is the essential part of AR development. While the free 3D warehouse and database makes it easy to retrieve the available 3D models, 3D modeling software (e.g., Google SketchUp) has made creating new models fairly easy. By adding or removing elements from the existing 3D models enables a designer to generate new models. At the same time, a model could be used to teach different subject maters. For example: the coffee shop model we created for the Spanish class could be used in a communication class to teach students how to evaluate communication patterns, etc. To some extent, a 3D model would continue to grow. As 3D modeling software becomes more user-friendly and accessible, it is predicted that more 3D models will become available and sustain the growth of AR.

Final Thoughts

We work as a team to develop the AR projects. Each of us focuses and explores different platforms and software to develop the projects. At the same time, we share and teach each other. It is always nice to have someone looking behind your back and giving you constructive feedback! If you could find a colleague that shares similar interests, then you probably would find it easy to create AR projects. If not, please don’t hesitate to reach out to the AR community on the Internet (or us). Remember Augmented Reality is – Doable, Affordable, and Sustainable!

Posted in Augmented Reality | 1 Comment

Augmented Reality Contact Lens

Can this be the holy grail of AR?
Innovega was founded to change how you look at the world.  By enhancing your normal vision using special contact lenses, we make it possible for you to view virtual and augmented reality images the same way you view your normal world.  There is no need for ugly and bulky head-gear because your eyes can now see digital images without traditional optical devices. more.

Posted in Augmented Reality | 1 Comment

Work Zone Safety

This augmented reality video serves as a review session for an assignment in OSH379: Construction Safety. The assignment requires students to study a series of images of a 3D work zone model to conduct an analysis of the problems with the work zone. In the video, Professor Ron Dotson demonstrated how a safety professional would address the safety issues that appeared in the 3D work zone. View the project video and download source files here.

Posted in Augmented Reality, Education, Safety & Security | Comments Off on Work Zone Safety