Google’s Project Tango and Augmented Reality

google_tango-1392983456493Google recently announced project Tango – a 3D mapping framework that will allow for simple scanning and virtual generation of a real world 3D environment. This project falls in the category of several sensing applications that utilize sensors such as Microsoft’s Kinect and PrimeSense’s 3D,  Pelican Imaging, Softkinetic, and PMD that has emerged in the recent years.

Project Tango includes SDK and a phone like sensing device that is similar to Kinect in functionality, although it appears to be missing a depth sensor and texture projector. These sensors allow the phone to make over a quarter million 3D measurements every second, updating its position and orientation in real-time, combining that data into a single 3D model of the space around the user.

This framework will be beneficial when creating augmented reality experiences, and I assume that there will be a close integration with Google Glass. I can see this being used in processes where a 3D point map of real space would be needed such as drone navigation in warehouses or augmented reality scenarios. Imagine creating a 3D point cloud of a factory floor, and incorporating various scenarios such as fire or falling piece of equipment that would play out when user physically approaches the trigger area. Such scenario would be great for training. Another example would be creating a scan of museum space, and loading various audio/visual material that would supplement existing exhibits. Such supplementary material would be visible with the aid of Google Glass or similar AR interface. Tango device could also be useful for 3D model generation, a topic we previously covered here, and here.

Tango offers a glimpse of things to come in near future. Google is doing a great thing here by developing a framework that will help the future generations of artists, scientists, hobbyists and other visionaries create new ways of interaction with our environment.

Posted in 3D Modeling, 3D point cloud, Augmented Reality, Education, Entertainment, Kinect, Technology | Tagged , , , , , , , | 1 Comment

Augmented Reality Magic Book: Solar System v.3.0 (Mobile and Desktop)

solar_system_v3This is the third iteration of the augmented reality solar system project. We posted an early version of this project using BuildAR framework that worked only on desktop computers in 2011. Then we added a Flash implementation of the same concept with downloadable lesson and source plans in 2012.

As we previously mentioned, we recently switched to Metaio framework which enables us to publish our projects as mobile applications (via free Junaio AR browser – app for iOS and Android) and desktop applications (downloadable standalone packages). This latest version of the Solar system comes with a redesigned book titled Augmented Reality Magic Book: Solar System. This book contains essential factual knowledge about the planets of the Solar system (NASA.gov, 2013), and it comes with a set of interactive AR markers that project multimedia content such as 3D models, videos, images, and audio. Each planet of the solar system has 2 markers: main marker with a 3D model of the planet, and another marker that contains supplementary content. The book is available for download for free in .pdf and .pub formats. The content is taken from NASA.gov, and if you have Microsoft Publisher, you are free to alter the content under this Creative Commons license (Attribution-NonCommercial 4.0 International).

Our intent is to make this book available to the general public, specifically K-12 teachers, parents, and students, with a goal to make learning more fun, engaging and constructive. This aligns with our ongoing goal to further explore the use of Augmented Reality in learning and education, and to provide broader community with free meaningful, useful and engaging AR content and framework.

Download the Augmented Reality Magic Book: Solar System: Publisher (.Pub) | Adobe Acrobat (.PDF)

Download desktop application.

For mobile applications, you will need to download Junaio browser app from Apple store or Android Marketplace to your mobile devices.

Enjoy!

Posted in Augmented Reality, Education, Magic book, Science | Tagged , , , , | 1 Comment

Generating 3D objects – Part 2

Model of Vespa being captured with 123d Catch

It’s been a year since we described how to create an accurate 3D models  by using ReconstructMe. A lot of things have changed in our little shop, and we reflected a bit on the work we created for the past 3 years. The use of Augmented Reality has expanded, and AR entered mainstream. Tonight I googled “Augmented reality in Education”, and I ended up reviewing over 30 search result pages of AR projects, articles and presentations on this subject. Amazing stuff.

We are planing to publish several new projects that are compatible with mobile devices and Google glass. We are now switching to Metaio SDK which will allow us to develop AR projects on mobile devices as well as on the PC platform. We will keep you updated.

Back to the topic of 3D scanning and 3D modeling. The latest app that comes pretty close to streamlining 3D model generation is Autodesk’s 123D Capture. If you are not familiar with this tool, 123D Capture will let you take multiple images (up to 40 on mobile app, up to 70 on PC) of an object you wish to turn into a 3D model,  and then convert them into a textured 3D model. Conversion is actually done on Autodesk’s servers, since the application uploads the images to Autodesk’s servers for processing. Depending on the server load, it takes around 5-10 minutes for model to be generated, and what comes back is a fairly accurate (80% complete) 3D model. If you perfect the process, you can create 3D models fairly rapidly. You can judge results by yourself:

We are really impressed with the final results, and will be using this tool in our upcoming projects. Meanwhile, check out this intro tutorial on 123D Catch:

Posted in 3D Modeling, Augmented Reality | Tagged , , | 1 Comment

Lunar Phases Astronomy AR Lesson

Augmented Reality Lunar Phases Lesson

Part of the Augmented Reality Lunar Phases Lesson

This lunar phases augmented reality lesson was developed as a part of my doctoral studies, and I used it as a primary  learning content used in the AR experimental group. I chose lunar phases for my research because (a) it is a concept that depicts material rich with spatial information; (b) it represents a concept that is often difficult to grasp; and (c) it was suggested as a suitable learning content for AR treatment by several studies.

The lunar phases lesson consists of 6 sections:

  1. General Introduction to the relationship between the earth and the moon,
  2. Introduction to the lunar phases,
  3. First quarter,
  4. Second quarter,
  5. Third quarter, and
  6. Fourth quarter.
Student Interacting with Augmented Reality Lunar Phases Lesson

Student Interacting with Augmented Reality Lunar Phases Lesson

Each section explains the relationship between the earth and the moon, and combined together, creates one coherent lesson about lunar phases. This lesson has been validated by 2 experts who hold PhD’s in astrophysics, and based on observations during the data collection (n=182), students enjoyed it very much. I must note that I was not able to measure the differences between the experimental group (AR) and control group (images and text treatment).

To use this material in your classroom (free), you will need to do the following:

  1. Download both “AR scene files” and “AR + text lunar phases” PDF document.
  2. Print out the “AR+text lunar phases” PDF, as this lesson will serve as your primary instructional material.
  3. Install free BuildAR viewer.  
  4. Plug in your Web cam into the computer. You will need a working webcam to run this AR lesson.
  5. Unzip the “AR scene files”.
  6. Open “Print these markers” folder, and print “All_Markers.doc”. Once printed, cut and separate the markers, and glue them to foam board pieces.
  7. Once all complete, click on “Scene_5.scn” to run the BuildAR software.

If you have any questions, please feel free to email me.

Here is an example of interaction between the student and lunar phases lesson:

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Posted in Augmented Reality, Cognitive Load, Education, Science, Technology | Tagged , , , , , , , , | 2 Comments

Generating 3D Models for Augmented Reality Projects with Xbox Kinect – Part 1.

Creating 3D models often requires hours of labor and knowledge of complex 3D modeling software. There is no way around it; if you want to create a 3D model of a specific object, you have to search for the model, download it and tweak it, pay a 3D modeling expert to create it, or spend numerous of hours learning software such as Blender, 3D MAX or Google SketchUp to create the model yourself. Simple models such as Earth, a building, or anything rectangular may not take a lot of time to create, but when creating complex models (e.g. a buffalo), the creation process becomes more grueling.

These three articles link 1, link 2, link 3 which deal with 3D printing and using Microsoft Kinect to scan the physical models to convert them to 3D models, got us thinking about using Kinect to create 3D models for Augmented Reality applications.

Tony Buser explains how to use Kinect as a 3D model scanner to create a 3D model in this video: 3D scan cleanup project. We have followed his scanning procedure to create 3D models. After several experiments, we have identified a way to use Kinect for 3D model scanning more effective and efficient. Below is the breakdown of our procedure:

  1. Buffalo

    Figure 1

    Prepare the model to be scanned. We used a wooden buffalo (Figure 1) and placed it on a  “lazy susan” (rotating circular tray, placed on top of a table to aid in moving food on a large table).  Rotating the buffalo with hands would create an inaccurate and deformed scan, so to get the most precise scan, we used the lazy susan. While rotating, you need to pay close attention to the rotating speed. Too slow or too fast will result in a deformed 3D model.

  2. Before you install ReconstructMe, make sure you satisfy the hardware requirements (see device compatibility matrix for graphics cards) and have the necessary software installed.  After you checked for software and hardware requirements and installed ReconstructMe, follow these instructions on how to use it properly.
  3. Scan the model. Make sure that the model you are scanning is positioned a minimum of 40cm from the device and placed in an area of 1 square meter.
  4. Once you are done with the scanning, you will be asked to save the model. The file is in  OBJ or STL format and will need to be touched up.

Obtaining the scanned 3D model generated via Kinect and ReconstructMe is the first step to creating your own 3D models. There are more steps to follow to have a completed 3D model. For instance, the scanned 3D model may have some missing areas (e.g., holes), rough surfaces, extraneous surfaces, and lack colors. Fixing the model and preparing it for the final use will be the subject of the second part of this tutorial, which we plan to publish by the end of November 2012.

Here is the video we produced to help you visualize the scanning process:

Posted in 3D Modeling, Augmented Reality, Education, Kinect, Technology | Tagged , , , , | 2 Comments

Occupational Safety Scaffolding

We just posted another video produced with the help of AR. Professor Ron Dotson reviews OSHA regulations for scaffolding in this AR video. This video serves as a preview session for an assignment in OSH 379: Construction Safety. Check out the video below and the project page here.

Download the project files here.

Posted in Augmented Reality, Safety & Security, Technology | Tagged , , | 1 Comment

Current State of AR

Three years have passed since GE astonished us with their Smart Grid flash based AR commercial. Since then, AR has been slowly being embraced as mainstream technology and was deployed on various devices and platforms. During the GE campaign, we saw the release of first Web based AR implementation – FlartoolKit (ArtToolKit AR libraries for Flash). Three years later, there are several Web AR implementations.  In2AR recently released Flash based AR SDK, and SLARToolkit was developed for the Microsoft Sliverlight platform.

One problem many novice users come across is the creation of 3D models for their AR projects. In general, 3D objects are difficult to create. Depending on the complexity, it may take several days to several weeks to master various 3D modeling software packages. In our research, we discovered that Google 3D Warehouse when used with Google Sketchup, offers myriad of free 3D models that can be used to create other 3D models, scenes and scenarios. Additionally, Blender has proven to be a great developmental and supplemental tool for creation of 3D content.

Microsoft Kinect 3D scanner may be an option for those who want to create complex 3D models or to digitize existing physical 3D models. For example, you could scan a 3D object (e.g. tea kettle) with Microsoft Kinect and ReconstructMe, prepare it in MeshLab, MeshMixer, and Blender, and export it as a full 3D object to be used in AR project. Tony Buser‘s 3D scan cleanup project with exclusion of 3D printing is a good example of this concept. This workflow has the potential to simplify the process of creation of 3D objects and make it easier for novices to create content for their AR projects.

In the area of fiducial marker recognition and processing, there are indications that fiducial marker based AR is being  replaced with image recognition algorithms. Fiducial markers have to be created with great care and are unsightly when compared to image based markers. It is easier to associate an image of a monkey with a 3D model of a monkey than associating the same 3D model with hiro fiducial marker.

And lastly, AR interaces such as see-through HMD’s are getting more advanced. In the past three years, several companies (Vuzix, Brother, Epson) introduced commercial grade AR see-through goggles. In the Spring of 2012, Google has also entered the AR interface market by announcing project Glass. Almost all of these interfaces are experimental or well above the $1000 price tag, but as this technology becomes more available, we can expect a price drop within a year or two.

The future for AR seems bright and it will be exciting to see what the next three years bring.

Posted in Augmented Reality, Flash, Technology | Tagged , , , , , , , , | Leave a comment

Augmented Reality (AR) Solar System Magic Book

We created this project as an idea generator for further AR projects at Critical Thinking session at EKU. The content for this AR solar system was taken from NASA Web site and the concept of AR Magic Book was originally created by Dr. Billinghurst. All project files are free and available to download below.

Project Files:

Posted in Augmented Reality, Education, Magic book, Technology | Tagged , , , | 4 Comments

Safety Audit and Document AR Project

This is another example of educational video produced with AR. Dr. Dunlap used previously created warehouse model to develop a student assignment around it (Safety audit and document).  There will be no posted files for this project as we already posted project files and “how to” here. This time we are only posting the video to the project.

Posted in Augmented Reality, Education, Safety & Security | Tagged , , , , , , , | 1 Comment

Consumer grade see through HMD glasses from Vuzix

We are a step closer to a consumer grade see through HMD’s this year with the upcoming product from Vuzix. Sean Buckley interviewed Vuzix engineers at CES 2012 and brought you this video:

Posted in Augmented Reality, Technology | Tagged , , | Leave a comment