Use Interactive Printed Models

Step 1: Install Talkit++ on your iOS devices

Talkit++ is an iOS application that plays audio and visual content as a user touches parts of a 3D print. With Talkit++, a visually impaired student can explore a printed model tactilely and use hand gestures and speech commands to get more information about certain elements in the model. Talkit++ detects the model and hand gestures using computer vision algorithms, simple accessories like paper stickers and printable trackers, and the built-in RGB camera on an iOS device. Based on the model’s position and the user’s input, Talkit++ speaks textual information, plays audio recordings, and displays visual animations.

Please visit this Github page for installation instructions

Step 2: Print sample models

We have prepared four sample models. Please download them using the link below and print them out in 3D printers.

1. The Globe Model

This model presents five continents. You need a 3D tracker (from the Step 3) to use this model.​


2. The Map Model

This model consists of 6 STL files. The model is a tactile version of Washington State regional map and has five pieces and a base.  Each piece has one interactive element with audio effect. You need to add 2D trackers (from the Step 3) to use this model.

3. The Volcano Model

The Volcano model has four interactive elements, and features an animation of a volcano eruption. You need to add 2D trackers (from the Step 3) to use this model.

4. The Plane Model

The Plane model has four interactive elements, and three of them have audio effects. You need a 3D tracker (from the Step 3) to use this model.​

Step 3: Prepare Accessories

There are two types of accessories required to use our interactive printed models.

1. Stickers:

We use red stickers, which can be found in Amazon. Alternatively, you can print the red bar below and use double-sided tape to stick the red paper on your fingertip.

2. Trackers:

There are two types of trackers. 2D trackers and 3D trackers. Regardless of their dimensions, the trackers consists of bicolor markers. You can find the markers we used in this document. Please download it and print it out in letter size papers. For 3D trackers, you also need to print this STL file. The document also contains instructions about how to assemble markers.

Step 4: Launch the Application and Enjoy

Please feel free to contact us if you have any questions.

Design Interactive Printed Models

Under Construction