Posted in: SERVO Blog (11/21 at 10:57 AM)

The Evolution of the Jade Robot

Creating the Jade Robot has been a remarkable journey for us. The Mimetics founders started as volunteers, providing robot workshops in 2001. Since then, we've brought robots to over 15,000 students and their families (Figure 2), working through five generations of robots (including three that I have designed, shown in Figures 3 through 5) in a wide variety of different venues. From the very beginning, we have strived to understand what excites children about robots and inspire them to look towards a future in STEM (Science, Technology, Engineering, and Mathematics). We've made our share of mistakes (err, "learning experiences"), but we've learned a lot (especially when it comes to listening) and have been lucky to have been helped by so many talented individuals who have given up their time and energy to help us in our workshop and classroom sessions, as well as provide us with guidance and ideas to make us better. Thirteen years later, we have something that allows us to bring robotics to a variety of different age groups from six to college level, with capabilities that make technology accessible and fun.

Figure 2. Classroom workshop, introducing students to robots and programming. These are just some of the 15,000 students we've worked with over the years.	Figure 3. The first robot we worked with. A breadboard wire-your-own based on the Tab Sumo-Bot chassis.

The genesis of the Jade robot was a result of my taking Lean Six Sigma classes while I was working at a large metropolitan electronics company and seeing what was possible in the manufacturing environment. As we were doing our early workshop sessions, I could see that we were continually having the same issues; the build process was successful (Figure 6) but it took a lot of time and energy. Generally in the latter part of the sessions, we didn’t have enough of either to do much more than demonstrate a few built-in features of the robot. 

Figure 4. The original Tab Electronics robot - nine volt battery fun!	Figure 5. The Tab.Sumo-Bot. It took quite a while, but we were able to come up with a process in which 35 or more robots could be built simultaneously without problems.

Along with this was an amazing amount of set-up and tear-down that was required for the sessions, along with the tools that went along with building the robots (to ensure proper spacing of wheels pressed onto axles, we included spark plug gappers as part of the build kit). The first step to changing this was to use pre-assembled robots.

Figure 6. the successful end to a workshop. Even astronauts get excited about robots!

Providing pre-built robots to the attendees was a game changer for us. What the attendees were doing in terms of actually experimenting with the robots at the end of the earlier sessions was now at the start; everyone was learning some of the basics about mobile robots and what they could do. Going forward, we could explain the concepts behind autonomous robots like the robotic explorers currently on Mars. By the end of the session, everyone had created a simple autonomous program to avoid obstacles that was similar to what Curiosity would run on Mars.

The workshop and classroom sessions were very successful, but had their own problems in terms of the logistics behind providing and installing PCs in different venues (with loading development tools being a major work item). What the attendees took away from these sessions was much more substantial (they were walking out with the knowledge of how to create an autonomous program for a robot), but the amount of work that was required for setup and tear-down wasn’t any less than what it was before and, just as importantly, few users could continue the experience once they left the workshops. 

This may seem like a surprising comment to many readers, but loading development tools along with communications device drivers (many robots require a USB to-serial connection) is beyond the capabilities of most users. We were amazed when we sent a sample robot and its associated paraphernalia to the science head for a major school board (after he attended one of our sessions and asked us to learn more about what we were doing), and he returned everything because they would have to install device drivers on a PC. So, not only did the need for PCs mean significant work for us to prepare for workshop sessions, but it was something that prevented many interested individuals from going forward in learning about robotics.

Figure 7. The Jade robot's main screen. Users can select from any of the eight pre-loaded applications or access the tools for configuring the robot.

The obvious solution was to eliminate the PC for beginning builders, and to simplify its use for folks that wanted to learn how to develop their own applications. To be fair, there are a number of educational robots on the market that provide the user with the ability to create simple programs, but their user interfaces typically are based on simple LEDs and text LCDs. These approaches leave a lot to be desired as they do not allow the user to create applications without instructions or be able to review and edit them. What I wanted to have was a GUI interface (Figure 7) that allowed users to not only select which programs to run, but allow them to configure the operation of the robot, as well as create their own programs without the need of a PC. This solution solves a major problem in working with the robot but it had major ramifications as to how the robot would be architected.

The vast majority of today’s low cost robots are based on eight-bit processors. These processors are fairly cheap, have excellent peripheral functions, and there are a lot of development tools available for creating applications on them. Unfortunately, they don’t have the horse power to manage multiple functions on a robot, along with a user interface (UI) and the memory requirements that go along with it.

Another issue that has always frustrated me with eight-bit processors is the speed decrease that occurs when you work with integer variables that are greater than eight bits in size; this can be a problem when you are working with sensor data. Typical hardware ADCs and PWMs use data sizes of 12 to 16 bits, necessitating use of larger integers which results in larger slower programs on eight-bit processors. It was clear that to come up with a next-generation robot, a next-generation processor was called for. I found that the next generation of 32-bit ARM Cortex processors were just what the doctor ordered. As well as executing significantly faster than their eight-bit predecessors, they generally have significantly more memory and I/O functions while being at the same or lower cost.

A further advantage are the high functionality Real Time Operating Systems (RTOS) and high performance development tools available for them. For the Jade robot, I settled on the Freescale Kinetis K22, running at 96 MHz with 512 Kbytes of Flash and 128 Kbytes of SRAM, and using the MQX RTOS. This proved to be a potent combination, allowing me to compartmentalize different tasks (such as the user interface, sensors, motors, and communications), keeping them simple and independent for easier software development.

Throughout the years of doing robot workshops, we learned the importance of listening to people. A great example of this was when we were using the Tab Sumo-Bot and were having problems with the screw that mounted the motors to its carrier. When the robot was developed (I was one of its creators), the motors that we were given as samples had larger holes than the ones provided for production. We realized this was a problem when an older gentleman approached me after a workshop saying that he used to work for a car company and they had a similar problem with screws. The solution was to put a bit of putty in the hole to hold the screw in place. Looking down at what he was showing me, I noticed that he was missing several fingers on each hand — I must confess that my first reaction was that maybe I shouldn’t be taking advice from him on how to assemble things!

What I did realize was that we had two problems that needed to be addressed, the immediate problem being the difficulty fitting the carrier attachment screws into the motors. The ultimate solution was to buy small self-tapping screws and provide them with the build materials in the workshops. Along with this, we also recognized the difficulty people with physical challenges would have in assembling anything. This further strengthened our resolve to develop a robot that would be preassembled, allowing as many people as possible to use and learn from it. This was a good example of moving past the obvious first reactions and acknowledging that there was an issue that needed to be improved upon. Ultimately, it resulted in a better, more inclusive product.

Figure 8. Jade as imagined by an industrial design house. Clean and interesting, but not what the customer wanted.	Figure 9. This is what actually excited teachers and customers. Function beats form.

We also learned the importance of not blindly listening to what the “experts” say. As a startup, we have been involved in a number of accelerators and with that goes experts that will tell you what to do. One piece of advice that we were repeatedly told was that we needed to have a nice, well designed cover for the robot. We took this advice and found an industrial design house that did some gorgeous concepts (Figure 8), which we were then told to take to prospective customers (teachers, in this case) and see which ones they liked best. As part of this canvas, I took along a hand-wired prototype to show off how I thought the UI would work, along with pictures of what the designers came up with. You can see a picture sample design alongside my prototype in Figure 9. To our amazement, 100% (every one) of the teachers we showed the renderings to said, “That’s nice” and then went on to say that what they really wanted was something that looked like the hand-built prototype (Figure 9), something their students could touch, and be able to see the components — not have it hidden away like so much technology that people have today.

We have done many sessions using the Jade robot with over 2,000 students and 50 or more teachers. With the exception of two students who wanted to have a plastic body that they could paint, no one else has complained about our choice to go with bare PCBs and exposed circuits all clearly labeled — many have, in fact, complimented us on making this choice.

Figure 10. Six LEDs that produce light at different wave lengths, plus a photodiode make up a reflectance spectrometer that allows the users of Jade to do real science and undertand some of the challenges NASA scientists have with their robots on other planets.	Figure 11. Sample reading from the spectrometer using the example spectrometer application.

 Another comment we got from teachers that we’re very glad we followed up on was the request for the robot to be able to do “real science.” This led to the inclusion of a “reflectance spectrometer” (Figure 10) on the Jade robot. This instrument consists of six different LEDs (each producing a different wavelength of light) pointing towards the underside of the robot, with a photodiode that collects the reflected energy. Based on the brightness of the light reflected off the surface the robot is running over, a waveform can be drawn and students can use it to compare against the known waveforms of different substances (Figure 11) and make a determination of what they are.

Figure 12. With the built-in user interface and spectrometer, groups of students can work together in very informal situations, learning Science, Technology, Engineering, and Mathematics (STEM).

They also learn about how difficult it is to make a definitive measurement in the “real world” as many substances (such as wood with its grain or rocks with different minerals) do not provide a homogeneous surface for the measurements. This doesn’t preclude the students from having a lot of fun doing spectrometer measurements of the different things in the classroom such as their clothes, as well as themselves and each other (Figure 12). It came as a surprise to us, but teachers hate batteries. 

They are expensive, they need to be properly disposed of and, even when locked up, they have the ability to grow legs and disappear. The Jade robot was designed with a LiPo battery to allay these concerns. The battery is actually quite large (2,200 mAh) and will easily power the Jade robot through a normal school day. Included with Jade is a wall wart charger and a micro USB cable.

This choice of battery resulted in a significant amount of pain for me. Most SERVO readers know that lithium batteries can release an amazing amount of energy in a short period of time, resulting is a large release of heat. We were asked to demonstrate a prototype with all the features of the finished Jade robot. Up to this time, we had been using AA cells, and while the battery life was nothing to write home about, we could get in some pretty good demos. What we didn’t know was that there was an intermittent short in the positive battery line (bared section of wire that would touch ground when the robot was moving and bouncing along the floor).

With the current provided by the battery, the heat caused the plastic of the robot’s chassis to melt and the intermittent short became solid. Fortunately, we got through the demo without any problems, but when I picked up the robot, I discovered that the bottom of it was blistering hot and molten plastic was dripping onto the palm of my hand. Trying to keep a pleasant smile on my face, knowing that it wasn’t a good idea to state that a product designed to be used by children was about to catch fire, I excused myself to walk calmly to the bathroom and thrust my hand in the coldest water I could get out of the tap. The final product not only is designed to prevent this situation in the future, but also has a temperature sensor on the battery.

While introductory users can program the Jade robot from its built-in OLED based UI, this is somewhat limited and does not give the same level of control of a more traditional programming environment. Most small, low cost robots have the ability to have programs downloaded into them, but the inclusion of the user interface meant that program storage would be handled differently.

One of the things that makes Jade significantly more advanced than other robots is the built-in file system that is integral to the operation of the robot. Along with executables, the file system also stores images, sounds, text, and UI definition (called “panel”) files. When the user creates an application, it is downloaded into the file system and can either be executed from the UI or using the “Jade Support” integrated development environment (IDE).

Figure 13. Programming the Jade robot is fast and easy.	Figure 14. The Jade Support Intergrated Development Environment showing Scratch program that's ready for downloading into the robot.

Originally, the Jade robot could only be programmed in a version of C (Figure 13) that was created for it, but this has been recently enhanced with the inclusion of its own version of the Scratch programming language (Figure 14). This programming language allows neophytes to very quickly and simply create applications and — with Jade Support — download and debug them. Because of the high degree of integration in Jade (the object sensors, light sensors, line sensors, motors, expansion peripherals, and built-in user interface; see Figure 15), there is a large number of APIs available to access them.

Figure 15. The features of the Jade robot are accessible through the different programming options available to the user.

In C, all these APIs and all the features of the UI panels are available to the programmer. Scratch provides the user with a subset of these functions, but does so without the need for reference information. So, while C programming will require documentation when accessing different capabilities of Jade, Scratch does not, and provides the user with a very simple and easy programming environment that they can start working with immediately.

Figure 16. Often, a highlight at the end of the workshop is when the students realize that entering the same program into multiple Jade robots and running them together will make the 'dance'. We have seen elaborate choreographies involving up to a dozen robots.

Like most readers, I’ve worked with and tried a plethora of different educational and hobbyist robots, and I have been very pleasantly surprised at how easy it is to create applications and try them out (Figure 16). The onboard programming is something that changed the way I thought about how programming robots could work – rather than the cycle of developing the program, loading it into the robot, placing the robot in the test environment, seeing it fail, taking it back to a PC, and repeating the process, the ability to program the robot in situ — as well as debugging the program right there — was also a real game changer. I was amazed at how quickly I could work through multiple iterations of a program while staying on the floor. 

Figure 17. With the built-in Bluetooth adapter and expansion capabilities, Jade can do some pretty unique things. Here, it is using the built-in camera port to send a selfie of itself to a PC.

When the Jade robot was being developed, I wanted to include an RF communications capability. Unfortunately, I could not find a hobbyist Bluetooth or Zigbee RF adapter that integrated well into the robot, so consequently I expanded my search to commercial devices. The search was escalated when we discovered what a joy it was being able to download applications and other files, as well as provide a debugging interface via RF rather than a physical connection. The Bluetooth connection also allows us to enhance the robot in ways that isn’t possible with other small robots (Figure 17).

After an exhaustive search, we found a very effective Bluetooth interface device which was built into the Jade robot and — while we have a USB interface for the same functions — all the initial users eschewed it for the Bluetooth connection. Connecting Jade robot to a PC using Bluetooth just takes a few seconds, and it eliminates the need to take the robot out of the environment to reprogram or debug it.

Figure 18. The Jade robot attracts attention everywhere it goes. Here it is being inspected by NASA Administrator Charles Bolden at Challenger Learning Centre in Toronto.

Looking back over the Jade robot development experience, we made three decisions that turned out to be serendipitous for us. I highly recommend them for anybody who is about to embark on developing their own product. The first is to base any PC software on Google Chrome apps. When we were showing prototypes of Jade to prospective customers, we discovered a very curious thing. While 90+% of the users will be on PCs, almost 100% of the decision makers use Macs. Developing PC software using Chrome apps means that the application will work (with 95% confidence) on all PCs from Windows XP to Windows 8, as well as Apple Macs and Linux workstations. The only thing required is for the Chrome browser to be installed. This development approach is still pretty new, but if you can create a web page with some JavaScript using Chrome apps, you can create applications that will work on all major PC platforms.

The second was to source locally. All of the major components in Jade robot were purchased and assembled close to home. This allowed us to buy on shorter lead times and lower quantities than if they were procured offshore. It also meant that we could respond to production issues very quickly (just a jump in the car). Also, my wife is happy because there haven’t been any calls at 3:00 in the morning over a scratchy overseas phone cable asking for “Mr. Myke.”

Figure 19. Having a 3D printer at our disposal allowed us to design a complex plastic chassis for the jade robot that was fraction of the cost of a stamped metal one.	Figure 20. this is the third generation of gripper that we have designed for Jade. It will be on Thingiverse soon, so anybody with a 3D printer can add a gripper/camera to their own robot.

The third recommendation is to buy a 3D printer. We actually got very far into the development process assuming that we would avoid any custom molded parts – the main chassis would be a piece of stamped steel. Unfortunately, what we didn’t count on was the number of standoffs and screws that would be required, as well as the time needed to assemble them. After purchasing our 3D printer, we were able to develop a very complex plastic chassis (Figure 19) with builtin standoffs, saving us 54 individual parts — about a $13 savings in product costs and 45 minutes in assembly time.

The 3D printer also gave us the ability to create some very interesting enhancements to the Jade robot — such as a gripper and camera unit — that we are looking forward to sharing in the near future (Figure 20). We’ve also got some ideas for different enhancements that we are expecting to be a lot of fun.

All of these designs will be put up on Thingiverse.com for anyone to try out, enhance, and modify. Also in the coming months, there will be an independent third party review of the Jade robot by Camp Peavy here in SERVO. 

Finally, I’m looking forward to discussing how to use small robots to introduce and excite children about science and technology. I’m also very interested in your feedback and ideas of what you would like to see in a robot. 

Figure 21.  The Jade robot is simply a lot of fun.

We have a ritual at the end of every workshop which is to get together and read through the comments of our attendees to understand how well we’ve done. We’re gratified to see that we are changing student’s perspectives on STEM subjects (often at the ages at which they are starting to think about careers) and that girls are just as successful working with the Jade Robot as boys. We cringe when somebody points out what we could do better, but every couple of workshops, we get the comment, “This was the best day of my life!” and that makes all the hard work and sacrifice worthwhile (Figure 21). SV

Mimetics' Chief Roboticist, Myke Predko is an award winning engineering manager with over 25 years experience in designing and developing hardware and software products and their manufacturing processes. Myke is also an established electronics educational author with over 15 books published on a variety of electronics and robotics themes, and was one of the designers of the Tab “Sumo-Bot.” He lives in Toronto, Canada with he wife, daughters, and their geriatric huskies.

 

 

Comments