Category: AI/Robotics

The Sketch-a-Net program successfully identified a seagull, pigeon, flying bird and standing bird better than humans (credit: QMUL, Mathias Eitz, James Hays and Marc Alexa)

The first computer program that can recognize hand-drawn sketches better than humans has been developed by researchers from Queen Mary University of London.

Known as Sketch-a-Net, the program correctly identified the subject of sketches 74.9 per cent of the time compared to humans that only managed a success rate of 73.1 per cent.

As sketching becomes more relevant with the increase in the use of touchscreens, it could lead to new ways to interact with computers. Touchscreens could understand what you are drawing enabling you to retrieve a specific image by drawing it with your fingers, which is more natural than keyword searches for finding items such as furniture or fashion accessories.

The improvement could also aid police forensics when an artist’s impression of a criminal needs to be matched to a mugshot or CCTV database.

The research also showed that the program performed better at determining finer details in sketches. For example, it was able to successfully distinguish “seagull,” “flying-bird,” “standing-bird” and “pigeon” with 42.5 per cent accuracy compared to humans, who only achieved 24.8 per cent.

Sketch-a-Net is a “deep neural network” program, designed to emulate the processing of the human brain. It is particularly successful because it accommodates the unique characteristics of sketches, particularly the order the strokes were drawn. This was information that was previously ignored but is especially important for understanding drawings on touchscreens.

Abstract of Sketch-a-Net that Beats Humans

We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans. Our superior performance is a result of explicitly embedding the unique characteristics of sketches in our model: (i) a network architecture designed for sketch rather than natural photo statistics, (ii) a multi-channel generalisation that encodes sequential ordering in the sketching process, and (iii) a multi-scale network ensemble with joint Bayesian fusion that accounts for the different levels of abstraction exhibited in free-hand sketches. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photo or sketch. Our network on the other hand not only delivers the best performance on the largest human sketch dataset to date, but also is small in size making efficient training possible using just CPUs.

Left: the rigid top fractures on landing, while the top made of nine layers going from rigid to flexible remains intact (credit: Jacobs School of Engineering/UC San Diego, Harvard University)

Engineers at Harvard University and the University of California, San Diego, have created the first robot with a 3D-printed body that transitions from a rigid core to a soft exterior. The robot is capable of more than 30 untethered jumps at a time and is powered by a mix of butane and oxygen.

The researchers describe the robot’s design, manufacturing and testing in the July 10 issue of the journal Science. Michael Tolley, an assistant professor of mechanical engineering at UC San Diego, and one of the paper’s co-lead authors, believes bringing together soft and rigid materials will help create a “new generation of fast, agile robots that are more robust and adaptable than their predecessors and can safely work side by side with humans.” And maybe help prevent (or cushion) those “that’s gotta hurt” falls experienced by some robots participating in the recent DARPA Robotics Challenge.

The idea of blending soft and hard materials into the robot’s body came from nature, Tolley said. For example, certain species of mussels have a foot that starts out soft and then becomes rigid at the point where it makes contact with rocks. “In nature, complexity has a very low cost,” Tolley said. “Using new manufacturing techniques like 3D printing, we’re trying to translate this to robotics.”

Soft robots tend to be slow, especially when accomplishing tasks without being tethered to power sources and other electronics, said Tolley, who recently co-authored a research review on soft robotics for Nature (Rus, Tolley, v. 521, pp. 467-475). Adding rigid components should help, without compromising the safety of the humans who would work with them.

In the case of the robot described in Science, rigid layers also make for a better interface with the device’s electronic brains and power sources. The soft layers make it less vulnerable to damage when it lands after jumping .

How it works

The robot is made of two nestled hemispheres. The top hemisphere is like a half shell, 3D-printed in once piece, with nine different layers of stiffness, creating a structure that goes from rubber-like flexibility on the exterior to full rigidity near to core. Researchers tried several versions of the design and concluded that a fully rigid top would make for higher jumps. But a more flexible top was more likely to survive impacts on landing, allowing the robot to be reused. They decided to go with the more flexible design.

The bottom half of the robot is flexible and includes a small chamber where oxygen and butane are injected before it jumps. After the gases are ignited, this half behaves very much like a basketball that gets inflated almost instantaneously, propelling the robot into a jump. When the chemical charge is exhausted, the bottom hemisphere goes back to its original shape.

The two hemispheres surround a rigid core module that houses a custom circuit board, high-voltage power source, battery, miniature air compressor, butane fuel cell and other components. In a series of tests, the robot jumped two and a half feet (0.75 m) in height and half a foot (0.15m) laterally. In experiments, the robot jumped more than 100 times and survived an additional 35 falls from a height of almost four feet.

Jacobs School of Engineering/UC San Diego, Harvard University | 3D-printed robot is hard at heart, soft on the outside

Wyss Institute/Harvard University | 3D Printed Soft Jumping Robot

Abstract of A 3D-printed, functionally graded soft robot powered by combustion

Roboticists have begun to design biologically inspired robots with soft or partially soft bodies, which have the potential to be more robust and adaptable, and safer for human interaction, than traditional rigid robots. However, key challenges in the design and manufacture of soft robots include the complex fabrication processes and the interfacing of soft and rigid components. We used multimaterial three-dimensional (3D) printing to manufacture a combustion-powered robot whose body transitions from a rigid core to a soft exterior. This stiffness gradient, spanning three orders of magnitude in modulus, enables reliable interfacing between rigid driving components (controller, battery, etc.) and the primarily soft body, and also enhances performance. Powered by the combustion of butane and oxygen, this robot is able to perform untethered jumping.

Researchers in France and the U.S. have developed a new technology that enables robots to quickly recover from an injury in less than two minutes, similar to how injured animals adapt. Such autonomous mobile robots would be useful in remote or hostile environments such as disaster areas, space, and deep oceans.

The video above shows a six-legged robot that adapts to keep walking even if two of its legs are broken. It also shows a robotic arm that learned how to correctly place an object even with several broken motors.

“When injured, animals do not start learning from scratch,” says Jean-Baptiste Mouret from Pierre and Marie Curie University. “Instead, they have intuitions about different ways to behave. These intuitions allow them to intelligently select a few, different behaviors to try out and, after these tests, they choose one that works in spite of the injury. We made robots that can do the same.”

The researchers developed an “Intelligent Trial and Error” algorithm that allows robots to emulate animals: the robots conduct experiments to rapidly discover a compensatory behavior that works despite the damage.

“For example, if walking, mostly on its hind legs, does not work well, it will next try walking mostly on its front legs,”  explains Antoine Cully, lead author of a May 28 cover article on this research in the journal Nature. “What’s surprising is how quickly it can learn a new way to walk. It’s amazing to watch a robot go from crippled and flailing around to efficiently limping away in about two minutes.”

Abstract of Robots that can adapt like animals.

Robots have transformed many industries, most notably manufacturing¹, and have the power to deliver tremendous benefits to society, such as in search and rescue², disaster response³, health care⁴ and transportation⁵. They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets⁶ to deep oceans⁷. A major obstacle to their widespread adoption in more complex environments outside factories is their fragility^6, 8. Whereas animals can quickly adapt to injuries, current robots cannot ‘think outside the box’ to find a compensatory behaviour when they are damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes⁹, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots^6, 8. A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage^10, 11, but current techniques are slow even with small, constrained search spaces¹². Here we introduce an intelligent trial-and-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot’s prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury.

Massachusetts Institute of Technology| MIT cheetah robot lands the running jump

The MIT researchers who built a robotic cheetah have now trained it to see and jump over hurdles as it runs — making it the first four-legged robot to run and jump over obstacles autonomously.

The robot estimates an obstacle’s height and distance, gauges the best distance from which to jump, and adjusts its stride to land just short of the obstacle, before exerting enough force to push up and over. Based on the obstacle’s height, the robot then applies a certain amount of force to land safely, before resuming its initial pace.

In experiments on a treadmill and an indoor track, the cheetah robot successfully cleared obstacles up to 18 inches tall — more than half of the robot’s own height — while maintaining an average running speed of 5 miles per hour.

“A running jump is a truly dynamic behavior,” says Sangbae Kim, an assistant professor of mechanical engineering at MIT. “You have to manage balance and energy, and be able to handle impact after landing. Our robot is specifically designed for those highly dynamic behaviors.”

Onboard LIDAR + path-planing algorithm –> autonomous control

As KurzweilAI reported last September, the engineers previously demonstrated that the robotic cheetah was able to run untethered— performed “blind,” without the use of cameras or other vision systems.

Now, the robot can “see,” with the use of onboard LIDAR — a visual system that uses reflections from a laser to map terrain (also used in autonomous vehicles). The team developed a three-part algorithm to plan out the robot’s path, based on LIDAR data. Both the vision and path-planning system are onboard the robot, giving it complete autonomous control.

The team tested the cheetah’s jumping ability first on a treadmill, then on a track. On the treadmill, the robot ran tethered in place, as researchers placed obstacles of varying heights on the belt. After multiple runs, the robot successfully cleared about 70 percent of the hurdles.

In comparison, tests on an indoor track proved much easier, as the robot had more space and time in which to see, approach, and clear obstacles. In these runs, the robot successfully cleared about 90 percent of obstacles.

The team is now working on getting the MIT cheetah to jump over hurdles while running on softer terrain, like a grassy field.

This research was funded in part by the Defense Advanced Research Projects Agency .

An intelligent handheld robot assisting a user in placing correct colored tiles (credit: University of Bristol)

What if your handheld tools knew what needs to be done and were even able to guide and help you complete jobs that require skills? University of Bristol researchers are finding out by building and testing intelligent handheld robots.

Think of them as smart power tools that “know” what they’re doing — and could even help you use them.

The robot tools would have three levels of autonomy, said Walterio Mayol-Cuevas, Reader in Robotics Computer Vision and Mobile Systems: “No autonomy, semi-autonomous — the robot advises the user but does not act, and fully autonomous — the robot advises and acts even by correcting or refusing to perform incorrect user actions.”

The Bristol team has experimented with tasks such as picking and dropping different objects to form tile patterns and aiming in 3D for simulated painting.

The robot designs are open source and available on the university’s HandheldRobotics page.

HandheldRobotics | The Design and Evaluation of a Cooperative Handheld Robot