Humans are entirely responsible for the pollution crisis that’s currently killing wildlife around the world, and ultimately, it’s going to be up to us to fix it. In the meantime, smart machines could help reduce our impact on species like seabirds, fish, and turtles and make our natural spaces a lot more pleasant in the process. Two Dutch engineers have invented an autonomous beach-cleaning robot called the BeachBot, which learns how to identify and clean up discarded cigarette butts using Microsoft’s artificial intelligence systems.

Edwin Bos got the idea for BeachBot (“BB” for short) after his four-year-old son dug into the sand at their favorite beach and produced a cigarette butt. Bos raked his hand through the sand and found more. “It’s such a beautiful place,” he says of Scheveningen Beach, located just outside The Hague in The Netherlands. “It really amazes me that all these things are just lying around. The filters of the cigarettes are full of micro-plastics. It’s bad that these end up in nature.”

Bos teamed up with fellow entrepreneur Martijn Lukaart to start TechTics, a company that combines artificial intelligence, mixed reality, and robotics technologies to create sustainable innovations. They produced the first AI-based detection algorithm that specifically spots cigarette butts in the sand and worked with students from Delft University of Technology to develop the BeachBot, which today uses a Convolutional Neural Network (CNN) – a self-learning algorithm – to help it see litter of all shapes and sizes in all kinds of lighting conditions.

That learning process relies on input from the public. The company solicits photos of cigarette butt litter through Microsoft Trove, an app that connects AI developers with photo-takers and grants them modest compensation for each image (about 25 cents). The app also lets participants see exactly how their images are used. TechTics is hoping to get thousands of photos to help BeachBot’s AI system recognize and remember exactly what cigarette butts look like. Eventually, the team also plans to add new types of litter to the mix.

“The system learns how to see pictures like a child recognizing an object for the first time,” says Christian Liensberger, lead principal program manager of Trove. “The bot does all the legwork. It goes to the beach, and it is the hero of cleaning,” he adds. “But to clean, it needs all these people to give it consistent data input. Without that, the bot will run into new situations it doesn’t understand. Machines like this only work because of people.”

A prototype BeachBot machine demonstrated its abilities in September 2020 for World Cleanup Day, scooping 10 cigarette butts with its gripper arms into an internal bin within 30 minutes. It uses one onboard camera to watch for people and obstacles, and another to look down at the sand. It’s battery powered and can currently run for about an hour before it needs a recharge. To make it more efficient, TechTips is developing two smaller “helper” robots that will cruise the beach looking for cigarette butts, then send BeachBot a map to their locations. A second demonstration is set for later this summer.