{"id":19022,"date":"2019-04-22T01:20:45","date_gmt":"2019-04-22T05:20:45","guid":{"rendered":"https:\/\/mdpair.com\/neuroflight-is-the-worlds-first-neural-network-enabled-drone-controller\/"},"modified":"2019-04-22T01:20:45","modified_gmt":"2019-04-22T05:20:45","slug":"neuroflight-is-the-worlds-first-neural-network-enabled-drone-controller","status":"publish","type":"post","link":"https:\/\/mdpair.com\/?p=19022","title":{"rendered":"Neuroflight Is the World\u2019s First Neural-Network-Enabled Drone Controller"},"content":{"rendered":"

Sections \tSearch \t \t\t\t \t\t\t\t \t\t\t\t\t\t\t \t\t \t\t \t\t \t \t \t\t\t\t\t \t\t\t\t\t\t\t\t \t\t\t\t\tNeuroflight Is the World\u2019s First Neural-Network-Enabled Drone Controller \t\t\t\t\t<\/p>\n

BU researchers are using competitive drone racing as a testing ground to hone AI-controlled flight<\/h2>\n

\t\t\t\t \t\t\t\t \t\t\t \t\t\t \t\t\t\t \t\t\t\t\t04.22.2019 \t\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t \t\t\t \t\t\t \t\t\t\t\"Photograph <\/p>\n

A passion for drone racing inspired Wil Koch, a BU computer scientist, to develop a machine-learning-enabled quadcopter drone controller that could advance technology for AI-controlled vehicles. Photos courtesy of Wil Koch<\/p>\n

After Wil Koch flew a friend\u2019s drone for the first time, operating it through \u201cfirst-person view\u201d where a person wears a headset connected to a video feed streaming live from a camera on the drone, he thought it was amazing. So amazing that he went out that same day and purchased his own system\u2014a video headset, controller, and quadcopter drone, named for the four propellers that power it.<\/p>\n

\u201cYou put the goggles on and they allow you to see live video transmitting from a camera mount on the drone,\u201d Koch says. It is \u201cby far, the coolest thing.\u201d <\/p>\n

First-person-view drone racing is gaining popularity among technology enthusiasts, and there are competitive races around the world. Just a few weeks after his introduction to the sport, Koch, a Boston University graduate researcher at the Rafik B. Hariri Institute for Computing and Computational Science, founded Boston Drone Racing<\/a> as a new BU Spark! computing club<\/a>. <\/p>\n

But because Koch thinks like a computer scientist, his mind soon turned to looking for ways that he could take \u201cthe coolest thing\u201d and make it even cooler. What if, he wondered, you could leverage artificial intelligence to fly a drone faster and more precisely than the standard set-up? <\/p>\n

Passion project<\/h3>\n

Koch probably would never have pursued the idea if not for that day when he flew his friend\u2019s drone from a bird\u2019s-eye view. But it was his newfound passion that would inspire a breakthrough in neural network technology, when he and a team of collaborators built Neuroflight<\/a>\u2014the first drone flight controller software powered by machine learning\u2014to optimize flight performance.<\/p>\n

\u201cHe didn\u2019t come to BU to work on Neuroflight\u2014his prior love was related to cybersecurity and defending against autonomous cyberattacks from \u2018zombie\u2019 computers,\u201d says Koch\u2019s faculty advisor Azer Bestavros, founding director of the Hariri Institute and senior author on the team\u2019s first public paper describing Neuroflight<\/a>. <\/p>\n

But after Koch fell in love with drone racing, \u201che flipped on me,\u201d Bestavros says with a laugh. Looking into research at the intersection of drones and artificial intelligence, Koch and Bestavros learned that General Electric and other industry titans were aggressively pursuing technology in that space. <\/p>\n

\u201cWil and I confirmed the value and potential of this line of work, thinking about control of autonomous vehicles and how you might use AI and machine learning to do that,\u201d says Bestavros, who is also a William Fairfield Warren Distinguished Professor and a College of Arts & Sciences professor of computer science. \u201cJust like the progression of technology in Formula 1 racing has created technologies we see in our own vehicles,\u201d he says their hope is that developing new solutions that withstand the extremes of drone racing will push the broader field of autonomous flight technology to a better place. <\/p>\n

\"Photograph<\/p>\n

First-person-view droning is a technique where a person wears a headset connected to a video feed streaming live from a camera on the drone. Wil Koch, pictured here, uses the technique to operate a drone outfitted with a Neuroflight controller, which utilizes a trained neural network to maneuver through dynamic environmental conditions like wind.<\/p>\n

Currently, drones and most other remote-controlled vehicles are operated through linear controllers that can\u2019t adapt to changing conditions. \u201cImagine you\u2019re driving a car on the road and one tire goes flat,\u201d Bestavros says. \u201cYou, as the driver, wouldn\u2019t do the same things you would do if you were driving the car with all four wheels. You\u2019d steer and accelerate differently.\u201d<\/p>\n

A typical quadcopter uses a conventional controller\u2014known as a proportional integral derivative, or PID, controller in the computer science world. That allows the operator to give the drone commands to move in a certain direction and velocity by moving the controller\u2019s joysticks. But current controller technology doesn\u2019t contain any inherent ability to adapt to changing conditions, like higher winds or (hopefully not) even the loss of a propeller. <\/p>\n

Watch Neuroflight\u2019s maiden voyage in the skies from its onboard camera. The successful flight proved it\u2019s possible to operate a drone that\u2019s outfitted with a neural-network-trained controller. Video courtesy of Wil Koch<\/p>\n

The Neuroflight controller, Koch says, is trained in computer simulation to adapt to a wide range of different events, correcting the drone\u2019s position inside a dynamic and changing, albeit digital, environment. After simulation training, the \u201ceducated\u201d neural network goes to work in the real world by sending signals to the drone motors, telling them how to respond so that the quadcopter moves in the exact way that its operator intends.<\/p>\n

\u201cPID is a linear control system, but the environment is nonlinear,\u201d says Koch, who is a College of Arts & Sciences graduate student in computer science. \u201cWe\u2019re ripping out that PID controller and dropping in a trained neural network.\u201d <\/p>\n

Neuroflight, hatched<\/h3>\n

After three months of experimental computer simulation, Neuroflight took to the skies on its maiden voyage in November 2018, a milestone for the world\u2019s first machine-learning-optimized drone controller. The drone racing community is already buzzing about the news, with online influencers highlighting the advance and breaking down how the technology works on their YouTube channels.<\/p>\n

\u201cUp until now, our primary goals were to identify if this was possible or feasible,\u201d Koch says. That \u201cit got off the ground and works in the real world\u201d is important in itself. But he\u2019s quick to add: \u201cWe haven\u2019t even begun the most important aspects of making Neuroflight more powerful than PID. At the moment, this is not something I would take to a race; it still needs attention and work to address particular things such as drift,\u201d which can cause a drone to veer off course.<\/p>\n

To take Neuroflight to the next level, Koch is working on building a digital twin of his racing drone.<\/p>\n

\u201cWe\u2019ve been training Neuroflight in simulation using a completely different aircraft,\u201d Koch says. \u201cNow that we have a baseline, our goal is to reduce that gap between simulation and real world, and the first step in doing that is to create a complete digital replica of the real drone.\u201d<\/p>\n

Koch will lean on his CAD software skills to build 3D models and assemble his drone\u2019s components in a computer-simulation environment. The more complicated step, he says, will be creating the dynamic models for the drone\u2019s motors and propellers.<\/p>\n

\u201cThe neural network shouldn\u2019t be able to tell the difference between simulated and real world. That\u2019s why we\u2019re building up the digital twin concept,\u201d he says. \u201cOur hypothesis is that since we\u2019re going to be increasing the realism in simulation, we should see increases in accuracy in the real world.\u201d<\/p>\n

The continuing work, Koch and Bestavros say, will showcase the true benefits of using machine learning for control, improving effectiveness in the face of uncertainty. <\/p>\n

\u201cWhat Wil is doing is part of the broader picture of using AI-controlled technology,\u201d as well as advancing human acceptance of AI controllers, Bestavros says. To move basic research in this space forward, he says, \u201cdrones are easy because they aren\u2019t carrying human beings.\u201d<\/p>\n

Today<\/h2>\n