I should probably be calling them UAVs…
Below is a text that I wrote last fall, to get my head around opportunities around use of drones in urban environment. Given the FAA’s Small Unmanned Systems Integration Roadmap, these scenarios are all rather far-future, if feasible at all, in the aerially-challenged US. But the word is a big place. This was written before the Amazon Prime Air stunt.
Fly safely !
Drones in the City
I fly drones. When I say that, I mean that I fly radio-controlled quadcopters and model airplanes. Those machines have various levels of autopilot functionality built in, from stabilizers that keep them level in the air, to “return to home” functionality that enables the machine to come back to the launch point on its own, to multi-waypoint mission capability.
The definition of a drone is a flying device that can operate without real-time control input from a human pilot. In general discussions, the term drone is sometimes used synonymously with quadcopter or multicopter, though the term does apply to fixed wing craft (shaped liked an airplane, vs. shaped like a helicopter) too. Drone is a popular-culture term. The more professional term is UAS, unmanned aerial system. The “system” part here is important, as it acknowledges that the flying device does not exist as a standalone gadget, but as a part of a larger infrastructure (technical, legal, procedural) that makes it work.
While flying drones as a hobby is legal and fun, the FAA has asserted that it is not yet legal to operate UAS commercially in the USA. Despite this fact, a cottage industry of drone operators exists, offering services like drone-based real estate photography and aerial photo surveys, to be disrupted by occasional Cease and Desist letters from the FAA. The FAA is working on a set of rules to legalize commercial drone operation soon.
Large UAS Today
In the meantime, the US government has been using UAS as a foreign policy tool and for applications like border security. From a use case perspective, these drone applications can be loosely described as “Aircraft Replacement”, in the sense that the craft take the place of manned aircraft, and in many cases go where deployment of manned aircraft would not be feasible. The UAS used in this way are generally on or near the scale of full-size aircraft, and offer performance envelopes of a similar order: they go fast, far and high, sharing airspace used by commercial aviation. Traditional defense industry, along with with smaller startups, have been developing the hardware used in military and intelligence UAS. Many types of UAS, on the scale of aircraft or small aircraft, are under development for civilian applications, such as land surveying, agricultural applications, research, and search and rescue. (Marine drones are also being developed for military and civilian uses.)
Applications for sUAS
Small drones a.k.a. s(mall)UAS, operate on a local scale. Our hobby drones, and the octocopter operated commercially by the real estate photographer down the street, fly ten or twenty minutes. Small fixed-wing drones can fly for one, two, or in some cases, three hours. Flights are usually done close to the pilot’s location, at line-of-sight distance.
It is not hard to imagine useful applications for these local-scale drones. The taco-copter, the pizza-copter and the doner-copter concepts and trials have generated much publicity. One could envision other delivery services, in the manner of a Fedex copter dropping off the shipment from Amazon on your doorstep. Small drones could monitor utility lines, inspect bridges, and check building facades. Small drones could quickly reach medical emergencies, bringing defibrillators, snake bite serum, or emergency medication when minutes count, before the ambulance arrives.
A small drone can readily be imagined as a tool used by an operator present on site, in the immediate vicinity of the operator. This is what the local real estate photography drone guy already does with it.
A lot of deeper potential lies in longer-range uses where the drone operates within a theatre that is a few square miles in size. On the scale of a municipality, for example. Here, the drone could be characterized as a vehicle-replacement device, since it can take on tasks that are performed with cars, light trucks, and full-size trucks. Kilos of airframes are in motion, instead of tons of steel. Energy, time and money are saved.
Challenges for Small Drones
Aside from the current legal situation, both the local use and the vehicle replacement use of small drones face quite a few problems in practice. Small drones have limited range. They are not particularly reliable, they are not very easy to operate, so they are not very safe. Drones come with a bad rep: they are scary. Let’s take this one at a time.
Drones Are Scary
The Reapers and Predators that we are used to seeing in the news are in the business of spying and killing, and there is nothing subtle about this. Views around the politics around these uses aside, this existing use of drones does not readily invite a vision of civilian and business drone use to everyone’s benefit as a next step in the evolution of the technology. Especially not in front of one’s own front door, and above one’s own back yard.
Most drones, including small ones, also simply look scary. One typical design features skeletal, high-tech macho machines, borrowing aesthetic clues from power tools and from assault rifles. Another design employs smooth, curvy shapes that are creepy like a science-fiction movie android. Drones look cool to the enthusiast, but rather uninviting to the uninitiated. Encountering a drone can be an unsettling experience, as the drone, as benign and useful as it may be, gives shape to the ominous connotations around privacy and “harm from above.” It is up to the drone industry to change these connotations.
Are Drones Are Dangerous?
The unease created by the technology is real. It must be addressed if the technology is to find acceptance for civilian use, specifically for the inevitable use of drones in populated areas.
Depending on the type of airframe, a small drone has one or several spinning propellers. Hobby users of drones are familiar with the very real potential for injury associated with propellers. Even with propeller guards, acting as barriers between propellers and the environment, there is the matter of kinetic energy in a flying object. On the other hand, within the roughly seventy years of model airplane use, as tracked by the AMA (Association of Model Aeronautics), only three people have ever been killed by model airplanes, the less-safe brothers of drones. Of these three people, two were the pilots themselves, and only one person was a bystander. There is risk associated with the technology, and it is a manageable risk.
Lithium batteries, used by small drones, are a source of risk. A battery that gets punctured, in a crash landing for example, is likely to spontaneously combust with a very hot flame, igniting materials around it. Hobby users are familiar with the incidences of burnt-out garages and cars. But lithium batteries are prevailing and improving, partially driven by research of safe power for electric cars.
Aviation, in general, is conducted away from the ground. Flying high is safer than flying low, because the ground, from an aircraft’s perspective, is a source of lethal obstacles. When considering local-scale drone use scenarios, we are considering close-to-ground operation of airborne vehicles, and even close-to-structures operation: near houses, near cars, near people. The very benefits of those missions are found in their proximity to structures, after all.
Aircraft are designed for reliability. Redundant systems, closely regulated maintenance intervals, and checklists minimize failures of critical systems. Small drones today have few such safeguards in place. Software bugs and hardware failures in small drones can and do result in unexpected behaviors, crashes, and fly-aways. Not all the time and not every day, but many small drone operators today have had experiences along those lines. Reliability is a matter of engineering and testing, and thus, cost. Professional drones available today, at price points of tens of thousands of dollars, often offer capabilities matched by hundred-of-dollars hobby systems, but they offer more of the engineering and testing that creates liability. The design and engineering of large drones, based on processes originally developed for building traditional aircraft, have a lead in reliability, trading this benefit for a much higher cost and for a slower rate of innovation.
Automation Increases Safety
In aviation, accident statistics are closely tracked. With large drones gathering the majority of drone flying hours today, drone flights have reached a point where they are already safer then manned military flight operations. The primary window of occurrence for drone accidents has been during take-off and landing, and the numbers here have been improving with increasing automation. In other words, as humans are taken out of the loop, drones fly better.
This observed trend is of great consequence, beyond automation of large drones. Many of today’s commercial airplanes already have very capable autopilots, the sophistication of which is going to grow, re-shaping the presence of a pilot on board to more of a psychological comfort factor, as opposed to a necessity. Autopilots are also going to find more deployment in small airplanes for general aviation, where most flying is done manually today, and where accident statistics are the worse for it.
Looking at small drones, some systems today offer “point and click” automation for flights, while others are operated using “traditional” rate control, in the manner of model airplanes, combined with some automation. Taking humans out of the loop is clearly an opportunity for small drones. But looking at possible low-altitude uses in built-up environments, small drones have to cope with more surprises and threats then large drones and airplanes. An airport is designed to be a “simple” environment, and flight at altitude happens in relatively wide-open space. In such a simple environment, automation can succeed today, even though automated sense-and-avoid capabilities are relatively primitive. Small drones do and will be taking off and landing in improvised sites, near buildings, near people. Their missions will involve proximity to structures. Automated systems today do not offer the situational awareness required to mitigate the risks involved in these types of missions. Until they do, humans have a place in the control loop for small drones, albeit more as the “eyes” of the system versus it’s “hands”: leaving rate control to automation, but providing judgement and directional input for obstacle avoidance.
Reliability, as a source of Safety, is also rooted in the handling of the drone behind flights: as with traditional aircraft, regular maintenance and pre- and post-flight checklists will increase reliability. With higher degrees of automation, the actual task of flying is becoming somewhat de-skilled, and open to many people. The surrounding procedures on the ground will, in comparison, gain in importance for reliability.
Operating Range is Limited
Drones receive control input via radio link, and they send back telemetry data and sensor data, such as video, via other radio links. Current radio systems are robust insofar as they frequency hopping to allow parallel operation of many craft, but they are not robust enough to deal with the unpredictable, hostile radio frequency environment that exists in populated areas, where all kinds of radio sources crowd the spectrum, and where the sheer power of some of these sources can scramble other systems in seemingly unrelated frequency ranges. in other words, a drone using a conventional radio system can easily find itself in a spot in a city where it’s radio suddenly does not work.
Operating range is also limited by batteries. Fixed-wing drones can fly up to a few hours on batteries. Multicopters, not relying on lift from wings, offer shorter durations. Even though battery technology and drive system efficiencies are improving rapidly, batteries remain a limiting factor.
So Many Opportunities !
The challenges add up. It is not likely that the upcoming FAA regulations simply open up the use of small drones in cities. It is, after all, the FAA’s mandate to keep things from falling on people’s heads. The current set of technologies and operating models for small drones don’t offer much to support this goal. The FAA already has, in principle, green-lighted the use of drones by law enforcement agencies. The FAA will likely enable some business use cases where the operator is on site, doing line-of-sight flight. Such a set of regulations would be a great starting point, but let’s look beyond, and consider how one could enable a wider set of use cases that harness the opportunities of autonomous or semi-autonomous drones with an operating range of several miles, to do things like delivery and other truly remote operations.
Distributed, Redundant Infrastructure
Reliable radio connectivity is the foundation that enables other solutions. The urban RF environment is likely to become more hostile in the future, rather then less. A radio connection is generally the more reliable the shorter the distance bridged is. To achieve short connection distances, without an operator who physically follows a drone around, a network must be used. Mobile phones have been using cellular networks for a long time. Mobile phones are tracked by multiple towers and connections are seamlessly handed off between towers. Mobile phone infrastructure is indeed an opportunity on which drone radio control could piggyback, replacing the need to maintain a direct point-to-point connection between the drone’s and the operator’s radio. A cellular network poses a latency challenge for direct flight control. This challenge can be addressed by optimizing the pilot’s control mechanism and the drone’s autopilot, so successful flight operation does not rely on extremely time-critical input from the operator.
Ad-Hoc Control Hand-Off
Automatic flight operation is a great promise. It is cost-efficient, and it can be safer then human operation, as long as the operating situation is within the system’s capabilities. But risky conditions can arise: malfunctioning motors and electronics, GPS outages, radio outages, unpredictable wind, crowded airspace, and unforeseen ground obstacles. Especially when landing, the nearby presence of people, buildings and property presents a challenge that can hardly be handled by a fully automatic system.
A drone that encounters a risky condition should be able to quickly get a human operator into the control loop, who can use human judgment and training to operate the craft safely. Such a system includes an autopilot that can wait for human input, and operate in a predictable, safety-focused way in the meantime. A typical flight could consist of a monitored, automatic take-off, an automatic flight, and a human operated of -supervised landing. Different types of standard procedures would be used for landing in designated, controlled landing areas (Drone-ports), and landings “in the field”, in somebody’s driveway for a delivery, for example.
Away from he craft itself, the control infrastructure should be able to hand control from one operator to another, based on factors like connection reliability and availability. This involves a network that can monitor it’s quality of service, and it involves a mechanism that allows ground stations (the site and system from where the drone is operated) to reliably synchronize state before a hand-off.
Predictable Sense and Avoid
A big current research & development topic for drones is Sense and Avoid. Sense and Avoid is the craft’s capability to identify other craft in the airspace, and to avoid collisions with them. Large drones today, on test flights in commercial air space, are required to be escorted by manned aircraft, to serve as eyes to provide the “sensing” capability. Most aircraft today use a standardized system that lets other aircraft and air traffic controllers know where they are in airspace. Air traffic controllers provide from-the-ground coordination of air traffic near airports, telling airplane pilots know where to go to maintain safe distance from other planes.
It is unlikely that small drones in an urban environment will simply rely on conventional transponders and air traffic control. Some form of transponder system is likely to be used, so there can be a direct craft-to-craft exchange of positions. Autopilots will implement a set of rules for predictable and safe behaviors during encounters with other craft. Sensing systems like video cameras with computer vision capability and sonar can help provide the autopilot of a drone with a level of situational awareness for these situations.
Besides equipping drones with the useful, but always-incomplete capability to deal with ad-hoc close encounters with other craft, an additional level of operational safety can be achieved by predicting encounters, so they can be avoided in the first place.
With an infrastructure that provides cellular-like connectivity between drones and ground stations, the infrastructure can provide a service that monitors current and planned drone flights in a given area, and provides proactive flight path coordination. Such a service would communicate with the ground stations, where flight paths would either be modified automatically or with operator’s input, directing drones out of each other’s way.
When considering drone flight paths over populated areas, at not-very-high altitudes, it is also very important to consider what exactly lies below the flight path. In crisis situations, during a crash, during a non-scheduled landing, and even for a flight that goes as planned, the specific properties and structures that a drone operates above are important for safety and for privacy. The lower the craft’s altitude, the more important ground awareness becomes.
One opportunity here is a map lookup service. Via the ground station software, the drone’s autopilot can get realtime visibility of a ground map that carries flight-relevant data, representing elevations of structures, locations of hazards, and property mark-up. A public interface could allow commercial services and members of the general public to provide the required data. The markup could, for example, identify schools’ grounds as no-fly zones during school hours, identify emergency landing spaces in back yards that owners rent out for the purpose, identify locations, dimensions and approaches for delivery locations that property owners have authorized, identify the location of mobile landing stations where drones can get fresh batteries, and generally carry per-parcel property information, combined with flight rights and policies that are based on standards and captured based on the political will of the local municipality.
Another opportunity is an onboard capability for drones to detect people (and animals) nearby. This capability could be a combination sensors like an infrared camera, and a realtime analysis software that provides information to the autopilot. During a landing, even during an emergency landing with reduced power and navigation capabilities, the drone could then make autonomous triage decisions that will avoid harm to the persons in the risk area – putting human safety first, in the manner of Asimov’s first law of robotics.
The combination of automated realtime flight monitoring, policy-based location aware flight behaviors, and autonomous triage capability can bring predictability, graceful coordination and safety to urban drone operations.
Modular Operation and Maintenance
Drone pilots will require some sort of license, and maintenance will also involve formal qualifications. Some organizations will choose to own their own systems and train their own personnel to operate the systems. Since equipment and training will come at a cost, there is an opportunity for businesses to offer sUAV operation as a service. Such a service provider would maintain a range of flying platforms, employ pilots, maintenance personnel, and a dispatch or scheduling service, and operate ground stations around the city, out of fixed locations and possibly out of trucks, based on demand. The drones operated by the service would then perform deliveries and other missions for customers who can remain focused on their core, non-drone business: baking pizza, ensuring the safety of infrastructure, making medications available to patients and so on. The actual fragmentation of businesses can play out in many ways, with infrastructure providers, maintenance providers and piloting services as components, for example. Organizations of large-enough size might in-source maintenance.
Modular Payloads & System Components
Besides the modular approach to business, the technical operation can also be structured in a modular way. Standards should emerge for quickly exchanging types of payload, both in terms of physically attaching them to craft, and in terms of providing relevant telemetry data and feedback: the same sUAV airframe might carry, for example, a “claw” holding a delivery package, a climate-controlled container with it’s own power source, or a smart camera that can find cracks in concrete on it’s own. Reconfiguration would happen on the spot, with little work involved, so these different payloads would be used on the same day.
Interchangeable payloads could also enable “Secondary payloads”, where an operator rents out some of the payload capacity to 3rd parties. For example, a delivery drone could carry a small suite of meteorological sensors, quietly gathering weather data during flights, on the side.
Standardized rails and attachment points on airframes would enable quick swapping of drive system components and interoperability. Firmware in components like motors, autopilots, sensors and motor regulators could communicate via a bus to allow automatic configuration, and to proactively reject the combination of incompatible components. Even if these types of standards are manufacturer-based and end up competing with each other, any bridges that are built, between systems that otherwise exist in isolation of each other, will benefit the ecosystem in it’s entirety.
When one sees a drone in the sky nearby, one should be able to find out where it comes from, where it is going, what it is doing and who is operating it. This transparency should be available both to the institutions that monitor and ensure the safety of operations, and for members of the general public.
From an institutional perspective, drone operation should be transparent so individual operator’s actual performance is a matter of record, encouraging a high standard of responsible, accountable behavior, with options for sanctions and rewards.
From the perspective of the public, always being able to find out what a specific drone is doing takes away the threat of anonymity, and provides a path for feedback on operations: “putting a face on the drone.”
Cars have license plates, and businesses operate while displaying their name. These mechanisms of accountability should apply to drone operations, mitigating potential risk through a degree of transparency. A drone could carry a transponder broadcasting it’s identification. The identification could be clear and open, or encrypted in some fashion so it is easily made available to the right parties, but not totally open – like a car’s license plate. The transmitted signal could be transmitted as a series of optical pulses, and read by pointing a smartphone at the drone from the ground, listing the operator and mission, and providing feedback mechanisms along the lines of existing business performance rating services. Providing such an identifier could be mandatory, or it could be part of best practices that are encouraged by professional organizations and rewarded by insurers of drone operations.
For drones operating near people, transparency can be taken a step further: people could talk to the operator. Technically, it is not too hard to envision a solution that allows use of a phone app or a standard number to contact a drone operator. As the caller receives information about the operator along with the call, the operator would also receive information about the caller and about the caller’s location relative to the drone. The operator’s ground station could integrate this information into telepresence displays, video feeds, and close range ritual awareness displays. Using positioning information, such a communication channel could be provisioned automatically, and used routinely. In delivery missions, the recipient of a delivery could for example coordinate the landing or hand-off with the operator, as a matter of common courtesy, or just say hello.
What about privacy for the suas? Not the popular question but it needs to be considered I think. Flights over private property. Consider a private domain for example? All of this could become hyper competitive with say Fedex monitoring very comprehensive competitive data.
Ground Control Station Design
Just as there are many different airframe design, there are many different types of ground control stations. Some sUAS are flown with “gamepad” like joystick controllers, others with traditional gimbal-based hobby RC transmitters. Some combine such manual control with software running on laptops that have radios as peripherals, and some rely entirely on software. The types of ground control stations for large UAS also vary greatly. It is intuitively obvious that some level of unification and standardization would benefit operators by making skills transferrable between systems.
Given the proven superiority of autopilots compared to rate control input by human pilots, such a unified solute would probably be a relatively strongly automated one, decoupling input from flight behavior. The actual solution, though, is a matter of careful human factors engineering and design, much of which remains to be tackled.
A mature ground control station, in combination with standardized, compatible sensors on the craft, could automatically handling telemetry data and data logs from such sensors, routing it to remote services for processing and storage without burdening the human operator.
Beyond optimizing the operator’s experience, ground control station software could also include standard components that create accountability, for example a standard way to capture time-stamped telemetry data in a tamper-proof way as reference information for insurance cases and safety reviews.
In Summary: A Matter of Systems
When looking at drones, we are looking at systems, as reflected in the professional terms UAS and sUAS. These systems, as they exist today, and as they are currently being developed, work well in isolation, but do not mesh well with humans and with the density and structures that humans surround themselves with. This is a general challenge in robotics, and in drone operations a.k.a. “aerial robotics” where a malfunction means that something falls on somebody’s head, the challenge is strongest.
The challenge can be addressed by building bridges between the aerial system’s capabilities and the requirements and expectations of humans and our surroundings. These bridges are additional systems, with input and output from all of us, and with input and output to drones and their operators. Some of the systems are technical, others are in the realms of behavior and expectation setting, and most of them are combinations of these factors. Once we get on the way working on solutions for these bridge systems, appropriate provisions in the legal realm – the legislation governing drone use – can be put into place.
The legal system will be the ultimate enabling framework, just as it is the ultimate showstopper until then. Legal provisions can only be put into place once the community of drone users demonstrates that it is up addressing the structural bridge building challenges first.
Luckily, drones people bring together unique software and hardware skills, coming from garages, hacker spaces and also from giant old aviation companies. We have access to cheap networking, sophisticated, low cost computing and communications infrastructure, and to an app-using, technically literate public. This public, is also growing a great amount of sensibility to privacy, safety and security matters, and will not be taken for fools by people showing them new toys, so we better take their concerns seriously.