Autonomy is a central pillar of the fourth industrial revolution. However, fully autonomous vehicles remain an area of development. Autonomous vehicles operate in a high-risk environment — if something goes wrong, lives could be at risk. To function effectively, the system needs human-like reasoning and decision-making capabilities.
There is a growing interest in ontology-based road environment modelling within the Intelligent Transportation Systems community. Ontologies offer many benefits for the development of autonomous vehicles; they have been used in representing road intersections, creating driving decision-making systems, and for providing meaning and context, both implicit and explicit, to sensor data, for example the relationships between perceived road objects.
Various levels of ontologies can be integrated using semantic reasoning (rules) to create a “common high-level ontology resulting in a unified knowledge base”.[1] Using an ontological approach allows the road environment to be modelled and road situations to be reasoned over.
The necessary knowledge for decision-making and vehicle motion planning can be deduced from this unified knowledge base. Ontologies provide a flexible architecture, as various lower-level maps can be used to feed information to the high-level ontology.
To complement this article, we have also created an animated video, ‘Reasoning on The Road’, which illustrates the capabilities of RDFox with broad strokes. Continue reading as we explore its themes below.
This article is inspired by award winning research undertaken by Haonan Qiu, and colleagues at the BMW Group and the University of Ulm.[2] The views expressed herein are those of the author.
For the full white Paper click here.
Researchers at BMW have suggested a novel approach for the progression of autonomous vehicles, which uses RDFox, a semantic reasoning engine, in its knowledge-spatial architecture.
Semantic reasoning is the “ability to make logical deductions from the information that is explicitly available”. This approach models the road environment, provides a dynamic update mechanism, and has semantic reasoning at its core. Reasoning allows information to be aggregated and integrated leading to knowledge of the environment and effective decision-making, e.g. changing lanes at an appropriate time so the car leaves the motorway.
RDFox provides a comprehensive software component that can be used to process and integrate data, in real-time, with constrained processing power. RDFox can deal with large, constant streams of data due to its incremental processing capabilities. This means that as new data comes in, only the calculations which are affected are updated.
Using rules RDFox can make real-time decisions resulting in motion control. By harnessing the power of semantic reasoning, logic can be integrated into the software. This means the rules and best practices associated with driving can be embedded within the system.
There are numerous benefits associated with the advent of autonomous vehicles, including increasing safety on the roads, real-time route optimisation, increased lane capacity, reduced energy consumption and time efficiency for the users. RDFox is suitable for this application because, like a (competent) human driver, it is a sophisticated reasoner, which can scale, and is optimised for fast, incremental reasoning. Speed of thought is critical in an autonomous vehicle because a difference measured in milliseconds could be an incident on the motorway.
In reality, the safe operating parameters of autonomous vehicles will determine whether and how they operate. This will determine, for example, maximum speeds, minimum distances between vehicles, and the minimum conditions that must be satisfied before manoeuvres are permitted, such as lane changes. The faster and more accurate the reasoning, the larger and more flexible the safe operating window, and the greater the benefits of autonomous transportation.
The in-memory solution and economical storage that RDFox offers also bolster its viability, as it provides the ability to hold information on-board the vehicle effectively, whilst also integrating various datatypes.
Whilst RDFox has all of the capabilities synonymous with graph databases, its incremental semantic reasoning capability make it uniquely suitable to meet the demands of autonomous driving.
Semantic reasoning is the key to giving autonomous systems salient information and decision-making capabilities whilst reducing their dependency on big data feeds and huge processing power.
The contribution made by researchers at BMW demonstrates improvements in various areas, including: the ability to embed domain knowledge on safe driving practices; data volume issues; monitoring of sensors; and explainability.
Rules and common sense help humans plan and infer responsible driving behaviour for situations on the road. A driver might slow down when driving through a residential neighbourhood on a sunny weekend because they expect more pedestrians and children playing with balls near the road. Semantic reasoning makes it possible to model these implicit concepts so that the appropriate decisions can be applied at scale without having to explicitly state each variation individually.
An autonomous vehicle must also be able to anticipate potential outcomes and respond in real-time. For example, if someone walks out into the road unexpectedly, the car must respond. By modelling domain expertise on safe driving practices into the system we can improve the safety element of autonomous vehicles.
Furthermore, of the mistakes made by humans driving, 98.3% are due to inattention or ineffective driving. When such errors are combined with a deliberate reduction of the safety margin, through excessive speed or driving too close to the vehicle in front, the result is a serious and potentially fatal incident. By removing the human element that is prone to lethally combining these errors and violations, there will likely be a reduction in road incidents, which result in more than 1.2 million deaths and 50 million injuries a year, globally. While you can never eliminate human error (after all, software must be written and sensors designed), by shifting the decision-making moment from a time-critical situation to that of a lab, the likelihood of disaster is decreased to be comparable with other manufacturing tolerances, such as a break failing.
The volume and computation pressure of data processing is a huge concern for those developing methods using HD Maps. Qiu and her colleagues have found that using RDFox, they can retain all relevant information, for as long as is required, in in-memory data stores.
By using spatial reasoning, a ‘spatial window’ can be created. This defines a range using a specific distance around the car that information will be stored for. It determines information to pre-load for areas soon to come into the spatial window, and once beyond the range, the information will be incrementally deleted from the system.
This limits the amount of data stored and processed at any given time, greatly improving processing efficiency. The map is incrementally updated whilst progressing along a route. For the first time, it’s possible to do all this on board directly.
At present, the various maps accessible by manufacturers are in incompatible formats due to the lack of standardisation. RDFox is well suited to solving this issue due to its ability to integrate data from various locations and formats, into a unified knowledge graph which can then be reasoned over.
By using RDFox the manufacturer can integrate the existing map knowledge base into their system, and build upon this, rather than starting from scratch. This represents a serious breakthrough in map development.
Rules offer the solution to safety concerns over sensors. When information is incomplete because sensors are erroneous or missing, rules can be used to determine decision-making, and can rely on the wealth of other information incorporated into the system. For example, if a sensor fails, then the car will be able to aggregate the information from other sensors to ensure that safe driving continues.
Additionally, when sensor data is inconsistent, rules can be used to flag this to the car in real-time. For example, rules can prevent false emergency breaks by validating the consistency of the derived knowledge from the fused sensor and map data ontologies.
As safety is a huge concern for integrating autonomous vehicles into the mainstream road systems, being able to explain a vehicle’s actions is of interest to the developers. At present, machine learning techniques are unable to provide explainability. Should an incident occur, or should the user be interested in why the autonomous vehicle took a specific path, they can’t get an explanation from the system.
With RDFox, the system can provide explainability. This would allow for accountability, and also provides transparent information to the Manufacturer and investigators. Just as the black box, together with a learning culture, has helped transform safety in flight, transparency in autonomous vehicle control systems will help achieve the same for road transport. It is, in fact, astonishing to think that aviation is orders of magnitude safer than driving.
There is a growing body of research which uses ontologies and semantic reasoning in autonomous vehicles and other autonomous applications. The novel approach put forward in the research conducted by Haonan Qiu, colleagues at BMW and the University of Ulm, explored the use of ontological approaches for dynamic map processing, and reasoning for decision-making and motion control.
The inclusion of RDFox in environmental modelling and knowledge-spatial architecture for the facilitation of autonomous vehicle decision-making opens new possibilities for applying human-like reasoning to this field. The choice of RDFox was integral to this research, due to its unique ability to perform with speed and correctness in the acutely constrained and time-sensitive cockpit of the autonomous vehicle.
The results are encouraging, and suggest that the ontological approach using semantic reasoning could contribute to the strive for full autonomy in vehicles and, of course, other areas of robotics.
The video featured in this article is the property of Oxford Semantic Technologies.
The team behind Oxford Semantic Technologies started working on RDFox in 2011 at the Computer Science Department of the University of Oxford with the conviction that flexible and high-performance reasoning was a possibility for data-intensive applications without jeopardising the correctness of the results. RDFox is the first market-ready knowledge graph designed from the ground up with reasoning in mind. Oxford Semantic Technologies is a spin-out of the University of Oxford and is backed by leading investors including Samsung Venture Investment Corporation (SVIC), Oxford Sciences Enterprises (OSE) and Oxford University Innovation (OUI).
.png)
